Ansuz

« tikzpeopleで新しいキャラ | Home | Ad stramineum hominem »

Live video streaming with free tools (A Gruesome Orgy Of Yak Shaving!)

I recently set up my own server for streaming live video over the Web. That was complicated and difficult, and although I found some useful resources on the Web, none of them were complete and many of them were out of date. So here are my notes on what worked, both for my own reference and in the hope they may be useful to others.

Technologies covered here, more or less in order of signal flow from the origin of the video content to the viewer's screen: OBS Studio for generating the video signal; FFmpeg for encoding it; OpenSsh for securely transmitting the signal to the "repeater" machine that serves it to the users; HLS stream formatting; the Apache Web server; and the VideoJS HLS Javascript-based video player for displaying the stream in the viewer's browser.

These notes are current as of late January 2018. Pay attention to the date on these notes or any other information sources about this topic, because the relevant technologies change over time and there's a lot of outdated information out there.

Choosing the technology

I want to be able to create a video signal in real time - pointing a Web camera at myself as I talk, using a microphone to capture my voice, also capturing live screen imagery from my computer as I use it and maybe also playing back recorded audio and video files - and broadcast that signal in real time much like a television broadcast, to viewers who are using Web browsers. It is necessary that it work with my computers, both my Slackware Linux desktop machine where I do the recording, and the Linux-based Web servers I use. I want it to work for viewers who have any popular Web browser and no special configuration such as plug-ins. I am willing to buy commercial products to help with this, but only on a perpetual-license basis (no "subscriptions"), only if the price is reasonable (let's say, semi-arbitrarily, Cad$300 maximum outlay for all commercial products needed), and if I pay money for something, then it must really work - no excuses.

Last time I experimented with video streaming was a few years ago and at that time I used a Flash-based system. It worked, and it was based on free software, but I got a lot of negative feedback from viewers who claimed they were "unable" (which, it quickly turned out, really meant unwilling - this was not a problem that could be solved with any technical fix) to use a Flash client because of the technology having originated in a commercial product. Since that time, Flash's commercial backers have declared their own implementation to be obsolete, the free software implementations have fallen by the wayside too, and it seems pretty clear that Flash is not a realistic option in 2018. That's a shame, because even though there are now a couple of competing standards for displaying live streaming video in Web browsers, and they're old enough that they ought to be usable, deployment and support is still uneven.

There are several layers of protocols and formats involved in video streaming. There are "codecs" (which are fundamentally compression schemes), for both audio and video; examples include H.264 for video and AAC for audio. There are "container formats" which mark up data from the codecs into files that can be transferred between pieces of software. And in the case of live streaming, there are "streaming formats" which often partake of both container-like packaging and network protocol-like issues to describe the way that the data in containers gets to the viewer. There are usually many-to-many relations among these levels; a given streaming format can work with more than one container format, and a given container format can work with more than one codec. However, the separation among the layers is usually not very clean, with codec and container issues still being relevant when you work on the streaming layer and not all combinations being possible.

The two major streaming formats at the present time are HLS and DASH. HLS originated with Apple. It is the only streaming format supported natively by iOS devices, and it is also supported natively by Safari on Macs. However, it can also work on nearly all major desktop browsers despite not being native, by including a piece of player software written in Javascript on the Web page where the stream will display. This is the streaming format I ended up deploying for my system, described in these notes.

The underlying technology for HLS works by splitting the ongoing live audio/video signal into many segments, each a standalone video file a few seconds long (using H.264 and AAC codecs, and the MPEG TS container format), which are served by a Web server. The Web server itself is just an ordinary Web server that serves files out of a directory on request. Some other piece of software has to update that directory every few seconds by writing new TS containers as new chunks of video become current, and possibly deleting the old ones. There's also an index file (in what seems to be an embraced-and-extended version of the WinAmp playlist file format, I'm not making this up) that gets updated with each new video segment. Clients view the stream by requesting new segments as they become current, and fresh copies of the index file, with some special handling to smoothly concatenate the signals from the successive segments and handle timing and buffering issues.

DASH is a very similar technology to HLS, based on many small container files that get updated over time and served through relatively straightforward HTTP, with a frequently-updated index file on the side. Minor technical differences include efforts to make the containers not be valid standalone files by themselves (so as to save the cost of repeatedly transmitting their headers), and to separate each stream (audio, video, and possibly multiple different formats of each) into its own sequence of container files. Having multiple streams in separate container-sequences makes it easier and cheaper for clients to "adaptively" pick out only the streams they want, but also creates problems keeping them synchronized which, in my own testing, seem to be nearly insurmountable for live streams where the sync is a little iffy to begin with.

DASH is favoured, and natively supported, by many major browsers notably including Firefox and Chrome. It is not natively supported by Apple products, though as in the other direction it appears possible that in most if not all cases it's possible to use Javascript to play it even in browsers that lack native support. DASH is in principle a more open standard, more able to use a variety of codecs and container formats and with more preference given to codecs that are less patent-encumbered (for instance, audio for DASH is most often encoded with Vorbis). However, I have so far not been able to get it to really work properly - HLS works for me - and I'm quite put off by the fact that nearly all resources I can find on DASH treat its "adaptive" streaming of static files as the main purpose of using it. "Adaptive" streaming means the capability to send streams at different bit rates according to the size of the screen on a mobile device - which is not worthless, but is very low on my priority list. Doing live streaming, my main interest here, seems to be an intended feature of DASH too, but is not well-covered in online resources which all seem to start by assuming you have a static file. (In which case, why not just put your static file on a Web server, embed it with HTML5, and use HTTP range requests, which work for everyone, instead of a "streaming format" at all?)

Another possibility for doing live streaming over HTTP is to make the live stream look like a static file and let the browser use those range requests. This approach requires special support on the server side to present to clients something that looks like a container file of unlimited length, most likely in the WebM container format. Clients designed to read static video files can usually handle this form of streaming very well with little or no Javascript player trickery. But it requires a server that will correctly receive the live stream via whatever protocol, buffer it, and then translate client requests into chunks of the buffer appropriately. Apache doesn't seem able to do it except possibly in a limited way with some experimental modules that aren't easy to use. Nginx did it last time I explored these issues a few years ago, but I was only able to get it to work using Flash codecs and a Flash client.

There used to be a free software program called "ffserver" which would do infinite-file streaming with WebM, and that worked really well; but the FFmpeg team declared ffserver to be end-of-life in 2016 (not finally deleting it from their repository until late 2017), and they trotted out the old "you can't demand anything from volunteers" chestnut in response to complaints.

After spending a couple days trying, it appears to me that it is no longer realistically possible in 2018 to run the last version of ffserver, with its many dependencies downgraded as necessary to get it to run, in a way that will really work with all major browsers and will be sustainable. I got as far as having the "infinite WebM" streaming system working, but to support Apple products I still needed HLS on the side, ffserver's HLS didn't work, and if I'm implementing HLS anyway then it becomes relatively easy to add the Javascript to make the HLS work on non-Apple products, at which point "infinite WebM" is less important.

So, having chosen HLS as the basic streaming format for serving live video to viewers, there are a bunch of other issues to solve. I wanted to use OBS Studio for real-time generation of the video signal. It has a built-in copy of FFmpeg which it uses to encode the data, and that can be configured to send the stream off to a network connection but only via a limited selection of protocols. I ended up using another instance of FFmpeg on the server that answers Web requests (which I call the "repeater" host by analogy with radio communication) to receive the network signal and generate the directory full of constantly-updating video files. Apache Web server on that machine serves them out, as it would any arbitrary directory of files. None of the protocols offered by OBS Studio's captive FFmpeg offer any kind of security or authentication, so I had to tunnel that link through SSH. And alongside the video files in the Web server directory, I had to offer the Javascript-based player software for non-Apple browsers.

Below is a diagram of the entire system as I built it. Three hosts are involved: the "transmitter" where the video originates, the "repeater" which accepts client connections and serves out the current stream data, and the "receiver" where a viewer watches the signal. There can be an arbitrary number of receivers all connecting to the same repeater, up to bandwidth limits. It is also quite possible, and useful for testing, for two or all three of these instrumentalities to be hosted on the same computer, though in the final deployed system they would normally all be separated.

Now, I'll describe the steps I had to take to get this installed and working. I assume a reasonable understanding of Linux system administration, and access to suitable hosts for the transmitter and repeater. A significant amount of prerequisite software is needed but you probably already have it installed; I'll cover only the things I needed to install specifically for this project, more or less in chronological order. (That is to say, the order one would do these steps if starting with full knowledge of what depends on what - of course the order in which I actually performed the steps myself involved a lot of mistakes and backtracking and was not as described here.)

Installing x264

The video needs to be encoded with the H.264 codec, and FFmpeg desperately wants to use hardware acceleration for that, especially on systems which are incapable of hardware acceleration. The FFmpeg developers also have a thing about the GPL. As a result, the software for encoding H.264 that actually works, is not included in FFmpeg, and its bindings are not compiled into FFmpeg by default; you must install it manually. This is the "x264" package. It needs to be installed on both the transmitter and the repeater hosts.

As of January 25, 2018, the current Git version from http://git.videolan.org/git/x264.git works. (EDIT: As of December 2019 that is a dead link; it appears to have moved to https://code.videolan.org/videolan/x264 but I haven't tested the current version there.) It needs to be configured with --enable-shared in order to build libraries that will work with FFmpeg.

./configure --prefix=/usr/local --libdir=/usr/local/lib64 \
  --enable-shared

I used --prefix and --libdir as shown to make sure that binaries and libraries went in the proper places for my Slackware system, but --prefix is probably unnecessary (this should be the default value anyway) and --libdir is probably unnecessary for non-Slackware distributions.

Installing FFmpeg

Installing FFmpeg requires some care. It's a big, complicated program with many dependencies, it needs specific compile-time configuration to work in this application, and other software often depends on it (which makes configuration changes dangerous). My Slackware system came with FFmpeg 3.4.1 installed in prefix /usr, and a lot of my other software (notably, my Web browsers) depends on it and on its being in that location. But the transmitter side of the streaming system needs a different FFmpeg installation. What worked for me, though it's quite ugly, was this:

• Remove FFmpeg 3.4.1 from /usr using the Slackware package manager.
• Compile my own FFmpeg from a Git checkout and install it in /usr/local .
• Compile OBS Studio, having it build against the Git FFmpeg which at this point is the only one installed.
• Reinstall Slackware's FFmpeg 3.4.1 in /usr with the package manager.

After doing this, distribution-supplied things like Web browsers work; the captive FFmpeg in OBS Studio works; and command-line FFmpeg complains about library mismatches, but it works. Presumably, it would be better in some sense to compile and install just one custom FFmpeg that will work for streaming transmission and also support all the other things that depend on the distribution's FFmpeg; but that seemed more difficult.

After removing the distribution's FFmpeg and checking out the source code from Git (https://git.ffmpeg.org/ffmpeg.git) I configured FFmpeg as follows:

./configure --prefix=/usr/local --libdir=/usr/local/lib64 --enable-gpl \
  --enable-libx264 --enable-libvpx --enable-libopus --enable-shared

The --prefix and --libdir options direct it to install in /usr/local, with Slackware's convention for 64-bit library locations.

The --enable-gpl option is needed to say you're willing to accept the freely offered licensing of volunteer-contributed software, because FFmpeg thinks the GPL is "too restrictive" even though bindings for hardware that only works with proprietary drivers are turned on by default!

The --enable-libx264 option is the critical one, needed to allow FFmpeg to build against the x264 package you just installed (which is GPL, so --enable-gpl is needed for this option to be available).

I used --enable-libvpx and --enable-libopus to build in codecs needed for WebM when I was experimenting with WebM, but these are probably not necessary for the HLS installation I ended up using. Note that libopus requires installing another package first, before you build FFmpeg.

The --enable-shared option builds shared libraries from FFmpeg, which are needed on the transmitter host for the OBS Studio build to create its captive version. Note that if you build without this config option, then reconfigure with it and try again, the build will fail, and "make clean" doesn't seem to resolve that. A stronger form like "make distclean" might work, but I ended up blowing the whole tree away and starting over with a fresh Git checkout.

Note that the FFmpeg "configure" script is not a standard one from Autotools and it runs silently for a long time before producing any output; be patient.

Installing OBS Studio

OBS Studio insists on having a 3D accelerated video driver, specifically "OpenGL 3.2" or higher. Every desktop machine has a hardware-accelerated video card, but the drivers for them are often proprietary or unsupported under Linux, and don't work, or don't work at anywhere near current version numbers. There are also software 3D libraries that are supposed to provide this kind of capability using CPU power instead of the video card. These often don't work either. You can supposedly find out what version of OpenGL you have by typing "glinfo", but on my machine that says I have "OpenGL 3.0" even though OBS Studio reports that it is finding OpenGL 3.3 and seems to be happy with that. I have no sensible explanation for this discrepancy, nor any useful advice on how to get OBS Studio to be satisfied regarding 3D acceleration on machines where it isn't.

As of late January 2018, the Git master version of OBS Studio does not work for me, notwithstanding that that bug report which seems to be the same problem is listed as "solved." I found version 21.0.1 (built from a distribution tarball) to work.

OBS must be built and installed after FFmpeg has installed with all needed codecs and shared libraries configured in, because OBS Studio builds its own copy of FFmpeg into itself instead of running the command-line ffmpeg program, and so at run time OBS seems to be limited to the FFmpeg codecs that were available at OBS build time. See comments above about the sequence for uninstalling and reinstalling different versions of FFmpeg. Also note that if you have run CMake while a configuration of FFmpeg was installed that did not work for the OBS Studio build, then you change the FFmpeg installation, and you re-run CMake in the OBS Studio tree to try again, it may pick up "cached" incorrect configuration information from the previous run. You must delete the CMakeCache.txt file before rerunning CMake if you want to be sure it will see the current state of FFmpeg installation.

OBS Studio only needs to be installed on the transmitter, not on the repeater, and it will probably not be practical to build and install it on the repeater if that's a headless machine, given the heavy requirements for graphics support.

Web server and Javascript player

Serving HLS video just requires a directory in which you can throw a bunch of files and have them appear at some public-facing URL. I was able to use my existing Apache Web server configuration without any special configuration, just by creating a directory in an appropriate place. Serving the files over HTTPS rather than HTTP works if your server does that, and may be a good idea if you want to be able to embed your videos in a Web site that is served over HTTPS (otherwise you run the risk of the broken-lock "mixed content" warning in the browser).

The Javascript player needed for non-Apple clients to receive HLS streams is available from http://videojs.github.io/videojs-contrib-hls/ . The space of Javascript video players is highly competitive and mostly occupied by commercial products, many of which are expensive and subscription-licensed. The videojs-contrib-hls player I point to seems to be one of the best free ones, where best is determined primarily by "does it actually work at all?". Paying for a commercial product is no guarantee of a "yes" to that question.

Deploying the player requires two Javascript files, video.js and videojs-contrib-hls.js, and a CSS file, video-js.css. These three files seem easiest to obtain by viewing the source of the player's demo page and downloading the URLs it links to. In particular, note that at least one of the filenames in the example HTML code shown on the demo page, does not match the filename the demo page actually uses for its own copy.

These files total about 1.4M, which is outrageous, but bear in mind that viewers will likely be downloading much more than that from the server anyway if they watch streaming video for any length of time.

It probably makes most sense to dump all the relevant files in the same directory where FFmpeg will be writing the HLS file collection, though other configurations are possible. There also needs to be an HTML Web page (index.html would be a sensible name for it) that points to all these things. Here's a minimal outline for it.

<html>
  <head><title>Live video</title>
  <link href="video-js.css" rel="stylesheet">
</head>
  <body>
    <video id="videoPlayer" controls class="video-js vjs-default-skin" width="640" height="360">
      <source src="live.m3u8" type="application/x-mpegURL" />
    </video>
    <script src="video.js"></script>
    <script src="videojs-contrib-hls.js"></script>
    <script>
      var player = videojs('videoPlayer');
      player.play();
    </script>
  </body>
</html>

SSH tunnel

FFmpeg's network connections are unencrypted and unauthenticated, which makes it unwise to use them over the public Internet; and the limited configurability of the FFmpeg captive inside OBS Studio makes it difficult to replace them. What I do is use an SSH tunnel. When I run ssh on the transmitter host to connect to the repeater host (which I'd be doing anyway just to start up the software on that side) I specify -L 11069:localhost:11069 , where 11069 is a randomly chosen port number. Then as long as I have the ssh connection up, any connections coming into port 11069 on the transmitter host will be forwarded through the encrypted connection and applied to port 11069 on the repeater host. The transmitting software connects to port 11069 on the transmitter host instead of going directly through the Net to the repeater, and by so doing it's really talking to the repeater software on the repeater host.

Using the same port number at both ends makes it a little easier to test with transmitter and repeater on the same host, because I can just omit creating the tunnel, run both sets of software on the same host, and they will see each other with no changes from the eventual separate-host configuration.

There is still a potential small exposure in that someone could connect directly to port 11069 on either host and have access to some part of the uplink of my system. Since the repeater software only accepts one connection per invocation, this is not a huge issue; it is only exposed for the brief time between when I tell it to listen and when I connect to it with my transmitter - like leaving my door unlocked for the few seconds between when I insert the key and when I walk through it. Also, I don't really use 11069 as my port number but choose new random ones periodically. And since all the connections to this port number at both ends are local to single machines, I can block all external connections to the chosen port number with firewall rules.

In principle it would seem that it might be better to use pipes to send the data through ssh to stdin of some command, without involving TCP ports, but that's a problem because of the limitations on where OBS Studio's captive FFmpeg can actually send things. It wants to talk to a TCP port and doesn't offer very many other options; in particular, it won't let me just type in an arbitrary shell command to be the destination of the stream. There might also be issues of separating out the video data from the significant amount of status and command information that FFmpeg wants to exchange via stdio.

HLS packaging

The program on the repeater host that receives the stream from the transmitter and maintains the HLS file collection, needs to be running before the transmitter starts sending data. Here's a recommended invocation, to be run in the Web-served directory on the repeater host, probably from within a tunnel-enabled ssh session.

ffmpeg -nostdin -f matroska -strict -2 \
  -i tcp://localhost:11069?listen -f hls -c copy \
  -hls_flags delete_segments -hls_list_size 30 -hls_time 2 live.m3u8

Note that in this and all other ffmpeg command lines, the order of options is critical. Specifying things like formats in the wrong place will apply them to other files or streams than the ones you intend.

The -nostdin option tells FFmpeg not to complain if it's put in the background. Putting it in the background may be convenient in some scripted situations, and even if you plan to leave it in the foreground, this option will make it less fragile to accidental keystrokes in the window where it's running.

The -f matroska option specifies that the Mastroka container format will be coming in over the wire. This must match the setting used at the transmitter end; I recommend Mastoska as a container that (unlike several other popular ones) will work in the context of an infinite stream where the receiver cannot seek to the end to read metadata placed there.

The -strict -2 option tells FFmpeg not to complain about "experimental" codecs. It's not clear whether the codecs used in this particular case really require this option, but it is at least harmless and not having it caused me a lot of trouble in some of my experiments leading up to the present system.

The -i tcp://localhost:11069?listen option specifies the port number on which to listen for the transmitter. It should be localhost even if transmitter is remote. The SSH tunnel, if used, should point here.

The -f hls -c copy options tell ffmpeg to generate and maintain the HLS structure of files that can be served by a Web server in real time; it should not re-encode but just use the encoding that comes in on the wire, repackaged from Matroska into HLS containers. The incoming encoding had better be appropriate for HLS, or either FFmpeg or the video player will fail.

The -hls_flags delete_segments option says to garbage-collect old segments when they fall out of the current transmission window, set below to one minute.

The -hls_list_size 30 options says to store 30 segments (one minute, with nominal 2-second segments) rather than the default 5. This should make pausing and very brief replays a little more possible for the viewer. It may increase latency a little depending on the player's buffering behaviour.

The -hls_time 2 says to use 2-second segments, which is the default but specified explicitly here to reduce the possibility for mistakes. It ought to match related settings on the transmitter end, though I have noticed that in practice most of my segments are 2 seconds but they occasionally vary in size to as little as 1 second or as much as 3, and the system still seems to work. Inconsistent segment size seems to be less of a problem for HLS than it was for DASH when I was experimenting with DASH.

The page live.m3u8 is where the HLS files will be written. Specify the name of the playlist file, and the other related files will be given names derived from it in the same directory. It ought to be somewhere that the Web server will serve.

CPU consumption for this command is negligible, less than 1% on my desktop; that is by virtue of the -c copy which means it's not reencoding, only repackaging the encoded stream. If you must reencode on the repeater by using something other than -c copy, CPU consumption may be much higher. However, re-encoding on the repeater may be a useful thing to do nonetheless if you happen to have more CPU cycles available there than on the transmitter; you could send from transmitter to repeater in some format that is CPU-cheap and not too lossy, and then let the repeater do the work of putting it into a more efficient or more compatible format for the viewers. In principle, one could even introduce another machine, separate from the transmitter and the repeater, to do re-encoding from the transmitter's most convenient format to what's needed on the repeater.

Transmitting from a static Matroska file

It's useful to debug smaller sections of the system before trying to run the whole thing at once, because when something breaks it's better to have as few parts not already known to work as possible. So here's an example of a command to transmit an existing recorded video file in Matroska format up to the repeater without involving OBS Studio. When running this, the HLS packaging command above should already be running and waiting for a connection on the repeater machine, which could either be the same host as the transmitter or a remote host with an SSH tunnel.

ffmpeg -re -i kemono_friends_01.mkv \
  -vf framerate=fps=15 -map 0:0 -map 0:1 -f matroska -strict -2 \
  -c:a aac -b:a 128k -ar 44100 -c:v libx264 -s 640x360 -b:v 800k -g 30 \
  tcp://localhost:11069

The -re option tells FFmpeg to transmit only at realtime speed; otherwise it will run as fast as it can, which is probably a lot faster than realtime. Uplinking faster than realtime speed doesn't mean the video will play faster, but it may mean that the player occasionally skips far ahead or stops, as segments needed for realtime playback are deleted or replaced.

The -i kemono_friends_01.mkv option names the file to transmit. It's not necessary to specify the codecs and other details of the input file, as long as they are ones that this build of FFmpeg can handle; it will auto-detect them.

Specifying -vf framerate=fps=15 runs a video filter to change the frame rate to 15 frames per second by adding or dropping frames. I'm standardizing on 15 fps for my live stream, though other options are possible; other parts of the system will need their settings changed to account for this one if you try to use some other rate. Unfortunately, in a live setting with OBS Studio as the source it's not possible to get it to stick really consistently to any specific frame rate, and the captive FFmpeg doesn't seem able to invoke video filters like this.

The particular file I was using for my tests actually contains three streams or tracks multiplexed into in the Matroska container: one video, one audio, and one containing English-language subtitles for the Japanese-language dialogue (respectively streams 0:0, 0:1, and 0:2 in FFmpeg's notation). The options -map 0:0 -map 0:1 tell the transmitting FFmpeg to only transmit the video and audio, not the subtitles. It may actually be possible to send soft-subs like these all the way up the pipe to the browser; during my DASH experiments I found it was doing that by default, putting the Matroska subtitle track in a separate sequence of container files. And some player software, maybe even including the package I'm using, seems able to read and display subtitles in some format (I'm not sure which) at the browser end. However, that's not something I expect to use in my actual deployment and it doesn't seem worth spending a lot of time on just for testing.

Next come options for the network transmitting side:

Using -f matroska specifies the container format to use on the wire; note this must match the similar option on the repeater.

As above, I use -strict -2 to reduce complaints about "experimental" codecs, even if it might not really be needed in this particular case.

The options -c:a aac -b:a 128k -ar 44100 specify the AAC audio codec (which is important for compatibility with HLS-reading browsers), bit rate 128k, sample rate 44100. This bit rate is a little on the high side for live streaming but I anticipate streaming demos of my synthesizer products, for which good sound quality may be important.

Similarly, the -c:v libx264 -s 640x360 -b:v 800k options specify the H.264 codec, to be encoded by the software encoder (not one of the hardware-accelerated ones FFmpeg will try to choose by default). Note the frame size 640x360, which is a 16:9 aspect ratio to match the desktop monitors I'm using. Specifying a frame size this way implicitly invokes a software scaling filter, which increases the CPU demand and might cause quality problems on things like interlaced video content. Bit rate is 800k.

The -g 30 option specifies that there should be a key frame every 30 frames, which at 15 fps is every 2 seconds. This ought to line up with the segment size of 2 seconds specified on the repeater, because HLS can only begin a new segment on a keyframe. In practice, the synchronization to 2-second intervals doesn't actually seem to work very well. I'm not sure if that's because of extra keyframes triggered by fast-moving video content, or if this option is being ignored entirely, or what. However, the varying segment sizes don't seem to really cause any trouble (they are visible only when I manually peek into the .m3u8 file to see what's going on), at least not in HLS. With DASH, and its separate audio and video streams, I often had the sequence numbers for the two streams diverging from each other, and that may well have been why I couldn't get the player to work.

Finally, tcp://localhost:11069 is the output destination. This could be localhost for a repeater on the same machine; the name of a remote machine for INSECURE unencrypted transmission to a remote repeater host; or localhost again for the local side of an SSH-secured tunnel that will transmit the stream to a remote repeater.

This command in this configuration consumes about 110% of a CPU core on my desktop machine while it's running.

Transmitting from a Web cam

As another test, and something that might be useful in a simpler setting where the complexity of OBS Studio is not wanted, here's a command line for transmitting from a Web cam using FFmpeg to do the encoding.

ffmpeg -f v4l2 -i /dev/video0 -f alsa -i hw:4 \
  -vf framerate=fps=15 -f matroska -strict -2 \
  -c:a aac -b:a 128k -ar 44100 \
  -c:v libx264 -s 640x360 -b:v 800k -g 30 -pix_fmt yuv420p \
  tcp://localhost:11069

The options -f v4l2 -i /dev/video0 specify the "Video4Linux 2" hardware interface, which is the right one for most forms of video-capturing equipment, and the device name of my Web cam. This is for capturing the video stream as such, only; audio will be next.

The options -f alsa -i hw:4 specify reading the audio stream from the ALSA driver, device "hw:4", which is the microphone built into my Web cam.

Specifying -vf framerate=fps=15 resamples the frame rate to 15 fps, as in the other example; this is quite important with Web cam video where the native framerate is usually both slow and variable.

No mapping options are needed in this case; the default mapping just puts the two streams into the outgoing Matroska container as desired.

The options -f matroska -strict -2, as before, specify Mastroka-formatted output and no complaints about "experimental" codecs.

The audio encoding options -c:a aac -b:a 128k -ar 44100 are the same as in the previous example.

The video options are -c:v libx264 -s 640x360 -b:v 800k -g 30 -pix_fmt yuv420p . These are mostly the same as in the last example, but note that the aspect ratio is not really right for most Web cams. I stuck with it because I wanted to test the player at its normal size, even though it makes the video look distorted; but if using this for a real application I would want to do something else. Also note -pix_fmt yuv420p, which reencodes the "pixel format" to the one that Web browsers expect. Web cams often use something else by default, and that can lead to hard-to-debug failures on the player side if not fixed during encoding.

Finally, as in the previous example, tcp://localhost:11069 specifies the output destination, which should be the local or remote repeater.

This configuration consumes about 80% of a CPU core on my desktop machine.

Transmitting from OBS Studio

First of all, do not use the "stream" command in OBS Studio. It only supports a limited selection of protocols and formats and now that ffserver is a dead letter, I can't find software that handles those protocols well, with the desired output formats, to run on the repeater. Instead, tell OBS Studio to "record" and configure the recording feature to actually transmit the signal to the repeater instead of recording it.

Most settings are under Settings > Output > Recording, as shown.

Settings > Output > Output Mode = Advanced (needed to make other options appear)

Settings > Output > Recording > Type = Custom Output (FFmpeg)

Settings > Output > Recording > FFmpeg Output Type = Output to URL

Settings > Output > Recording > File path or URL = tcp://localhost:11069 (or other port or host for the repeater's input)

Settings > Output > Recording > Container Format = matroska

Settings > Output > Recording > Video Bitrate = 800 (kilobits per second, same as the command-line encoder examples)

Settings > Output > Recording > Keyframe interval (frames) = 30 (to match the 2-second HLS segment size)

Settings > Output > Recording > Rescale Output = off, but we must set the output size appropriately elsewhere

Settings > Output > Recording > Video Encoder = libx264 (Default Encoder)

Settings > Output > Recording > Video Encoder Settings (if any) = -pix_fmt yuv420p (probably will default correctly, but out of an abundance of caution; note that, sadly, OBS Studio will not accept arbitrary FFmpeg command-line syntax here or anywhere; it seems to be doing its own command-line-ish parsing of what is entered instead of really running the ffmpeg command-line utility.)

Settings > Output > Recording > Audio Bitrate = 128

Settings > Output > Recording > Audio Encoder = aac (Default Encoder)

Settings > Output > Recording > Audio Encoder Settings (if any) = -ar 44100

A couple more are on the "Audio" page. This screen shot also shows some of my local configuration not directly relevant to the live stream encoding setup.

Settings > Audio > Sample Rate = 44.1kHz

Settings > Audio > Channels = Stereo

And some more need to be set on the "Video" page.

Settings > Video > Output (Scaled) Resolution = 640x360 (should be changed as appropriate to match the aspect ratio of your canvas)

Settings > Video > Integer FPS Value (this is a drop-down box serving as a label for an input field; you may have to change it from "Common FPS Values")

Settings > Video > Integer FPS Value = 15

With these settings, OBS Studio works for me to transmit on my system to the repeater and player Web page configured as above.

UPDATE: a gotcha regarding sound inputs

Don't ever configure an "ALSA capture" sound input in OBS Studio to use the "default" input source if you're not certain that it can really read from there. Once this configuration has been made, OBS Studio can't capture audio from that input and also the GUI will hang if you try to undo the configuration change. I had to restore my config file in ~/.config/obs-studio/basic/scenes/full_vid.json from backups in order to get OBS Studio to work with my microphone again after I tried changing the input source to "default."

Future tasks

At the moment, the user-facing page at https://video.northcoastsynthesis.com/ is quite bare. I'd like to enhance it to include some kind of back channel, for people to make comments that I can see while I'm streaming and maybe even overlay on the video Streaming-chan style. ("Too much clothing!") At present, I can encourage people to comment on Twitter and Mastodon, but those both require accounts and separate browser tabs or windows.

I'd like to have some way of knowing how many people are tuned in in real time, and maybe even where they are geographically at least to the country level. It seems like the easiest way to do this is to write a script that scans the last few seconds of Web server logs to see which IP addresses have recently downloaded video segments from the live feed.

Video display in the Web player while I'm not live streaming is not well-controlled. At present, the player software just shows whatever was the state of the one-minute HLS window when I last turned off the uplink. If I transmit just over a minute of snow and white noise at the end of each streaming session before turning off the transmitter, that will fill the buffer and be what people see if they tune in while there's no signal. That's not bad, but something smarter could probably be done.

Not really an issue with the streaming transmission setup itself, but at the moment I don't have a way to capture "desktop audio," as in the sounds played by software I'm running on my computer, to OBS Studio. I can play sound to my speakers and pick it up with the Web cam microphone, but I think I'd like to explore using ALSA loopback to get desktop audio available as an input source for OBS Studio.

This Javascript player software is probably capable of also playing back static video files posted on the Web server, and it would be nice to get that working so I can post static videos as well as the live stream.

0 comments