Using ffmpeg to achieve HTTP Live Streaming (HLS) with webcam

Problem

Recently I have been working on this project from my lab. Basically it helps transportation agencies to count traffic and trespassing events at grade crossings (where vehicles and trains both use the road). We are going to install a customized hardware box in which a Jetson connected to a webcam is put. The problem is how to capture the video of the webcam and gives it back to our server as a live feed so that we can then broadcast to end users so that they can monitor the crossing.

Solution

While our recognition logic is written in Python with OpenCV2 as the framework to capture video frames from webcam, I feel like there is an easy way to solve the problem by using FFmpeg. Basically here we are going to use FFmpeg’s ability to capture webcam and transform the stream to HLS files including the playlist and the video chunks. Then we can set up a simple nginx server to serve these files locally and use ngrok (mentioned in the previous blog post) to expose that HLS server to our cloud server on Amazon EC2 and then we can relay the live feed from the field.

Since Jetson runs Ubuntu 16.04, we capture the video by using v4l2 device. v4l2 stands for video4linux2, which is a device used by ffmpeg to capture video from cameras on Linux. Here is the command for setting up the local HLS config.

ffmpeg -video_size hd720 -f video4linux2 -i /dev/video1 -c:v libx264 -preset veryfast -f hls -hls_time 4 -hls_flags delete_segments stream.m3u8

Here use -f to specify the input and the /dev/video1 is the physical address of our camera. In the middle is some common ffmpeg encoder settings. At the end, we have -f hls to specify output as HLS, -hls_time 4 to specify video chunks to be 4 seconds long. This may not be effective since ffmpeg only partition video after key frames. We’ll discuss how to limit the duration of a chunk exactly to a period if time. -hls_flags delete_segments means to delete segments that are out of the live window. We can also set it to keep all the segments in order to achieve playback functionality. Finally stream.m3u8 is the destination of your playlist. I run the command in my root folder of nginx server so it populates the playlist right in place.