This is part 3 in a multi part series about how localtv is setup. If you missed it , you can read part1 and part2 .
This entry will explain how we encode our video for modern devices and retransmission over the internet. I assume that you have followed part 2 and now have an uncompressed MPEG2 stream of tv channels. Our encoder will consume those streams.
Open Source Software
Similar to our use of tvheadend, our goal was to use as little custom software as possible and and keep it all open source. This has two benefits, first it’s free. Second you get to tap into a pool of developers maintaining the software and so not need to pay to make and maintain it yourself. The three key pieces of free software we do use here are Linux to power everything and where all sample commands will run, ffmpeg to encode the video and Apache to actually serve it on the internet.
Encoding with ffmpeg
Ffmpeg is the industry standard for video encoding and conversion. If you use a service that encodes and serves video such as YouTube it is almost certainly using ffmpeg in the backend. It is old, well tested and well maintained. This is what we will use.
HLS
HTTP live stream (aka HLS) is an internet video broadcast standard that Apple invented. While it was made for Apple devices it has become an industry standard. When you view video on the internet (such as YouTube) you are likely using HLS or something similar. The way HLS works is there is a playlist file that lists in order lots of little bits of video. All of this is served over a normal HTTP server like Apache. There are no social protocols involved it’s just files served over HTTP. HLS is the only format the native iOS video player will support.
Tying it all together
Ffmpeg has an option to output HLS files. You could use a different encoder and packager (such as Bento4) but having one is working well for me. HLS allows you to have multiple different streams of different quality for different connections but these examples have just one stream. Having multiple streams just involves duplicating the arguments to ffmpeg with different settings.
What we will do here is:
Mpeg2 -> ffmpeg -> Apache
Software Encoders
The final piece of the ffmpeg puzzle is picking an encoder. You will find two kinds of encoder, software encoders which work on the CPU and hardware encoders that run on the GPU. I’ve found software encoders to to produce superior quality video and are more compatible with streams. They are also easier to start with. In our example, we use a software encoder for Fox. There is something odd about that broadcast in Boston that it makes our hardware encoders fail but the software H264 encoder works fine. The software encoder also encodes closed captions properly none of our hardware encoders do it properly. That is why Fox is currently the only channel with closed caption support. This a basic command that will get a single channel working :
ffmpeg -loglevel info -hide_banner -re -y -i http://<MPEG stream> -err_detect careful \
-map 0:0 -map 0:1 \
-vcodec libx264 -sc_threshold 0 -g 48 -keyint_min 48 -c:a copy -ar 48k \
-maxrate:v 2000k -bufsize:v 3000k -b:a 192k -crf 20 -fps_mode auto \
-hls_time 6 \
-hls_list_size 20 \
-hls_flags delete_segments \
-master_pl_name 720.m3u8 \
-hls_segment_filename fox/%v_%03d.ts fox/%v.m3u8The one drawback here is -c:a copy which set it to copy the audio stream instead of re-encoding it. In is our command to encode FOX, we use a custom build of ffmpeg that enables the libfdk_aac codec which is not enabled by default for license reasons. This was done to change the existing AC3 audio to AAC to support Android and Roku which can’t play the native AC3 audio used in ATSC broadcasts.
if you run this command you will have a stream in the fox folder. Serve that via Apache and your iPhone will be able directly play a stream of the channel by opening the fox/720.m3u8 file in Safari.
There are two drawbacks to this command however, first is the CPU usage from the software encoder is high. The second, is you will be writing to the fox folder continuously and the disk introduces latency and increases wear and tear on you drive. You want the fox folder above to be on a RAM disk. This way all reading and writing happens entire in RAM. You should see a significant performance boost from this.
sudo mount -t tmpfs -o size=5000m tmpfs /mnt/ramdisk
cd /mnt/ramdiskHardware Encoders
As I mentioned above software encoders while easy to get going are not as efficient as hardware encoders. This is one of the benefits of using H264, lots of old hardware has hardware encoders. Even the humble raspberry pi 2 and 3 has h264 hardware encoders. If you are looking for the say that you can use up that box of spare computer parts you have held on for a rainy day, this is likely the day.
The kind of hardware encoder you use and what the quality of the video it outputs is entirely dependent on your GPU. Each manufacturer has their own encoders. For example, Intel has quick sync and Nvidia has NVENC. In our case we primarily use intel quick sync that is available on i5 and up chips. We have a 2012 dell desktop that encodes most of our channels. This older quicksync encoder while very efficient does not seem to support closed captions well. This is something we are working on addressing. All of our HD channels are encoded with this encoder. While a single 2012 desktop could maybe have encoder a single channel in software it can encode half a dozen HD channels with the hardware encoder and have no impact on the CPU. This is the command we use to encode with a intel quicksync:
ffmpeg -loglevel error -hide_banner -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i http://<MPEG2 stream> -vf 'scale=-1:720,format=nv12,hwupload' -map 0:0 -map 0:1 -threads 8 -y -acodec libfdk_aac -b:a 384k -b:v 3500k -maxrate:v 3500k -vcodec h264_vaapi -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 6 -hls_list_size 20 -hls_flags delete_segments -qp 28 -bufsize:v 6400k -hls_segment_filename streams/abc/1080_%03d.ts streams/abc/1080.m3u8In general it looks a lot like the software encoder earlier except we indicate the h264_vaapi hardware encoder. Depending on what the source material of your channel is you may want to scale everything to 720p. We do this for many channels:
-vf 'scale=-1:720,format=nv12,hwupload'The primary reason you would do this is if your source channel is 720p but your local broadcaster has upscaled it to 1080i there isn’t a lot of reason for you to try to send out 1080i if it was originally 720p. This is the case with several channels in Boston. The scale relates to the bit rate -b:v 3500k , ultimately this is how much data you want to represent single pixel. 1080i is much bigger than 720p so you would have to increase you bit rate dramatically for 1080i video. In real world terms this translates to much bigger files and more data used. As you will see in part 4, more data served means greater cost.
Our non HD sub channels (e.g. Laff an MeTV) are actually encoded by old Raspberry PIs (2 and 3s) I had lying around in a bin of old computer parts. A Raspberry PI 3 could in theory actually encode a HD channel but I have had issues with the CPU throttling and would need more active thermal management on the pi3. On the Pi (32 bit) we use an encoder called OMX:
ffmpeg -loglevel error -hide_banner -re -y -i http://<MPEG stream> \
-err_detect careful -map 0:v:0 -map 0:a:0 \
-vcodec h264_omx -sc_threshold 0 -g 48 -keyint_min 48 -c:a copy -ar 48k \
-b:v:0 3500k -b:a:0 192k -profile:v main \
-var_stream_map "v:0,a:0" -fps_mode auto \
-hls_time 6 \
-hls_list_size 20 \
-hls_flags delete_segments \
-hls_segment_filename laff/480_%03d.ts laff/480.m3u8
Protecting your stream
At this point you are broadcasting your tv streams in HLS format and any device can watch the video. However these streams are free for everyone to view. You should protect them in someway. There are different ways of doing this. Apple has a DRM scheme called FairPlay and Google has a scheme called Widevine. Oddly enough, Widevine is what the broadcasters use to DRM encrypt ATSC3. My understanding is main drawbacks of each is Apples DRM does not work on Android and Googles DRM does not work on iOS. There is an encryption scheme included in HLS that does work with both. We create an encryption key with openssl and include the following flag in all of our ffmpeg commands
-hls_key_info_file enc.keyinfoWe rotate our encryption keys continuously and deliver it to clients out of band. In our case it is pushed to all clients via Apple’s APNS. The key is never persisted anywhere and needs to be fetched every time the app starts. Generally this works well but if you have ever seen an error about fetching playback urls, this is what the issue probably is. For us to send you the decryption key we need verify you are in our service area. We ask for location access on the client but also run many checks on the server to determine where you are (or if you are trying to use a VPN).
In theory everyone is receiving the same video encoded and encrypted once. It’s not perfect but it does make it really painful to view our streams when you are not authorized. It is also not something a casual pirate will be able to do.
At this point you have replicated Localtv+ as it exited in late 2023. We quickly discovered even with good upload speeds we can’t serve more than a few concurrent viewers. We added a CDN in-between to address this and were able to scale up to thousands of viewers. In part 4 we will discuss content distribution networks (CDNs) and costs.