Share this
by THEOplayer on June 16, 2020
HLS, or HTTP Live Streaming, is an adaptive HTTP-based protocol and it was initially created and released by Apple in 2009 to solve the problems of efficient live video and VOD delivery to viewers’ devices, especially Apple devices. One of the main focus points was the problem of scaling, as faced with protocols such as RTMP and RTSP (used for Flash). Over the last years, the HLS protocol has become one of the most popular protocols, for both compatibility and quality of experience, and is widely supported on major devices, browsers and platforms. In this blog post we will discuss exactly how HLS works, the adaptive bitrate process it uses, the latency introduced by using HLS, why the spec is widely used across the industry, as well as how we can combat latency.
How does HLS work?
As it caused one of the biggest scaling issues for RTMP, HLS removed the need for an open connection, and instead uses HTTP served and cacheable files. On a high level, HLS is fairly simple: a video stream is split up in small media segments, meaning that instead of sending out a continuous file, small files are being made with a certain length. The maximum length of such a segment in HLS is called the Target Duration. A player would then need to download these segments one after the other, and simply play them in order within a playlist.
Initially, the container format for HLS was MPEG-2 Transport Stream, but in 2016 fragmented MP4 (fMP4) support was added, which is the preferred format today for HTTP-based streaming, including MPEG-DASH and Microsoft Smooth Streaming.
For players to be able to identify which segment should be downloaded, HLS makes use of a Playlist file. These files list the segments in order. For live streams, new segments will be added at the end of the Playlist file. When a player updates the Playlist file (the protocol dictates it should be reloaded about every Target Duration), it will see the new segments listed and can download and play them.
HLS achieves scalability through the use of a Content Delivery Network (CDN) and just ordinary web servers. CDNs can help and adapt streams for a high spike in viewers, for example during a live sporting event. This was not the case for streaming protocols like RTMP, where CDN support is starting to decline quickly.
ABR Switching and HLS
HLS supports an adaptive bitrate process called adaptive bitrate switching, which allows for a quality stream based on the viewers’ internet connection, as well as the ability to switch bitrates at any time. To allow for adaptive bitrate switching, HLS Playlists are grouped in a Master Playlist which can link to different streams, allowing for a player to choose the stream with the bitrate and resolution best suited for its network and device. Having multiple streams prevents interruptions and buffering during the stream if there is a need to adjust for the bandwidth, adapting to each viewer's circumstance.
Apple provides General Authoring Requirements and suggested targets for encoding and HLS streamhere.
Fig. 2 - ABR switching between HLS manifests based on the network and device.
What Causes Latency with HLS?
The latency introduced by HLS is related to the Target Duration. In order for a streaming server to list a new segment in a playlist, this segment must be created first. As such, the server needs to buffer a segment of ‘Target Duration’ length, before publishing it. Worst case scenario, the first frame a player can download is already ‘Target Duration’ seconds old.
Fig. 3 - Segments generation and Target Duration
Segments usually have a duration ranging from 2 to 6 seconds (this used to be 10 seconds up until June 2016). Most streaming protocols, HLS included, have determined the need for a healthy buffer of about three segments, and also a fourth segment usually being buffered. This is supposed to allow for robustness in case of network or server issues. The reasoning here is that segments need to be encoded, packaged, listed in the playlist, downloaded and added to the player’s buffer as a whole. This often results in 10-30s latencies. This is of course ignoring any encoding, first mile, distribution, and network delays. To read more about the full architecture of HLS, here is Apple’s overview.
Why Should You Use HLS?
HLS should be used anytime when quality of experience for the viewer is the first priority. Because the spec is widely adopted across devices, browsers and platforms, it’s compatibility makes it a go-to for a majority in the industry. HLS is widely supported on: iOS, Android, Google Chrome, Safari, Microsoft Edge, Linux, Microsoft and smart TV platforms.
How Can We Combat Latency?
In the HLS spec, having smooth, scalable playback is prioritised over having low latency. Because HLS uses HTTP served and cacheable files, a significant latency is introduced to the stream, and while HLS was becoming more widely adopted, the market changed from not just needing a scalable approach, but also needing lower latencies. Strategies and improvements were suggested to combat latency, and people were actively looking for solutions to deliver video in real-time. In our next blog, we will discuss how the need for lower latencies and near “real-time” video streaming gave the industry the solutions to provide Low Latency HLS.
Interested in learning more about Low Latency Streaming?
Watch our Webinar: Low Latency
Hosted by our CTO Pieter-Jan Speelmans and VP of Innovation Johan Vounckx
Contact our LL-HLS Experts
