Share this
LL-HLS Series: The Evolution of LL-HLS
by THEOplayer on June 16, 2020
HTTP Live Streaming, or HLS, was initially created and released by Apple in 2009 to solve the problems of scaling. The HLS protocol has become one of the most popular protocols used today, and is widely supported. It’s biggest disadvantage, as with other similar HTTP based streaming protocols, is latency. In this blog post we will discuss the evolution of Low Latency HLS, how the different variants and updates have changed over time, and how they have impacted the video streaming industry. To read more about how HLS works, check out our previous blog post.
The Need for Lower Latencies
While HLS has become widely adopted over the last years, the market changed. Latency has become a bigger problem, leading to spoiled experiences. Strategies and improvements were suggested to combat latency, and people in the industry have been actively looking for solutions to deliver video at lower latency.
It was Twitter’s Periscope platform that first implemented a number of these improvements and bundled them into LHLS in 2016. While a proprietary version by Periscope, it resolved some drawbacks within HLS and aimed to provide low latency in the range of two to five seconds. The protocol was developed to be compatible with the HLS standard in such a way that default players across different platforms could fall back to standard HLS behaviour. LHLS introduced two approaches in order to reduce latency within LHLS:
- Leveraging HTTP/1.1 chunked transport for segments
- Announcing segments before they are available
While reducing the segment size could be another possible approach, it has its limitations. In a best case scenario, each segment starts with an Intra frame (IDR-frame), which allows a player to start the playback of a segment immediately, without the need to wait and download an earlier segment. As I-frames are significantly larger than predicted frames (P-frames), reducing the segment size (and adding in more I-frames) would increase the overall bandwidth used.
The approach used by Periscope tackled the problem of a large buffer, as a part of that buffer was announced up front, without it being available. As a result, this actually reduced the mandatory backoff from the live point, but also the ability to build a buffer. As such, a player should know the location for a segment as soon as possible. By anticipating segment creation, and already listing the location of future segments, a player can anticipate which files need to be loaded. Announcing the segments early, players which were not LHLS aware could still play the stream as if it were a normal HLS stream, and still get an improvement in latency.
This update removed both the latency introduced by segmenting the video stream, as well as the latency introduced by the playlist. Additionally, the approach is scalable over standard CDNs, given they support HTTP/1.1 chunked transfer. To read more about the first LHLS update, check out THEO’s blog post.
The Answer from the Industry
Based on the progress Periscope made with LHLS, a larger group of companies started to experiment with combinations of chunked transfer and HLS. Out of this, a community-based L-HLS specification was drafted. While based on the same concepts as Periscope's LHLS, it introduced a new tag for future segments instead of replacing the last segments in the playlist. This allows to remove all impact on players not compatible with the L-HLS specification and enables them to build a large buffer as with normal HLS.
While this alternative version of HLS was being finalised, in 2019, Apple released a protocol extension for Low Latency HLS later referred to as LL-HLS. The extension still provided backwards compatibility with regular HLS, but gave an Apple approved (and Apple ecosystem-compatible) approach to low latency with HLS. Players which do not understand the protocol extension will play back the same streams, with a higher latency. This allows publishers to offer a single HLS solution both for optimised and non-optimised players. In essence, the update was quite simple: segments are allowed to be split up into smaller parts, which can be downloaded as the larger segment is constructed.
The biggest challenges in this first version of LL-HLS came with the HTTP/2 Push requirement. With the announcement of the protocol extension, Apple stated this requirement would be “widely adopted” by CDNs, however this wasn’t usually the case. The new protocol extension required a number of HTTP/2 features (including multi stream control, H2 push and H2 ping). Many of the major CDN’s were not ready for this, which made large scalability seem like a huge headache for most in the industry. It also caused additional problems for use cases such as SSAI and content replacement.
The industry felt Apple missed out on a simpler solution using chunked transfer encoding (CTE). Using a CTE-based solution would have streamlined the process of delivering low latency content on both HLS and MPEG-DASH streams. The extension also meant a delay in implementation. For SSAI, the specification required an intimate collaboration between the playlist manipulator and the CDN. Tying together both playlist generation and data delivery (through segment push) would require new algorithms and scalability strategies for CDN providers. To read a full update of all the changes, check out THEO’s blog post, or this blog post made by our friends at MUX.
The Latest LL-HLS Update
In early 2020, Apple announced an update to the draft LL-HLS specification. After the concern and friction within the industry, the update removed the HTTP/2 Push requirement and instead introduced a new tag to announce upcoming segments. This update made the LL-HLS protocol very similar to the community version of low latency L-HLS.
The new tag introduced, was called #EXT-X-PRELOAD-HINT. With this tag, a server publishing a low latency HLS stream is able to announce the most likely location of the next media data which will become needed to continue playback. This allows a player client to perform a request, allowing the data to flow in as soon as the next part of a segment becomes available. This process can then be repeated, allowing the removal of additional round-trip time when loading new media data (which was the main reason to use HTTP/2 push).
The updated spec required significant changes to those (THEO included) who had already started implementation of LL-HLS tools. The advantages due to the new change however are significant for the industry and will remove the HTTP/2 push adoption barrier, simplifying the deployment of the technology. At the end of April, Apple published the updated RFC of HLS, including LL-HLS, which makes large future changes unlikely. This allows the industry to go all-in on LL-HLS without running the risk of having to make significant changes in architecture.
Other important updates of LL-HLS spec include:
-
The usage of the new tags (EXT-X-PART, EXT-X-PRELOAD-HINT, EXT-X-SERVER-CONTROL, EXT-X-SKIP, etc.) has been specified in use together with "normal" HLS streams. This means some of these can also be used without low latency mode (i.e. the ability to support blocking playlist reloads and delta playlists.).
-
The addition of CAN-SKIP-DATERANGES tag to allow skipping of daterange tags.
-
There is a LL-HLS profile which has been defined. It contains most of the data as it was in the preliminary specification for LL-HLS:
- Blocking playlist reload is no longer mandatory, but is recommended.
- Multiple preload hints can be listed for the same type.
- It is no longer defined how long parts must remain in the playlist (parts must be removed after 3 target durations, and should appear at "live edge", but "live edge" is not defined).
The Adoption of LL-HLS Today
The newest LL-HLS update presented another advantage by easing compatibility with the major alternative low latency solutions currently in the market, such as LL-DASH and community L-HLS. In fact, the new LL-HLS specification feels like a superset of L-HLS, which also makes use of a tag to announce the location of the next segment. With that being said, it seems that the industry is giving up on community L-HLS, and the race has begun to move forward and adopt the newest update of the Apple LL-HLS spec.
With Apple’s recent solidification of the changes from the preliminary extension into the official update of the HLS specification, it's safe for people to start investing in this.
What's Next?
Now that we’ve discussed how HLS, LHLS, and LL-HLS have evolved and changed over the last decade, we want to go more in-depth on how exactly the most up-to-date spec works, problems it’s fixed, what challenges and opportunities the spec brings to the table. Watch this space for the next blog.
THEO currently has a beta player available that supports the newest LL-HLS update. If you’re interested, don’t hesitate to contact our experts.
Interested in learning more about implementing a Low Latency solution?
On-Demand Webinar: Apple’s LL-HLS is Finally Here
Hosted by our CTO Pieter-Jan Speelmans
Contact our LL-HLS Experts
Share this
- THEOplayer (46)
- online streaming (40)
- live streaming (35)
- low latency (32)
- video streaming (32)
- HESP (24)
- HLS (21)
- new features (21)
- THEO Technologies (20)
- SDK (19)
- THEOlive (17)
- best video player (17)
- cross-platform (16)
- html5 player (16)
- LL-HLS (15)
- online video (15)
- SmartTV (12)
- delivering content (12)
- MPEG-DASH (11)
- Tizen (11)
- latency (11)
- partnership (11)
- Samsung (10)
- awards (10)
- content monetisation (10)
- innovation (10)
- Big Screen (9)
- CDN (9)
- High Efficiency Streaming Protocol (9)
- fast zapping (9)
- video codec (9)
- SSAI (8)
- Ultra Low Latency (8)
- WebOS (8)
- advertising (8)
- viewers expercience (8)
- "content delivery" (7)
- Adobe flash (7)
- LG (7)
- Online Advertising (7)
- Streaming Media Readers' Choice Awards (7)
- html5 (7)
- low bandwidth (7)
- Apple (6)
- CMAF (6)
- Efficiency (6)
- Events (6)
- drm (6)
- interactive video (6)
- sports streaming (6)
- video content (6)
- viewer experience (6)
- ABR (5)
- Bandwidth Usage (5)
- Deloitte (5)
- HTTP (5)
- ad revenue (5)
- adaptive bitrate (5)
- nomination (5)
- reduce buffering (5)
- release (5)
- roku (5)
- sports betting (5)
- video monetization (5)
- AV1 (4)
- DVR (4)
- Encoding (4)
- THEO Technologies Partner Success Team (4)
- Update (4)
- case study (4)
- client-side ad insertion (4)
- content encryption (4)
- content protection (4)
- fast 50 (4)
- google (4)
- monetization (4)
- nab show (4)
- streaming media west (4)
- support matrix (4)
- AES-128 (3)
- Chrome (3)
- Cost Efficient (3)
- H.265 (3)
- HESP Alliance (3)
- HEVC (3)
- IBC (3)
- IBC trade show (3)
- React Native SDK (3)
- THEOplayer Partner Success Team (3)
- VMAP (3)
- VOD (3)
- Year Award (3)
- content integration (3)
- customer case (3)
- customise feature (3)
- dynamic ad insertion (3)
- scalable (3)
- server-side ad insertion (3)
- video (3)
- video trends (3)
- webRTC (3)
- "network api" (2)
- Amino Technologies (2)
- Android TV (2)
- CSI Awards (2)
- Encryption (2)
- FireTV (2)
- H.264 (2)
- LHLS (2)
- LL-DASH (2)
- MPEG (2)
- Microsoft Silverlight (2)
- NAB (2)
- OMID (2)
- Press Release (2)
- React Native (2)
- Start-Up Times (2)
- UI (2)
- VAST (2)
- VP9 (2)
- VPAID (2)
- VPAID2.0 (2)
- ad block detection (2)
- ad blocking (2)
- adobe (2)
- ads in HTML5 (2)
- analytics (2)
- android (2)
- captions (2)
- chromecast (2)
- chromecast support (2)
- clipping (2)
- closed captions (2)
- deloitte rising star (2)
- fast500 (2)
- frame accurate clipping (2)
- frame accurate seeking (2)
- metadata (2)
- multiple audio (2)
- playback speed (2)
- plugin-free (2)
- pricing (2)
- seamless transition (2)
- server-side ad replacement (2)
- subtitles (2)
- video publishers (2)
- viewer engagement (2)
- wowza (2)
- "smooth playback" (1)
- 360 Video (1)
- AOM (1)
- API (1)
- BVE (1)
- Best of Show (1)
- CEA-608 (1)
- CEA-708 (1)
- CORS (1)
- DIY (1)
- Edge (1)
- FCC (1)
- HLS stream (1)
- Hudl (1)
- LCEVC (1)
- Microsoft Azure Media Services (1)
- Monoscopic (1)
- NAB Show 2016 (1)
- NPM (1)
- NetOn.Live (1)
- OTT (1)
- Periscope (1)
- Real-time (1)
- SGAI (1)
- SIMID (1)
- Scale Up of the Year award (1)
- Seeking (1)
- Stereoscopic (1)
- Swisscom (1)
- TVB Europe (1)
- Tech Startup Day (1)
- Telenet (1)
- Uncategorized (1)
- University of Manitoba (1)
- User Interface (1)
- VR (1)
- VR180 (1)
- Vivaldi support (1)
- Vualto (1)
- adblock detection (1)
- apple tv (1)
- audio (1)
- autoplay (1)
- cloud (1)
- company news (1)
- facebook html5 (1)
- faster ABR (1)
- fmp4 (1)
- hiring (1)
- iGameMedia (1)
- iOS (1)
- iOS SDK (1)
- iPadOS (1)
- id3 (1)
- language localisation (1)
- micro moments (1)
- mobile ad (1)
- nagasoft (1)
- new web browser (1)
- offline playback (1)
- preloading (1)
- program-date-time (1)
- server-guided ad insertion (1)
- stream problems (1)
- streaming media east (1)
- support organization (1)
- thumbnails (1)
- use case (1)
- video clipping (1)
- video recording (1)
- video trends in 2016 (1)
- visibility (1)
- vulnerabilities (1)
- zero-day exploit (1)
- November 2024 (1)
- August 2024 (1)
- July 2024 (1)
- January 2024 (1)
- December 2023 (2)
- September 2023 (1)
- July 2023 (2)
- June 2023 (1)
- April 2023 (4)
- March 2023 (2)
- December 2022 (1)
- September 2022 (4)
- July 2022 (2)
- June 2022 (3)
- April 2022 (3)
- March 2022 (1)
- February 2022 (1)
- January 2022 (1)
- November 2021 (1)
- October 2021 (3)
- September 2021 (3)
- August 2021 (1)
- July 2021 (1)
- June 2021 (1)
- May 2021 (8)
- April 2021 (4)
- March 2021 (6)
- February 2021 (10)
- January 2021 (4)
- December 2020 (1)
- November 2020 (1)
- October 2020 (1)
- September 2020 (3)
- August 2020 (1)
- July 2020 (3)
- June 2020 (3)
- May 2020 (1)
- April 2020 (3)
- March 2020 (4)
- February 2020 (1)
- January 2020 (3)
- December 2019 (4)
- November 2019 (4)
- October 2019 (1)
- September 2019 (4)
- August 2019 (2)
- June 2019 (1)
- December 2018 (1)
- November 2018 (3)
- October 2018 (1)
- August 2018 (4)
- July 2018 (2)
- June 2018 (2)
- April 2018 (1)
- March 2018 (3)
- February 2018 (2)
- January 2018 (2)
- December 2017 (1)
- November 2017 (1)
- October 2017 (1)
- September 2017 (2)
- August 2017 (3)
- May 2017 (3)
- April 2017 (1)
- March 2017 (1)
- February 2017 (1)
- December 2016 (1)
- November 2016 (3)
- October 2016 (2)
- September 2016 (4)
- August 2016 (3)
- July 2016 (1)
- May 2016 (2)
- April 2016 (4)
- March 2016 (2)
- February 2016 (4)
- January 2016 (2)
- December 2015 (1)
- November 2015 (2)
- October 2015 (5)
- August 2015 (3)
- July 2015 (1)
- May 2015 (1)
- March 2015 (2)
- January 2015 (2)
- September 2014 (1)
- August 2014 (1)