Mini Guide on Low Latency

When it comes to live video, it's hard to deliver an actual, real live streaming experience. The latency problem is especially noticeable when video is distributed using online streaming platforms. You may have heard the stories of watching a sports event and hearing the neighbours cheer, before seeing the start of an attack. In this mini guide, we'll present the five things you should know about low latency.

1) WHAT IS LATENCY?

Within a streaming environment, latency is most easily experienced by measuring the time when something is recorded in real life and when it is seen on the screen, for example waving your hand in front of a camera and watching it move on the screen a certain time later.

Having a high latency on streams, and especially in interactive live experiences, can have a significant impact on user experience. Imagine a live stream with a latency of about 20s with an interactive chat box where the audience can ask questions:

this would mean the person starring in the video could perform some action at a certain time T0. This image would show up on viewers’ screens 20s later.

During this time, the broadcaster would simply continue with whatever he is doing. However, if a user would then type in and ask a question about the action at T0, showing up on the broadcaster’s screen, the broadcaster might be doing something completely different already, and have lost context. In return a (wrong) answer would still take another 20s before the answer gets back to the viewer!

2) WHAT CAUSES LATENCY?

• Encoding & packaging: Latency introduced is very sensitive to the configuration and quality of the output signal needed to be achieved. Certain streaming protocols can introduce additional latency, as they can only output a chunk of media once it has been completely ingested.
• First mile upload: Uploading the packaged content to a CDN is often restrained by the business case. For example, an upload will have a significantly larger impact if done over a wireless connection at an event, compared to a leased line setup coming from a news studio.
• CDN propagation: In order to deliver content at scale, most media pipelines leverage content delivery networks. As a result, content needs to propagate between different caches, introducing additional latency.
• Last mile delivery: A user's network connection can have a significant impact on latency. The user could be on a cabled network at home, be connected to a wifi hotspot, or using amobile connection in order to access the content. Also, depending on geographical location and position of the closest CDN endpoint, additional latency could be added.
• Player buffer: Video players have to buffer media in order to ensure smooth playback. Often the sizes of buffers are defined in media specifications, but in some cases there is some flexibility in this area. Also, optimising buffer configuration can go a long way given the significant impact of this component.

3) LATENCY VS SCALABILITY VS QUALITY

                                        LATENCY           <>       SCALABILITY      <>           QUALITY

When looking at latency, we must also consider two other key factors at play: scalabilty and quality.

SCALE

In the past, streaming protocols were severely limited in their scalability. However, streaming protocols such as HTTP Live Streaming (HLS) and MPEGDASH make use of HTTP, and as a result can use standard HTTP caches and CDNs in order to scale. A downside of these protocols however, is that scalability is traded for latency.

QUALITY

Achieving a higher quality for the enduser often results in higher bandwidth requirements. Due to higher resolutions and frame rates, or more time needed on the encoder side in order to shrink the bitstream at high quality.

BALANCE

While new technology and compression algorithms will shift the achieved results, and make lower latency more achievable, finding the right balance within this latency trade off triangle will always be important. Impact of technological improvements can already be seen in the advances made by cloud computing, which significantly reduced scaling challenges as well as new media codecs such as VP9, HEVC and AV1. Reducing the required bandwidth significantly, compared to older techniques.

 4) LATENCY INDUSTRY STANDARDS

While optimising latency is surely important, a question we often get is “How low should your latency be?”. The answer will depend on the business case. In general, we advise our customers to make a number of changes to reduce latency already to be within the 18-30s range. This can be achieved with minimal effort and cost.

In contrast to regular broadcasting, for a real live show, the delay from the actual events to the moment it gets shown on a television set, today, is something between 4 -10s. Significantly lower than the industry standard protocols HLS & DASH as shown in the image.

When latencies comparable to broadcasting are to be achieved, it is interesting to have a look at recent advancements in protocols such as the newer Low Latency CMAF of Chunked CMAF standards being implemented by vendors.

In the instance that a business case requires ultra low latency, or real time latency, solutions such as RTMP or WebRTC are required. This results in high costs when scaling and increasing quality.

5) WHEN IS LOW LATENCY IMPORTANT?

Obviously no one benefits from very high latencies. However, there are a number of cases where low latencies, even below broadcast latencies, are required.

Second screen experience: when watching a television show in parallel with regular broadcast, latencies preferably match at least the latency of the broadcast itself. Furthermore, every form of interactivity require the latencies to be synced as much as possible.

Live sports, E-sports and video game streaming: timing in live events, especially if the audience interacts with the live events which is becoming more frequent, reaching live latencies is becoming crucial.

Video Chat: this is where real-time latencies come in to play. Everyone knows the case where a live journalist is interviewed by the news anchor at a distant location. It takes several seconds before the correspondent can respond to the anchor's question. This negatively impacts the user experience. 

Any questions left? Contact our THEO experts.