The Hidden Complexities of Time in Live Streaming

Live streaming systems face a critical technical problem: multiple independent clocks that disagree on time and tick at different rates. A source claiming 25 frames per second may deliver perceptibly more or fewer frames when measured against local system time.

This isn't theoretical complexity—it's operational chaos. Without consistent time reference, audio drifts from video, multi-camera feeds desynchronize, and viewers experience noticeably different latencies.

The underlying issue compounds when mixing multiple sources. Each stream carries its own clock. The time information embedded into the media itself isn't a real-world timecode but a simple counter that wraps every 27 hours. Attempted fixes through SEI timecodes frequently fail due to encoder misconfiguration; for example, we've documented professional equipment where audio and video timecodes loop at different intervals, even one that simply failed after 13 hours.

Robust streaming infrastructure must anticipate this chaos: inconsistent timing, clock drift, and outright incorrect data from external sources. Time synchronization isn't an implementation detail—it's the architectural foundation that prevents systemic failure.

At Norsk, we've spent years wrestling with the challenges of time synchronization, clock drift, and discontinuities. In this white paper, you'll learn about why time sync is so complicated, as well as strategies to ensure your audio and video syncs perfectly, every time.