Synchronize audio and video playback on the web

The Web Audio API makes it possible to properly achieve AV synchronization.

François Beaufort

The outputLatency property of an AudioContext instance provides an estimate of audio hardware's output latency (for example, that of Bluetooth earbuds or of an external USB audio interface). This property is useful when you want to:

• Synchronize the existing audio material and the newly recorded material. (in a music production scenario)
• Synchronize the Web Audio output and other media (e.g. video or MIDI playback).

In this WebCodecs demo (source), the WebCodecs API is used to decode a MediaStream into raw video and audio data, and then played back into a HTML <canvas> element with audio data coming from an Audio Worklet. The outputLatency property allows the demo to determine when a given audio timestamp is reaching the user's ears and then properly paint video frames to match that.

Try it out for yourself, play the video with your favorite Bluetooth headset (🎧), wait for the bird (🐦) (see above), and toggle the checkbox (☑️) to observe audio playback changes. The total output latency value is updated in real time.

AudioContext outputLatency

Hero image by Wahid Khene on Unsplash.