WebM Container Support for Opus Tools

[anthumchris]

How feasible would it be to add official tooling support for WebM Containers? Like Ogg, WebM seems to be a preferred, royalty-free web standard moving forward. Currently, Ogg containers are not supported by Media Source Extensions (MSE), but MSE does support WebM (see w3c/media-source#245). I am currently using third-party tools like ffmpeg to create WebM Opus audio files for use with MSE.

Use Case

Based on my research & experimentation, MSE currently provides the lowest possible latency for Opus playback in browsers with slow connections, allowing audio to be played as soon as the first audio bytes are downloaded. Additionally, MSE offers native platform Opus decoding without needing to use WebAssembly and the Web Audio API to decode and play PCM audio. Would be great to use official Opus tools to achieve this.

Without MSE, native browser playback requires up to 100k to be downloaded (HTTP range requests for start (metadata) and last (duration calculation) bytes). This is problematic for slow internet connections capable of sustainably streaming an Opus file. For example, a 64 bitrate Opus file over a 72 kbps connection yields a 12s playback latency in Chrome and 5s in Firefox:

https://fetch-stream-audio.anthum.com/72kbps/opus/house--64kbs.opus?cacheBust=1

(more throttled endpoints)

[anthumchris]

I haven't been able to determine what MSE bytestreams Safari supports. I probed with:

console.log([
  MediaSource.isTypeSupported('audio/webm; codecs="opus"'),
  MediaSource.isTypeSupported('audio/mp4; codecs="opus"'),
  MediaSource.isTypeSupported('audio/mp2t; codecs="opus"'),
  MediaSource.isTypeSupported('audio/mpeg; codecs="opus"'),
  MediaSource.isTypeSupported('audio/aac; codecs="opus"')
])

[guest271314]

@anthumchris

Currently, Ogg containers are not supported by Media Source Extensions (MSE)

FYI: MSE on Chromium 105 now supports passing ('opus') EncodedAudioChunk from WebCodecs https://plnkr.co/edit/cAbZqVXfzZMFOaxL?preview.

[guest271314]

@anthumchris

See https://github.com/guest271314/WebCodecsOpusRecorder. From my testing the WebCodecs configuration and Opus packets in a single file has less total size than Opus in WebM produced by Chromium MediaRecoder implementation.

[vadimkantorov]

it would already be useful even if it's just supported in opusdec for demuxing+decoding

[vadimkantorov]

Btw it appears that mkv + opus can significantly save space on omitting silence: https://video.stackexchange.com/questions/37055/possible-to-force-ffmpeg-to-omit-silence-in-audio-files-and-produce-discontinuou/37056?noredirect=1#comment47208_37056

if opusenc can support output webm streams + silence suppression, it would be very practical and not force ffmpeg usage and be faster