Use WebTransport instead of WebSocket

[Zubnix]

Compositor-proxy currently uses WebSockets to communicate application [video] frame data. This has well known drawbacks for use in real-time communication as it's essentially a TCP connection. Instead we should use WebTransport which offers UDP semantics which is far better suited for real-time communication.

Initial preparing work has already been completed by switching to the uwebsocketjs library which will eventually support WebTransport

[ehfd]

Thanks for the information! Looking forward for its maturation.

[Zubnix]

fyi webrtc data-channels have been implemented and subsequently dropped for being too slow because of the SCTP congestion control overhead. That is, on a 300Mbit connection with zero packet loss, you'd get 30Mbit throughput which is unacceptably slow.

So we patiently await the support and arrival of WebTransport.

[ehfd]

https://caniuse.com/webtransport

https://caniuse.com/webcodecs

All major browsers other than Safari now support WebTransport. Firefox is working on WebCodecs.

[ehfd]

selkies-project/selkies-gstreamer#48 (comment)

We've been investigating WebTransport but there are a few catches outlined above (requiring valid HTTPS certificates and other issues).

But overall, the web browser stack is ready.

More references:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1571
https://github.com/kixelated/moq-rs
https://github.com/wanjohiryan/warp
https://github.com/wanjohiryan/Arc3dia

[ehfd]

fyi webrtc data-channels have been implemented and subsequently dropped for being too slow because of the SCTP congestion control overhead. That is, on a 300Mbit connection with zero packet loss, you'd get 30Mbit throughput which is unacceptably slow.

We (https://github.com/selkies-project/selkies-gstreamer) use WebRTC DataChannels for passing user input and clipboard, and use the native WebRTC MediaStreams for video and video. It's relatively inflexible but more usable than using DataChannels for everything.