lorenzo
May 12, 2023, 4:48pm
1
Hi all,
we’ve started implementing support for the new VP9 and AV1 simulcast functionality that recent versions of Chrome are shipping. In the process, we took advantage of this to also fix the broken AV1 SVC support, that now works as expected instead. If this sounds of interest to you (and it should, if you use simulcast in your application), please do take the time to test it:
meetecho:master ← meetecho:vp9-av1-simulcast
opened 04:38PM - 10 May 23 UTC
I recently read Chrome had started adding experimental support for [VP9 and AV1 … simulcast](https://groups.google.com/g/discuss-webrtc/c/-QQ3pxrl-fw). Janus plugins are currently hardcoded to only accept simulcast for VP8 and H.264, so I wanted to check how much work it would take to remove that constraint and accept it for VP9 and AV1 as well.
As you can see in this patch (which currently only targets the EchoTest plugin), it took just a few minimal changes, namely:
1. making sure that when we create the simulcast envelope in `janus.js` we provide the right scalability mode (`scalabilityMode: 'L1T2'` in this case)
2. removing any VP8/H.264 constraint in the EchoTest when detecting simulcast
3. modifying the `janus_rtp_simulcasting_context_process_rtp` function to have it be aware of keyframe detection for other codecs as well.
This was enough for me to test the EchoTest demo using simulcast for both VP9 and AV1 on a recent Chrome version:
http://localhost:8000/echotest.html?simulcast=true&vcodec=vp9
http://localhost:8000/echotest.html?simulcast=true&vcodec=av1
In the next few days I'll patch other plugins to support the same functionality, so that it can be tested in more interesting scenarios as well (e.g., VideoRoom or RTP forwarders).
What I haven't figured out yet is how to deal with temporal layers (which would be two in the `L1T2` case, for instance). For VP8, we parse the payload descriptor of the VP8 header to extract the temporal layer; for H.264 we don't do anything, instead, since temporal layers are not enabled when doing H.264 simulcasting. I don't know what should be done for VP9 and AV1 instead: should we parse the beginning of the payload the same way as we do for VP8? Or is a specific RTP extension used for the purpose instead? As such, at the moment, we don't do anything with those: I'll add support for that too as soon as I know more.
Feedback welcome!
Thanks,
2 Likes
lorenzo
May 16, 2023, 9:25am
2
For those interested, I’ve also written a blog post on the effort:
I only recently found out that Chrome has started shipping experimental support for VP9 and AV1 simulcast. If you've visited this blog before, you know what simulcast is already (if you don't, make sure you read this post first!) and you may recall...