This isn’t really a janus question although I am using janus gateway as the SFU. Basically what I need is a way to send a media stream to janus over a link that is expected to have packet loss without introducing massive amounts of latency AND not interrupting the stream for seconds at a time due to a single packet being dropped. The end application requires minimal latency, of less than 250ms, (50ms RTT worst case on the network side so pretty big budget for encode).
Currently I’m sending VP9 with some well tuned parameters to maximise the reliability. 1080p25 at 1.2mbps max bit rate and interframe bit rate of 400kbps with I-frames every 50. This is sent from the source to the SFU and forwarded on to the consumers, a VP8 encoder also consumes the RTP stream on the cloud server and encodes a redundant format for consumers that can’t decode VP9.
Obviously, the issue is once a single packet is dropped between the source and janus then all consumers loose video until the next keyframe is received entirely intact. Due to the remote (regional internet) nature of the source it’s pretty much impossible to avoid loss hence the low bit rate. What I am looking for is an intermediate codec, something that can be packetized with gstreamer and then reencoded as VP9 and 8 on the cloud side with minimal latency and most importantly handle packet loss gracefully. Video corruption or resolution loss is acceptable, complete stoppage of the image isn’t.
I have considered using AV1 but it seems like a huge can of worms. If there is a more suitable codec I’ve not considered that would be great. In terms of image fidelity I’m looking for something either equivalent to VP9 or at least better than VP8 at the given bit rate, something like h264 would require 5-8x the bandwidth for an equivalent image quality as an example.