Plan to support abs-capture-time extension in streaming plugin

General
1

I’d like to support including the abs-capture-time extension in RTP payloads to Janus and those be forwarded to all peers. The use case is returning the most recent capture timestamp to the source to validate that a message was produced by a peer while an up-to-date timestamped frame was on the screen to make guarantees about end-to-end latency in teleoperation.

My understanding is the extension is supported in Chrome 107, and I’ve implemented adding the extension from the source, now I only need to bridge the gap in Janus.

My understanding is that because plain RTP to Janus does not have an SDP, any extensions in the RTP headers can not be mapped. I would intend to add a configuration key to the streaming plugin to specify which extension ID is abs-capture-time to work around this.

From there I expect I can add abs-capture-time to the SDP for WebRTC peers and include the timestamps. Like the other abs-capture-time PR currently outstanding, I intend to only forward these values with no interpolation. I will likely build on top of that other PR since it adds some support for the extension.

Does this approach make sense? Does it sound like something that would be accepted upstream? Is there a better way to do this?

Ref: docs/native-code/rtp-hdrext/abs-capture-time - src - Git at Google.

2

I had also considered a configurable id:extension mapping in the configuration as a general purpose way to forward RTP extensions to peers, but this doesn’t have the same path forward for abs-capture-time interpolation.

3

There’s a PR to add support to the extension here, which no one provided feedback to yet. That PR doesn’t include the Streaming plugin but it should be easy enough for you to have it negotiated there.

Edit: sorry, re-reading your post it looks like you’re already aware of the extension, and wanted a way to figure out how to pass an external abs-capture-time value from plain RTP (Streaming) to WebRTC, since we strip extension values otherwise. In that case, yes, your idea looks reasonable. If you can prepare a PR that extends that one, I’ll be happy to review it.

1 Like