Understanding presentation time and audio skew

General
1

Hi there, I have this problem with AV sync in our MJR recordings. Basically when a device is disconnected there’s a gap between the packets (but the sequence numbers are fine). So I’m struggling to understand how the presentation time works - from this it seems to me that all one needs to do is use janus-pp-rec to convert the MJR to eg OPUS and then ffmpeg will automatically correct the audio packets based on these pts times. I did a bit of reserach on pts and ffmpeg but find it quite confusing, so I went through the source for the pp-rec and saw the pts is related to audio skew, I’ve done some initial tests but can’t quite understand how it’s supposed to work.

For reference here are the audio packets:

image
image1174×516 27.5 KB

If I run janus-pp-rec audio.mjr audio.opus and check the file stats:

Input #0, ogg, from 'audio.opus':
  Duration: 00:01:59.54, start: 0.000000, bitrate: 32 kb/s
  Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
    Metadata:
      comment         : {"t": "a", "c": "opus", "s": 1723628074546281, "u": 1723628102553226}
      encoder         : Lavf58.76.100

It’s about 29 seconds too short (the gap in the packets).

I’ve tested with various valuees for audio skew and it seems to fill in the silences but I can’t quite understand how it is supposed to work or whether there is a more standard way of using the saved pts times to correctly export audio MJR files.

2

opus postprocessor detects gaps like this

		/* if restamping is being used, do not evaluate the sequence number jump */
		if(tmp->prev != NULL && ((tmp->ts - tmp->prev->ts)/48/20 > 1) && (restamping || (tmp->seq != tmp->prev->seq+1))) {
			JANUS_LOG(LOG_WARN, "Lost a packet here? (got seq %"SCNu16" after %"SCNu16", time ~%"SCNu64"s)\n",
				tmp->seq, tmp->prev->seq, (tmp->ts-list->ts)/48000);

If a RTP timestamp gap does not have a sequence number gap too, pp-rec will detect the gap only if restamping option (-r) is being used. Skew detection is not the proper strategy, since that is only for sources with skewed clocks that deviates with time.
Please test with restamping (e.g. “-r 200”), pp-rec should detect the gap and insert the silences.

3

Thank you for your response Alessandro! I’ve written a tool to do some pre-preocessing of the mjr and have been experimenting with different restamping values.

4

I was wondering if not using a jitter buffer would have an effect on Janus MJR recordings going out of sync? So AFAIK we have no problems with the live streams, everything is in sync. The issue is some recordings are out of sync, so I’m wondering if the necessary corrections that the browser is applying should be applied to the MJR processing step? Is that possible?

5

The presence of both a jitter buffer and RTCP feedbacks in live decoders (such as browsers) greatly helps in terms of the quality of experience.
OTOH pp-rec acts offline without live buffers and feedback from the peer, hence it works on the whole packets set and potential sync issues must be analyzed by the user and solved with the existing strategies (like skew compensation and pkt restamping).

6

Thank you for your help Alessandro