I used WebRTC+SIP to call and play the device’s video stream, but Chrome couldn’t decode it. The log shows an avcodec_dend_macket error: -1094995529. But when I use WebRTC+RTSP to access the device’s video stream, it can be played normally. The sps and pps of the two video streams are the same. What changes does Janus make when using SRTP to forward video streams
Janus doesn’t touch the media stream. You likely just have to tweak the H.264 profile advertised in the SDP before setting the remote description to make the browser happy, since officially they only support baseline.
II have two devices that use WebRTC+SIP calling. One device has a profile_level_id of 42001f in its SPS, which can be decoded normally. The other device is 42e01f, which cannot be decoded. In Jauns’ logs, the devices that cannot be decoded there are many logs similar to
[rtp.c:janus_rtp_header_extension_parse_audio_level:222] b8 → v=1, level=56
[rtp.c:janus_rtp_header_extension_parse_audio_level:222] b7 → v=1, level=55
[rtp.c:janus_rtp_header_extension_parse_audio_level:222] 20 → v=0, level=32
[rtp.c:janus_rtp_header_extension_parse_audio_level:222] b6 → v=1, level=54
[rtp.c:janus_rtp_header_extension_parse_audio_level:222] b7 → v=1, level=55
[rtp.c:janus_rtp_header_extension_parse_audio_level:222] b8 → v=1, level=56.
The normal decoding device only has “Got an RTCP packet” log.Is this the problem causing the inability to decode
As I said, Janus doesn’t touch the media streams, so what you see on the Janus side is irrelevant (the log you shared shows an audio-level extension being parsed). Try forcing the profile level ID to be the one that you know works also for the other endpoint: it’s enough to do that by editing the SDP before it’s passed to the browser in a setRemoteDescription call. If you’re using janus.js, it’s something you can do in the customizeSdp callback: check existing demos to see how that works.
Notice that another issue may be that you’re simply not receiving a keyframe as the first frame: in that case, it would not be a profile error, but simply the browser not having a keyframe to decode the others. You can verify that by waiting long enough before hanging up the call (with Chrome, keyframes are sent every 3000 frames, so every 100 seconds for a 30fps video) to see if video finally appears.
Thank you for your answer, but my question still hasn’t been resolved.I checked the SDPs of two devices, from the browser to Janus and from Janus to the device, and they are almost identical. The profile_level_id in sdp is 42801f for both devices. When capturing packets from the device to Janus, the first three packets in H264 are SPS, PPS, and IDR, with SPS and PPS before each keyframe, and the keyframe interval is only a few seconds.
Their sps have some differences, and the sps that cannot be decoded are as follows. The actual profile_level_id is 42e01f:
H264 NAL Unit Payload
Profile_idc: Baseline profile (66)
1… . … = Constraint_set0_flag: 1
.1.. . … = Constraint_set1_flag: 1
..1. . … = Constraint_set2_flag: 1
0 … = Constraint_set3_flag: 0
… 0… = Constraint_set4_flag: 0
… .0.. = Constraint_set5_flag: 0
… . .00 = Reserved_zero_2bits: 0
Level_id: 31 [Level 3.1 14 Mb/s]
1… . … = seq_parameter_set_id: 0
.000 1101 = log2_max_frame_num_minus4: 12
011. . … = pic_order_cnt_type: 2
…0 10.. = num_ref_frames: 1
… . .0. = gaps_in_frame_num_value_allowed_flag: 0
… . ..0 0000 0101 0000 . … = pic_width_in_mbs_minus1: 79
… 0000 0101 101. = pic_height_in_map_units_minus1: 44
… . ..1 = frame_mbs_only_flag: 1
1… . … = direct_8x8_inference_flag: 1
0 … = frame_cropping_flag: 0
..1. . … = vui_parameters_present_flag: 1
0 … = aspect_ratio_info_present_flag: 0
… 0… = overscan_info_present_flag: 0
… .0.. = video_signal_type_present_flag: 0
… . .0. = chroma_loc_info_present_flag: 0
… . ..1 = timing_info_present_flag: 1
0000 0000 0000 0000 0000 0000 0000 0001 = num_units_in_tick: 1
0000 0000 0000 0000 0000 0000 0011 0010 = time_scale: 50
0 … = fixed_frame_rate_flag: 0
0 … = nal_hrd_parameters_present_flag: 0
0 . … = vcl_hrd_parameters_present_flag: 0
0 … = pic_struct_present_flag: 0
… 1… = bitstream_restriction_flag: 1
… .1.. = motion_vectors_over_pic_boundaries_flag: 1
… . .1. = max_bytes_per_pic_denom: 0
… . ..1 = max_bits_per_mb_denom: 0
0001 000. = max_mv_length_horizontal: 7
… . ..0 0111 . … = log2_max_mv_length_vertical: 6
… 1… = num_reorder_frames: 0
… .010 = max_dec_frame_buffering: 1
1… . … = rbsp_stop_bit: 1
.000 0000 = rbsp_trailing_bits: 0
The device sps that can decode are as follows, and the actual profile_level_id is 42001f
H264 NAL Unit Payload
Profile_idc: Baseline profile (66)
0 … = Constraint_set0_flag: 0
0 … = Constraint_set1_flag: 0
0 . … = Constraint_set2_flag: 0
0 … = Constraint_set3_flag: 0
… 0… = Constraint_set4_flag: 0
… .0.. = Constraint_set5_flag: 0
… . .00 = Reserved_zero_2bits: 0
Level_id: 51 [Level 5.1 240 Mb/s]
1… . … = seq_parameter_set_id: 0
.1.. . … = log2_max_frame_num_minus4: 0
..1. . … = pic_order_cnt_type: 0
…0 0111 = log2_max_pic_order_cnt_lsb_minus4: 6
010. . … = num_ref_frames: 1
0 … = gaps_in_frame_num_value_allowed_flag: 0
… 0000 0010 1000 0… . … = pic_width_in_mbs_minus1: 79
.000 0010 1101 . … = pic_height_in_map_units_minus1: 44
… 1… = frame_mbs_only_flag: 1
… .1.. = direct_8x8_inference_flag: 1
… . .0. = frame_cropping_flag: 0
… . ..1 = vui_parameters_present_flag: 1
0 … = aspect_ratio_info_present_flag: 0
0 … = overscan_info_present_flag: 0
..1. . … = video_signal_type_present_flag: 1
…1 01.. = video_format: Unspecified video format (5)
… . .0. = video_full_range_flag: 0
… . ..1 = colour_description_present_flag: 1
0000 0001 = colour_primaries: 1
0000 0001 = transfer_characteristics: 1
0000 0001 = matrix_coefficients: 1
1… . … = chroma_loc_info_present_flag: 1
.1.. . … = chroma_sample_loc_type_top_field: 0
..1. . … = chroma_sample_loc_type_bottom_field: 0
…1 . … = timing_info_present_flag: 1
… 0000 0000 0000 0000 0000 0000 0000 0001 . … = num_units_in_tick: 1
… 0000 0000 0000 0000 0000 0000 0011 0010 . … = time_scale: 50
… 0… = fixed_frame_rate_flag: 0
… .0.. = nal_hrd_parameters_present_flag: 0
… . .1. = vcl_hrd_parameters_present_flag: 1
… . ..1 = cpb_cnt_minus1: 0
0111 . … = bit_rate_scale: 7
… 1001 = cpb_size_scale: 9
0000 0011 1110 1… = bit_rate_value_minus1: 124
… .000 0000 0101 1101 11.. = cpb_size_value_minus1: 374
… . .1. = cbr_flag: 1
… . ..1 1111 . … = initial_cpb_removal_delay_length_minus1: 31
… 1111 1… . … = cpb_removal_delay_length_minus1: 31
.111 11.. = dpb_output_delay_length_minus11: 31
… . .00 000. . … = time_offset_length: 0
0 … = low_delay_hrd_flag: 0
… 1… = pic_struct_present_flag: 1
… .0.. = bitstream_restriction_flag: 0
… . .1. = rbsp_stop_bit: 1
… . ..0 = rbsp_trailing_bits: 0
What are the differences between them that cause errors and inability to decode Chrome WebRTC?When unable to decode, the browser outputs the following log:
Received H.264-IDR frame (SPS: 0, PPS: 0). Treating as key frame since WebRTC-SpsPpsIdrIsH264Keyframe is disabled
third_party\webrtc\modules\video_coding\codecs\h264\h264_decoder_impl.cc:404] avcodec_send_packet error: -1094995529。
Also, by connecting the device’s RTSP stream to the janus.streaming plugin, it can be played on Chrome WebRTC, both SPS and PPS are the same. In terms of video encoding processing,what is the difference between the janus streaming plugin and the Janus SIP plugin. I am using janus.js. Is there any way to find the reason why it cannot be decoded.
Neither plugin performs any form of video processing, since Janus doesn’t touch the media. The main difference between Streaming and SIP plugins is that the former crafts the SDP invite offer, and the user answers: not sure how you’re using SIP to set up the session instead. You may want to check the differences in the SDPs in the two cases.