Using janus and freeswitch together

Resources
1

Hello everyone,

We have developed a video calling application that utilizes the Janus videoroom plugin, and so far, the project has been running smoothly. However, our current goal is to enable simultaneous video recording. To achieve this, we have made the decision to integrate FreeSWITCH with Janus.

Our plan involves employing FreeSWITCH as a participant in subscriber mode. Specifically, we aim to retrieve an offer for everyone present in the room and then provide this offer to FreeSWITCH. Initially, this approach seemed promising. However, we’ve encountered a critical issue: when attempting to send both video and audio simultaneously, an “corrupted audio stream” error is thrown in the fs_cli.

It’s important to note that individually sending either video or audio works without triggering any errors. However, the challenge arises when both video and audio are sent together. This leads to the aforementioned error in the fs_cli.

In our quest to address this issue, we’ve drawn inspiration from the repository at GitHub - giavac/janus_freeswitch_integration: Integrating Janus and FreeSWITCH for an heterogeneous conferencing system. While this repository isn’t precisely aligned with our requirements, it is closely related to what we aim to achieve.

We are seeking insights into why this error occurs and exploring potential solutions. Any ideas or suggestions on how to resolve this specific issue would be immensely valuable. Thank you in advance for your assistance.

Here is freeswitch logs

[WARNING] switch_core_media.c:3210 Could not change to payload type 96, ignoring...
[DEBUG] switch_core_media.c:3187 alternate payload received (received 96, expecting 111)
[WARNING] switch_core_media.c:3210 Could not change to payload type 96, ignoring...
[ERR] mod_opus.c:931 Decoder Error: corrupted stream fs:4080 plc:false!
[DEBUG] switch_rtp.c:1930 rtcp_stats_init: audio ssrc[60862957] base_seq[476]
[DEBUG] switch_rtp.c:1930 rtcp_stats_init: audio ssrc[4200984672] base_seq[579]
[DEBUG] switch_rtp.c:1930 rtcp_stats_init: audio ssrc[60862957] base_seq[478]
[DEBUG] switch_rtp.c:1930 rtcp_stats_init: audio ssrc[4200984672] base_seq[580]
[DEBUG] switch_core_media.c:3187 alternate payload received (received 96, expecting 111)
[WARNING] switch_core_media.c:3210 Could not change to payload type 96, ignoring...
[ERR] mod_opus.c:931 Decoder Error: corrupted stream fs:4080 plc:false!

here is janus and freeswitch(verto) logs
(I have replaced IP addresses with 0.0.0.0 for security purposes.)
janus version: 0.x

2

I don’t think you can use WebRTC to get FS and Janus to communicate: to my knowledge, FS still doesn’t support muxing audio and video on the same port, while Janus mandates support for it. You may have to use RTP forwarders from Janus to push RTP to FS (possibly orchestrating a fake SIP call).

3

Thank you for your response. I appreciate it. I’m curious about your thoughts on this matter. Is the approach of performing composite recording on a separate server the correct method, or would using ffmpeg be a better choice in this case?

4

We have our own videomixer for composed recording, but usually just record to .mjr (Janus format) and then postprocess ex-post to put them all together.