Janus recording, video on/off broadcast manipulation and post processing

General
1

I have generally one huge personal issue I need to solve(videoroom plugin legacy):

I would like to replace all the parts in a video that has actually no video recorded (video turned off/video not sent/cam off(resulting still image), audio still sending).

To solve this, I can think of 2 ways:

  1. (Interesting) Modify Janus server: If sending to janus_videoroom.c “configure” command with “video:false”, start replacing the video stream with default local image/source/other video, file from the local server (by modifying the janus_videoroom.c), at least for recording purpose if not broadcasting it.

  2. (Not bad)Doing post processing with ffmpeg, and detecting and replacing the missing video with an image or black screen(I am sick of searching that).

Anyways since the 2. way is not about Janus, can you please tell me if the 1. way is possible.

In short: The interesting solution would be to display default image/video/loop, and broadcast it at the same time, if the video is off(on the server side: I actually not turning the video off, but replacing the source with a local file), is there a way to change video source from the server side, in Janus, to a local file modifying the videoroom plugin or other files?

Any help would be very appreciated.

I feel a bit weird that I could not find an answer from the web.

2

I think I need to use the canvas. It’s the only way to solve all the problems with the video, including aspect ratio changes etc. Was doing things the wrong way around.

3

It all depends on what you mean by “video off”.
If the browser local track is being muted then you’ll end up with a mjr with blank frames.
If the local encoder is stopped through Transceiver API you’ll have a mjr woth gaps that pp-rec can fill.
Otoh, a configure video: false will still record a whole mjr, since only the relaying to the subscribers is stopped.
Renegotiating and disabling video will instead stop the recording mjr and eventually start a new one when video is re enabled.