CasparCG-based MAV/EVS replacement [BETA]


#1

CasparCG 2.3.0 DEV with added replay module created by Jan Starzak (@jstarzak)

Release:

Quick and dirty client (for reference only):

This is now my free time project but I will try to polish it. Please fill free to submit ideas, issues or pull requests.

Original idea (copied from old forum post by @jstarzak)

I’d like to share with the community (and quite frankly, also seek some help here) my project which is a replay system based on CasparCG (version 2.0.3, because 2.1.0 would blue-screen my machine). In order to be truly replay-worthy, I decided to add my own consumer and producer to the Caspar framework. The producer and consumer are based on the libjpeg-turbo library, and as such use JPEG as a codec. This produces large files, but is easy to handle in terms of software development and works quickly enough to allow doing multiple input and output channels on the same machine.

The system uses “replay buffers” that are split into two pieces: the video essence file and an index file. This allows simultaneous access to the buffer by a single input and multiple outputs. The video essence file is a stream of JPEG images (one image per every field/frame) and the index is simply a short header and a stream of byte offsets in the video essence file for each field/frame.

Other than that, the rest of the Caspar server is relatively unchanged (I had to modify the code a little bit to make MAV files show up after “CLS” command) - I think that shows Caspar’s remarkable flexibility.

I have tested the system in both PAL and 1080i50, and it does work - with some quirks (more on that later). I haven’t tested this for systems other than 50Hz (59.94Hz and LFF modes might be tricky). Hardware performance is key here, but I believe all servers to spec to the SVT hardware recommendations should handle easily at leas one input and one output channel.

Here is how to use the custom “MAV” version

The minimal configuration for the MAV edition requires at least two channels: one for ingest and another one for output. This usually means two Decklink cards (or a Decklink DUO/QUAD): one for input, and another one for output. Beyond that you can have multiple input and output channels, in various configurations. Always have only one replay buffer per ingest channel.

In the configuration file:
Configure all ingest channels as empty channels with proper system modes (f.g. PAL or 1080i50000). Leave the consumers tag empty.
Configure all output channels as porper channels, like you normally would, f.g. with Decklink consumers.
Start the server

Play the input from your ingest decklink card on your ingest channel:

play 1 decklink 1

Add the replay consumer to your ingest channel:

add 1 replay test-replay

The “test-replay” is a file-name for your replay buffer. Two files will be created in your CasparCG “media” folder: a TEST-REPLAY.MAV file and a TEST-REPLAY.IDX file. The first one is the video essence, and the second one is a manifest and index file. Should you copy your replay buffers, always copy both files.
Play your replay buffer on your output channel:

play 2 test-replay

Voila! You have your own replay system.

You can seek within your replay buffer using the CALL ACMP 2.0 command:
This will seek to the 100th frame in the replay buffer:

call 2 seek 100

This will seek to the 100th frame counting from the end of the replay buffer:

call 2 seek |100

This will move the playhead of your replay 50 frames forward:

call 2 seek +50

This will move the playhead of your replay 50 frames backward:

call 2 seek -50

You can also change the speed of your playback:
Will make the playback go at normal speed

call 2 speed 1

Will make the playback go backwards at twice the normal speed

call 2 speed -2

Will make the playback go at half the speed

call 2 speed 0.5

Will make the playback go at 3/4 of normal speed

call 2 speed 0.75

The consumer can use custom subsampling and JPEG quality settings. Use sytanx:

ADD 1 REPLAY FILE_NAME [QUALITY (int:1-100)] [SUBSAMPLING (444|422|420|411)]

You can also use CasparCG’s auto-play feature along with transitions by using the LENGTH param. You can combine SEEK, LENGTH, SPEED and AUDIO in the PLAY/LOAD/LOADBG command:
This will make the playback start at 100th frame from the begining, play at 0.75x speed, and end playback after 100 frames (buffer time). Any transition (if specified using a f.g. LOADBG (…) AUTO MIX 25) to another clip will happen in the last 25 frames (output time):

play 2 test-replay seek 100 speed 0.5 length 100 audio 1

File format

For every replay there are two files.
Index file has 52 bytes of header followed by 64-bit indexes - one for every frame.
Data file contains (for every frame): 32-bit length of audio data, then raw audio data, then JPEG compressed video data.


SEEK and play a growing recording file
#2

Seems really interesting !
On what version did you based your dev ?


#3

Had some time today to test.

only on my laptop soo not the best test in the world :wink:

everything seams to work nicely !
build a little client for it today and everything works like expected.

only thing though is play speed 1

when i

call 2 speed 1

the speed is different all the time. (could be performance off my laptop)
is this a known issue on your side

nice work


#4

@Maurice78 I will try to check speed issue. Please check in log whether Libjpegturbo library is used - it should be noted in early stage of start process.
@timmalos It is based on current GitHub checkout of master branch and merged with adapted latest replay module from version 2.1 MAV 9.


#5

I think it does

[2019-11-22 14:24:51.493] [info] ############################################################################
[2019-11-22 14:24:51.517] [info] CasparCG Server is distributed by the Swedish Broadcasting Corporation (SVT)
[2019-11-22 14:24:51.517] [info] under the GNU General Public License GPLv3 or higher.
[2019-11-22 14:24:51.517] [info] Please see LICENSE.TXT for details.
[2019-11-22 14:24:51.517] [info] http://www.casparcg.com/
[2019-11-22 14:24:51.517] [info] ############################################################################
[2019-11-22 14:24:51.517] [info] Starting CasparCG Video and Graphics Playout Server 2.3.0 745335667 MAV 10 Dev
[2019-11-22 14:24:53.514] [info] Initializing OpenGL Device.
[2019-11-22 14:24:53.607] [info] Initialized OpenGL 4.5.0 NVIDIA 385.73 NVIDIA Corporation
[2019-11-22 14:24:53.610] [info] [replay] JPEG lib version: libjpeg-turbo 62
[2019-11-22 14:24:53.625] [info] Initialized replay module.
[2019-11-22 14:24:53.847] [info] Initialized ffmpeg module.
[2019-11-22 14:24:53.857] [info] Initialized oal module.
[2019-11-22 14:24:53.866] [info] Initialized decklink module.
[2019-11-22 14:24:53.866] [info] Initialized screen module.
[2019-11-22 14:24:53.873] [info] Initialized newtek module.
[2019-11-22 14:24:59.174] [info] Initialized html module.
[2019-11-22 14:25:01.075] [info] Initialized flash module.
[2019-11-22 14:25:01.106] [info] Initialized bluefish module.
[2019-11-22 14:25:01.106] [info] Initialized image module.
[2019-11-22 14:25:01.106] [info] “D:/casparcg-server-2.3.0-mav10-dev-windows\casparcg.config”:
[2019-11-22 14:25:01.106] [info] -----------------------------------------
[2019-11-22 14:25:01.106] [info] <?xml version="1.0" encoding="utf-8"?>
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] media/
[2019-11-22 14:25:01.106] [info] log/
[2019-11-22 14:25:01.106] [info] data/
[2019-11-22 14:25:01.106] [info] template/
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] secret
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] 1080i5000
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] 1080i5000
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] 3
[2019-11-22 14:25:01.106] [info] true
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] 5250
[2019-11-22 14:25:01.106] [info] AMCP
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] localhost
[2019-11-22 14:25:01.106] [info] 8000
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] 6250
[2019-11-22 14:25:01.106] [info] false
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] 127.0.0.1
[2019-11-22 14:25:01.106] [info] 5253
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info]
[2019-11-22 14:25:01.106] [info] -----------------------------------------
[2019-11-22 14:25:01.114] [info] Initialized OpenGL Accelerated GPU Image Mixer for channel 1
[2019-11-22 14:25:01.157] [info] video_channel[1|1080i5000] Successfully Initialized.
[2019-11-22 14:25:01.158] [info] Initialized OpenGL Accelerated GPU Image Mixer for channel 2
[2019-11-22 14:25:01.159] [info] video_channel[2|1080i5000] Successfully Initialized.
[2019-11-22 14:25:01.835] [warning] Device does not support video-format: 1080i50
[2019-11-22 14:25:01.837] [info] DeckLink SDI [2-3|1080i5000] Enabled embedded-audio.
[2019-11-22 14:25:01.837] [info] DeckLink SDI [2-3|1080i5000] Disabled low-latency mode.
[2019-11-22 14:25:01.837] [error] DeckLink SDI [2-3|1080i5000] Failed to enable external keyer.
[2019-11-22 14:25:01.863] [info] DeckLink SDI [2-3|1080i5000] Initialized.
[2019-11-22 14:25:01.863] [info] Initialized channels.
[2019-11-22 14:25:01.866] [info] Initialized controllers.
[2019-11-22 14:25:01.866] [info] Initialized osc.
[2019-11-22 14:25:01.994] [info] DeckLink SDI [2-3|1080i5000] Reference signal: not detected.


#6

First of all,thanks to @ jstarzak and @krzyc for your work. This is truly amazing.

I have been using the Mav server for a while now and I have a pretty solid client that “fixes” some of the unstability problems Mav 9 had.

Mav10 looks very promising and it is so much more stable already.

In both versions I still crash the decklink output whenever speed 0 and another speed order are given too fast. Also the speed 1 in MAV10 fluctuates and goes usually slower, as @Maurice78 pointed out. That did not happen that often in Mav9, and when that happened the server showed an “Underflow” message.

I have done extensive tests with CasparMav in several types of machines and right now the system is very promising, however, I am in search of a developer that could help create an ‘easy’ way to convert the Mav files into anything else. Ideally through a custom ffmpeg build.

The secret sauce must be here:

“Index file has 52 bytes of header followed by 32-bit indexes - one for every frame.
Data file contains (for every frame): 32-bit length of audio data, then raw audio data, then JPEG compressed video data.”

If you manage to convert a mav file don’t hesitate to contact me please ; )


#7

@Sidon I will try to look at crash issues.

Regarding ffmpeg demuxer - here is a quick and dirty demuxer for MAV files: