Huge latency: architectural reasons?

Hi!

I’m learning about CasparCG server and the question for the moment is “latency”.
Created docker image which builds CasparCG server from sources and then runs it. Added some logs for clarity. Now, what I’m seeing is:

For the scheme: Decklink → server → Decklink

  • After receiving a frame the module stores it into internal buffer, where the frame spends 196 milliseconds until it’s requested
  • The frame gets requested when Decklink consumer schedules a frame into the card, all processing takes roughly 1-2 milliseconds
  • Then the frame spends 80 milliseconds before Decklink reports completion

Now, the question: what might be a reason for such a long waiting on the first stage? I understand that the system is 160 + 36, where 36 milliseconds is a distance between input and output card ticks. But where this 160 ms comes from? If we already have a buffer for 2 frames in Docklink output what’s the reason for having input frame to wait?

Thanks

PS: in case this helps, here’s logs (I have added some logging code to have this). There’s two runs made with a bit different logs. Both show 250-280 millisecond for a frame but details differ.

[2026-10-10 00:57:24.675] [info]    (213)VideoInputFrameArrived, n=4998
[2026-10-10 00:57:24.675] [info]    (213)       pts=100000000, dur=40000

[2026-10-10 00:57:24.871] [info]    (111) decklink_consumer:SceduledVideoCompleted
[2026-10-10 00:57:24.871] [info]    (111)   decklink_consumer:pop popped frame ts=100000000
[2026-10-10 00:57:24.871] [info]    (111)   decklink_consumer:pop popped frame ts=100000000
[2026-10-10 00:57:24.871] [info]    (111) decklink_consumer:SceduledVideoCompleted BMD=2908000

[2026-10-10 00:57:24.872] [info]    (111) decklink_consumer:SceduledVideoCompleted scheduling ts=100000000 for BMD=2910000
[2026-10-10 00:57:24.872] [info]    (111) decklink_consumer:schedule_next_video BMD=2910000

[2026-10-10 00:57:24.951] [info]    (111) decklink_consumer:SceduledVideoCompleted BMD=2910000

And second run (with some changes in logs):

[2026-10-12 16:47:15.184] [info]    (211)VideoInputFrameArrived, n=3474
[2026-10-12 16:47:15.184] [info]    (211)       pts=69520000, dur=40000
[2026-10-12 16:47:15.185] [info]    (211)decklink:producer buffering frame ts=69520000/1, buffer size=2
[2026-10-12 16:47:15.186] [info]    (211)decklink:producer buffering frame ts=69520000/2, buffer size=3

[2026-10-12 16:47:15.231] [info]    (109) decklink_consumer:ScheduledVideoCompleted 000 BMD=2106000
[2026-10-12 16:47:15.231] [info]    (96) stage_impl()
[2026-10-12 16:47:15.231] [info]    (96)decklink:producer output buffer size=3
[2026-10-12 16:47:15.231] [info]    (96)decklink:producer popped frame ts=69520000/1
[2026-10-12 16:47:15.231] [info]    (96) producer:receive_impl, frame ts=69520000/1
[2026-10-12 16:47:15.231] [info]     layer:receive frame ts=69520000/1
[2026-10-12 16:47:15.231] [info]    (96)decklink:producer output buffer size=2
[2026-10-12 16:47:15.231] [info]    (96)decklink:producer popped frame ts=69520000/2
[2026-10-12 16:47:15.231] [info]    (96) producer:receive_impl, frame ts=69520000/2
[2026-10-12 16:47:15.231] [info]     layer:receive frame ts=69520000/2
[2026-10-12 16:47:15.231] [info]    (96) stage_impl() END
[2026-10-12 16:47:15.271] [info]    (98) output sending 1 ts=69520000 ... 
[2026-10-12 16:47:15.271] [info]    (105) decklink_consumer:send. frame ts=69520000, field=1
[2026-10-12 16:47:15.271] [info]    (109) decklink_consumer:ScheduledVideoCompleted DONE

[2026-10-12 16:47:15.311] [info]    (98) output sending 2 ts=69520000 ... 
[2026-10-12 16:47:15.311] [info]    (105) decklink_consumer:send. frame ts=69520000, field=2
[2026-10-12 16:47:15.311] [info]    (105) decklink_consumer:send DONE. frame ts=69520000

[2026-10-12 16:47:15.351] [info]    (109)   decklink_consumer:pop popped frame ts=69520000
[2026-10-12 16:47:15.351] [info]    (109)   decklink_consumer:pop popped frame ts=69520000
[2026-10-12 16:47:15.352] [info]    (109)   decklink_consumer:ScheduledVideoCompleted 222 scheduling frame ts=69520000 for BMD=2111000
[2026-10-12 16:47:15.352] [info]    (109) decklink_consumer:schedule_next_video BMD=2111000
[2026-10-12 16:47:15.352] [info]    (109) decklink_consumer:ScheduledVideoCompleted DONE

[2026-10-12 16:47:15.431] [info]    (109) decklink_consumer:ScheduledVideoCompleted 000 BMD=2111000

Most of that time is going to be copying it to the GPU, compositing and mixing the layers, and copying back to CPU memory to output to the consumers. You could have a code spaces AI look into the subject further for more specific information

I think this boils down to the architectural decision of prioritising smooth output instead of low latency.

First question is what framerate and format are you doing? that can wildly affect what those millisecond numbers mean. And whether you are doing interlaced or progressive.

From the log it looks like its interlaced, and perhaps i50?

If so, then one thing to be aware of is that the decklink produces and consumes the fields in pairs. But internally we run progressive, consuming and compositing those frames in pairs so that we can keep the upper+lower aligned from input to output.

The compositing process is run pipelined, to minimise the risk of gpu pausing between channel ticks I have a feeling this adds a frame of delay there, as it means there is always one (field or frame?) in that pending state.

Then on the decklink consumer side the driver is buffering 3 or 4 frames, so that output stays smooth even when casparcg lags briefly, and the consumer has a buffer of 1 (or 2 when interlaced), ready for the next pull by the driver.
From what I have been playing with in another project, this could likely be cut down by a few frames, but at the risk of stutter when cpu spikes (eg when loading a clip)

On the producer side, it looks like we aren’t trying to buffer any frames, but do allow a buffer to fill up so that when the system does lag and the driver is trying to refill its buffer as fast as possible, the producer doesnt run out of frames. The buffer in the producer should really be hovering between 0-1 frames (or 0-2 when channel is interlaced) as a frame comes in then gets consumed.

Hopefully some of this helps, even though it isnt in any sort of order.

Thanks for the information!

The video mode is: 625i. Interlaced, yes.

Okey, I understand the system now.