Initializing DeckLink fails for 1080i5000 on 2.3.0 LTS

General
, , ,
1

Hi there,

we stumbled upon a strange bug in our setup. Initializing the DeckLink card fails under some circumstances (1080i5000 and cold boot) but is perfectly possible under other circumstances (e.g. 1080p5000 or after a warm reboot)

We have a Win10 machine with a DeckLink Duo 2, current BMD 11.6 drivers and Server 2.3.0 LTS. If you take a look at the Caspar log files you will see, that in case of a cold boot and DeckLink format 1080i5000, it hangs somewhere during checking the reference signal, because the line “Reference signal: not detected” is missing in the first case.

1) After cold boot

[2020-10-21 17:32:50.511] [info]    ############################################################################
[2020-10-21 17:32:50.513] [info]    CasparCG Server is distributed by the Swedish Broadcasting Corporation (SVT)
[2020-10-21 17:32:50.514] [info]    under the GNU General Public License GPLv3 or higher.
[2020-10-21 17:32:50.514] [info]    Please see LICENSE.TXT for details.
[2020-10-21 17:32:50.514] [info]    http://www.casparcg.com/
[2020-10-21 17:32:50.514] [info]    ############################################################################
[2020-10-21 17:32:50.514] [info]    Starting CasparCG Video and Graphics Playout Server 2.3.0 a4a7c053 Dev
[2020-10-21 17:32:51.122] [info]    Initializing OpenGL Device.
[2020-10-21 17:32:51.128] [info]    Initialized OpenGL 4.5.0 NVIDIA 442.50 NVIDIA Corporation
[2020-10-21 17:32:51.139] [info]    Initialized ffmpeg module.
[2020-10-21 17:32:51.140] [info]    Initialized oal module.
[2020-10-21 17:32:51.140] [info]    Initialized decklink module.
[2020-10-21 17:32:51.141] [info]    Initialized screen module.
[2020-10-21 17:32:51.143] [info]    Initialized newtek module.
[2020-10-21 17:32:51.321] [info]    Initialized html module.
[2020-10-21 17:32:51.322] [info]    Flash support is disabled
[2020-10-21 17:32:51.322] [info]    Initialized flash module.
[2020-10-21 17:32:51.328] [info]    Initialized bluefish module.
[2020-10-21 17:32:51.328] [info]    Initialized image module.
[2020-10-21 17:32:51.329] [info]    "C:/VTURE\ingest\dist\programs\server\casparcg_3\casparcg.config":
[2020-10-21 17:32:51.329] [info]    -----------------------------------------
[2020-10-21 17:32:51.329] [info]    <?xml version="1.0" encoding="utf-8"?>
[2020-10-21 17:32:51.329] [info]    <configuration>
[2020-10-21 17:32:51.329] [info]       <paths>
[2020-10-21 17:32:51.329] [info]          <media-path>media/</media-path>
[2020-10-21 17:32:51.329] [info]          <log-path>log/</log-path>
[2020-10-21 17:32:51.329] [info]          <data-path>data/</data-path>
[2020-10-21 17:32:51.329] [info]          <template-path>template/</template-path>
[2020-10-21 17:32:51.329] [info]       </paths>
[2020-10-21 17:32:51.329] [info]       <lock-clear-phrase>secret</lock-clear-phrase>
[2020-10-21 17:32:51.329] [info]       <channels>
[2020-10-21 17:32:51.329] [info]          <channel>
[2020-10-21 17:32:51.329] [info]             <video-mode>1080i5000</video-mode>
[2020-10-21 17:32:51.329] [info]             <consumers>
[2020-10-21 17:32:51.329] [info]                <decklink>
[2020-10-21 17:32:51.329] [info]                   <device>3</device>
[2020-10-21 17:32:51.329] [info]                   <embedded-audio>true</embedded-audio>
[2020-10-21 17:32:51.329] [info]                   <latency>normal</latency>
[2020-10-21 17:32:51.329] [info]                   <buffer-depth>3</buffer-depth>
[2020-10-21 17:32:51.329] [info]                </decklink>
[2020-10-21 17:32:51.329] [info]             </consumers>
[2020-10-21 17:32:51.329] [info]          </channel>
[2020-10-21 17:32:51.329] [info]       </channels>
[2020-10-21 17:32:51.329] [info]       <controllers>
[2020-10-21 17:32:51.329] [info]          <tcp>
[2020-10-21 17:32:51.329] [info]             <port>5253</port>
[2020-10-21 17:32:51.329] [info]             <protocol>AMCP</protocol>
[2020-10-21 17:32:51.329] [info]          </tcp>
[2020-10-21 17:32:51.329] [info]       </controllers>
[2020-10-21 17:32:51.329] [info]       <amcp>
[2020-10-21 17:32:51.329] [info]          <media-server>
[2020-10-21 17:32:51.329] [info]             <host>localhost</host>
[2020-10-21 17:32:51.329] [info]             <port>8000</port>
[2020-10-21 17:32:51.329] [info]          </media-server>
[2020-10-21 17:32:51.329] [info]       </amcp>
[2020-10-21 17:32:51.329] [info]       <osc>
[2020-10-21 17:32:51.329] [info]          <default-port>6253</default-port>
[2020-10-21 17:32:51.329] [info]       </osc>
[2020-10-21 17:32:51.329] [info]    </configuration>
[2020-10-21 17:32:51.329] [info]    -----------------------------------------
[2020-10-21 17:32:51.345] [info]    Initialized OpenGL Accelerated GPU Image Mixer for channel 1
[2020-10-21 17:32:51.347] [info]    video_channel[1|1080i5000] Successfully Initialized.
[2020-10-21 17:32:51.373] [info]    DeckLink Duo 2 [1-3|1080i5000] Enabled embedded-audio.
[2020-10-21 17:32:51.373] [info]    DeckLink Duo 2 [1-3|1080i5000] Disabled low-latency mode.
[2020-10-21 17:32:51.373] [error]   DeckLink Duo 2 [1-3|1080i5000] Failed to enable external keyer.
[2020-10-21 17:32:51.375] [info]    DeckLink Duo 2 [1-3|1080i5000] Initialized.
[2020-10-21 17:32:51.375] [info]    Initialized channels.
[2020-10-21 17:32:51.377] [info]    Initialized controllers.
[2020-10-21 17:32:51.377] [info]    Initialized osc.
[2020-10-21 17:32:51.828] [info]    async_event_server[:5253] Accepted connection from 127.0.0.1 (1 connections).
[2020-10-21 17:33:00.621] [info]    Received message from 127.0.0.1: REQ qm8bpeh LOAD 1-1 "E://Testclips/Test.mxf"\r\n
[2020-10-21 17:33:00.630] [info]    Sent message to 127.0.0.1:RES qm8bpeh 202 LOAD OK\r\n
[2020-10-21 17:33:00.750] [warning] [ffmpeg] [mxf @ 0000022E6791A040] Stream #0: not enough frames to estimate rate; consider increasing probesize
[2020-10-21 17:33:00.750] [warning] 
[2020-10-21 17:33:00.822] [warning] [ffmpeg] [Parsed_amerge_0 @ 0000022E723CCAC0] No channel layout for input 1
[2020-10-21 17:33:00.822] [warning] 
[2020-10-21 17:33:00.823] [warning] [ffmpeg] [Parsed_amerge_0 @ 0000022E723CCAC0] Input channel layouts overlap: output layout will be determined by the number of distinct input channels
[2020-10-21 17:33:00.823] [warning]

It says initialitized and the server receives commands but the decklink card isn’t doing anything. If you press “q” in the server windows even the uninitializing hangs.

2) After warm reboot

[2020-10-21 17:35:13.828] [info]    ############################################################################
[2020-10-21 17:35:13.829] [info]    CasparCG Server is distributed by the Swedish Broadcasting Corporation (SVT)
[2020-10-21 17:35:13.830] [info]    under the GNU General Public License GPLv3 or higher.
[2020-10-21 17:35:13.830] [info]    Please see LICENSE.TXT for details.
[2020-10-21 17:35:13.830] [info]    http://www.casparcg.com/
[2020-10-21 17:35:13.830] [info]    ############################################################################
[2020-10-21 17:35:13.830] [info]    Starting CasparCG Video and Graphics Playout Server 2.3.0 a4a7c053 Dev
[2020-10-21 17:35:14.467] [info]    Initializing OpenGL Device.
[2020-10-21 17:35:14.472] [info]    Initialized OpenGL 4.5.0 NVIDIA 442.50 NVIDIA Corporation
[2020-10-21 17:35:14.484] [info]    Initialized ffmpeg module.
[2020-10-21 17:35:14.485] [info]    Initialized oal module.
[2020-10-21 17:35:14.485] [info]    Initialized decklink module.
[2020-10-21 17:35:14.486] [info]    Initialized screen module.
[2020-10-21 17:35:14.487] [info]    Initialized newtek module.
[2020-10-21 17:35:14.668] [info]    Initialized html module.
[2020-10-21 17:35:14.669] [info]    Flash support is disabled
[2020-10-21 17:35:14.669] [info]    Initialized flash module.
[2020-10-21 17:35:14.676] [info]    Initialized bluefish module.
[2020-10-21 17:35:14.676] [info]    Initialized image module.
[2020-10-21 17:35:14.676] [info]    "C:/VTURE\ingest\dist\programs\server\casparcg_3\casparcg.config":
[2020-10-21 17:35:14.676] [info]    -----------------------------------------
[2020-10-21 17:35:14.676] [info]    <?xml version="1.0" encoding="utf-8"?>
[2020-10-21 17:35:14.676] [info]    <configuration>
[2020-10-21 17:35:14.676] [info]       <paths>
[2020-10-21 17:35:14.676] [info]          <media-path>media/</media-path>
[2020-10-21 17:35:14.676] [info]          <log-path>log/</log-path>
[2020-10-21 17:35:14.676] [info]          <data-path>data/</data-path>
[2020-10-21 17:35:14.676] [info]          <template-path>template/</template-path>
[2020-10-21 17:35:14.676] [info]       </paths>
[2020-10-21 17:35:14.676] [info]       <lock-clear-phrase>secret</lock-clear-phrase>
[2020-10-21 17:35:14.676] [info]       <channels>
[2020-10-21 17:35:14.676] [info]          <channel>
[2020-10-21 17:35:14.676] [info]             <video-mode>1080i5000</video-mode>
[2020-10-21 17:35:14.676] [info]             <consumers>
[2020-10-21 17:35:14.676] [info]                <decklink>
[2020-10-21 17:35:14.676] [info]                   <device>3</device>
[2020-10-21 17:35:14.676] [info]                   <embedded-audio>true</embedded-audio>
[2020-10-21 17:35:14.676] [info]                   <latency>normal</latency>
[2020-10-21 17:35:14.676] [info]                   <buffer-depth>3</buffer-depth>
[2020-10-21 17:35:14.676] [info]                </decklink>
[2020-10-21 17:35:14.676] [info]             </consumers>
[2020-10-21 17:35:14.676] [info]          </channel>
[2020-10-21 17:35:14.676] [info]       </channels>
[2020-10-21 17:35:14.676] [info]       <controllers>
[2020-10-21 17:35:14.676] [info]          <tcp>
[2020-10-21 17:35:14.676] [info]             <port>5253</port>
[2020-10-21 17:35:14.676] [info]             <protocol>AMCP</protocol>
[2020-10-21 17:35:14.676] [info]          </tcp>
[2020-10-21 17:35:14.676] [info]       </controllers>
[2020-10-21 17:35:14.676] [info]       <amcp>
[2020-10-21 17:35:14.676] [info]          <media-server>
[2020-10-21 17:35:14.676] [info]             <host>localhost</host>
[2020-10-21 17:35:14.676] [info]             <port>8000</port>
[2020-10-21 17:35:14.676] [info]          </media-server>
[2020-10-21 17:35:14.676] [info]       </amcp>
[2020-10-21 17:35:14.676] [info]       <osc>
[2020-10-21 17:35:14.676] [info]          <default-port>6253</default-port>
[2020-10-21 17:35:14.676] [info]       </osc>
[2020-10-21 17:35:14.676] [info]    </configuration>
[2020-10-21 17:35:14.676] [info]    -----------------------------------------
[2020-10-21 17:35:14.700] [info]    Initialized OpenGL Accelerated GPU Image Mixer for channel 1
[2020-10-21 17:35:14.702] [info]    video_channel[1|1080i5000] Successfully Initialized.
[2020-10-21 17:35:14.726] [info]    DeckLink Duo 2 [1-3|1080i5000] Enabled embedded-audio.
[2020-10-21 17:35:14.726] [info]    DeckLink Duo 2 [1-3|1080i5000] Disabled low-latency mode.
[2020-10-21 17:35:14.726] [error]   DeckLink Duo 2 [1-3|1080i5000] Failed to enable external keyer.
[2020-10-21 17:35:14.728] [info]    DeckLink Duo 2 [1-3|1080i5000] Initialized.
[2020-10-21 17:35:14.728] [info]    Initialized channels.
[2020-10-21 17:35:14.730] [info]    Initialized controllers.
[2020-10-21 17:35:14.730] [info]    Initialized osc.
[2020-10-21 17:35:14.832] [info]    **DeckLink Duo 2 [1-3|1080i5000] Reference signal: not detected.**
[2020-10-21 17:35:15.120] [info]    async_event_server[:5253] Accepted connection from 127.0.0.1 (1 connections).
[2020-10-21 17:35:28.039] [info]    Received message from 127.0.0.1: REQ 9ooadt3 LOAD 1-1 "E://Testclips/Test.mxf"\r\n
[2020-10-21 17:35:28.049] [info]    Sent message to 127.0.0.1:RES 9ooadt3 202 LOAD OK\r\n
[2020-10-21 17:35:28.146] [warning] [ffmpeg] [mxf @ 0000029F51FCA040] Stream #0: not enough frames to estimate rate; consider increasing probesize
[2020-10-21 17:35:28.146] [warning] 
[2020-10-21 17:35:28.171] [warning] [ffmpeg] [Parsed_amerge_0 @ 0000029F5CA7DC80] No channel layout for input 1
[2020-10-21 17:35:28.171] [warning] 
[2020-10-21 17:35:28.171] [warning] [ffmpeg] [Parsed_amerge_0 @ 0000029F5CA7DC80] Input channel layouts overlap: output layout will be determined by the number of distinct input channels
[2020-10-21 17:35:28.171] [warning]

After a reboot the DeckLink card is working perfectly. Another workaround is to choose other formats like 720p5000 oder 1080p5000, kill the server process and start the server again with 1080i5000. The ref signal part will appear in the logs after that.

We suspect there is something going wrong in the code. The class “reference_signal_detector” seems to print no result in case 1).

class reference_signal_detector
{
    com_iface_ptr<IDeckLinkOutput> output_;
    BMDReferenceStatus             last_reference_status_ = static_cast<BMDReferenceStatus>(-1);
  public:
    reference_signal_detector(const com_iface_ptr<IDeckLinkOutput>& output)
        : output_(output)
    {
    }
    template <typename Print>
    void detect_change(const Print& print)
    {
        BMDReferenceStatus reference_status;
        if (output_->GetReferenceStatus(&reference_status) != S_OK) {
            CASPAR_LOG(error) << print() << L" Reference signal: failed while querying status";
        } else if (reference_status != last_reference_status_) {
            last_reference_status_ = reference_status;
            if (reference_status == 0)
                CASPAR_LOG(info) << print() << L" Reference signal: not detected.";
            else if (reference_status & bmdReferenceNotSupportedByHardware)
                CASPAR_LOG(info) << print() << L" Reference signal: not supported by hardware.";
            else if (reference_status & bmdReferenceLocked)
                CASPAR_LOG(info) << print() << L" Reference signal: locked.";
            else
                CASPAR_LOG(info) << print() << L" Reference signal: Unhandled enum bitfield: " << reference_status;
        }
    }
};

Has anyone experienced the same with 2.3.0 LTS, BMD driver 11.6, DeckLink Duo 2 and 1080i5000?

Best Regards!

2

It says something about not able to initialize the external keyer. I usually use a channel section like this:

<video-mode>1080i5000</video-mode>
<straight-alpha-output>false</straight-alpha-output>
<consumers>
	<decklink>
		<device>1</device>
		<embedded-audio>true</embedded-audio>
		<channel-layout>stereo</channel-layout>
		<latency>normal</latency>
		<keyer>external</keyer>
		<key-only>false</key-only>
		<buffer-depth>3</buffer-depth>
		<custom-allocator>true</custom-allocator>
	</decklink>            
</consumers>
3

We tried different configs with all possible values for the keyer parameter. The behaviour didn’t change at all. By the way, there seems to be the same issue on Caspar 2.2 if that helps for tracking down the error.

4

Hello,
I am also facing the same problem with Server 2.3.3 LTS: after a cold boot (from OFF state), the decklink card does not output anything. After a reboot, everything works fine.
Difference between working and not working : when it does not work, the following line is not present in the logs:

**DeckLink Duo 2 [1-3|1080i5000] Reference signal: not detected.**

I have tried different decklink cards, different slots, different BIOS settings but the only way to make it work is to reboot.

Hardware is a (rather old) SuperMicro X9DAL, dual CPU (Xeon E5-2430)

Any suggestion or solution since 2020 ?

5

I had similar problems with a Decklink Duo 2 in a Z420; the first time it booted (I even made sure there was no power at all left on the PSU) it wouldn’t add the decklink properly via AMCP.

Then if I restarted the server and added the decklink again it would work.

We tried to pinpoint the issue, but it was quite specific to this configuration: Z420 with Duo2; The Extreme cards worked fine, and other HP machines worked fine as well.

We still don’t know how to fix it and we kind of avoid that combination of hardware for now.

6

I sometimes also have the problem, that it does not initialize the Decklink card correctly on the first start of the server, but I only need to restart Caspar, not the whole computer. So it is not an big issue to me. It’s an ASUS Mainboard TUF X299 MARK 1 with an Intel CPU Core i9-10900X 3.7 GHz.

Have you tried to set the resolution also in the control panel applet?

7

as I wrote in another topic, I have found the following workaround for this problem :
I launch a ffmpeg script (manually or at login with task scheduler) to initialize the decklink outputs, for example:
ffmpeg -f lavfi -i testsrc -t 15 -s 1920x1080 -f decklink -format_code Hi50 -pix_fmt uyvy422 "DeckLink Duo (1)"
(repeated for each output).
The casparcg server is launched after that script and the decklink outputs work flawlessly.