Server crashing on media (or template) playback - AMD Driver issue?

quoole · March 11, 2024, 12:40am

[2024-03-11 00:11:52.362] [info]    Starting CasparCG Video and Graphics Playout Server 2.3.2 4de6d18f Dev
[2024-03-11 00:11:52.693] [info]    Initializing OpenGL Device.
[2024-03-11 00:11:52.706] [info]    Initialized OpenGL 4.5.0 Core Profile Context 22.11.2.221130 ATI Technologies Inc.
[2024-03-11 00:11:52.707] [info]    Initialized image module.
[2024-03-11 00:11:52.708] [info]    Initialized ffmpeg module.
[2024-03-11 00:11:52.708] [info]    Initialized oal module.
[2024-03-11 00:11:52.708] [info]    Initialized decklink module.
[2024-03-11 00:11:52.708] [info]    Initialized screen module.
[2024-03-11 00:11:52.708] [info]    Initialized newtek module.
[2024-03-11 00:11:52.793] [info]    Initialized html module.
[2024-03-11 00:11:52.793] [info]    Flash support is disabled
[2024-03-11 00:11:52.793] [info]    Initialized flash module.
[2024-03-11 00:11:52.794] [info]    Initialized bluefish module.
[2024-03-11 00:11:52.794] [info]    "C:/Users\natha\Downloads\casparcg-server-v2.3.3-lts-stable\casparcg.config":
[2024-03-11 00:11:52.794] [info]    -----------------------------------------
[2024-03-11 00:11:52.794] [info]    <?xml version="1.0" encoding="utf-8"?>
[2024-03-11 00:11:52.794] [info]    <configuration>
[2024-03-11 00:11:52.794] [info]       <paths>
[2024-03-11 00:11:52.794] [info]          <media-path>media/</media-path>
[2024-03-11 00:11:52.794] [info]          <log-path>log/</log-path>
[2024-03-11 00:11:52.794] [info]          <data-path>data/</data-path>
[2024-03-11 00:11:52.794] [info]          <template-path>template/</template-path>
[2024-03-11 00:11:52.794] [info]       </paths>
[2024-03-11 00:11:52.794] [info]       <lock-clear-phrase>secret</lock-clear-phrase>
[2024-03-11 00:11:52.794] [info]       <channels>
[2024-03-11 00:11:52.794] [info]          !--
[2024-03-11 00:11:52.794] [info]          <channel>
[2024-03-11 00:11:52.794] [info]             <video-mode>1080p2500</video-mode>
[2024-03-11 00:11:52.794] [info]             <consumers>
[2024-03-11 00:11:52.794] [info]                <screen>
[2024-03-11 00:11:52.794] [info]                   <device>1</device>
[2024-03-11 00:11:52.794] [info]                   <aspect-ratio>default</aspect-ratio>
[2024-03-11 00:11:52.794] [info]                   <stretch>fill</stretch>
[2024-03-11 00:11:52.794] [info]                   <windowed>true</windowed>
[2024-03-11 00:11:52.794] [info]                   <key-only>false</key-only>
[2024-03-11 00:11:52.794] [info]                   <vsync>false</vsync>
[2024-03-11 00:11:52.794] [info]                   <borderless>false</borderless>
[2024-03-11 00:11:52.794] [info]                   <interactive>true</interactive>
[2024-03-11 00:11:52.794] [info]                   <always-on-top>false</always-on-top>
[2024-03-11 00:11:52.794] [info]                   <x>3300</x>
[2024-03-11 00:11:52.794] [info]                   <y>100</y>
[2024-03-11 00:11:52.794] [info]                   <width>480</width>
[2024-03-11 00:11:52.794] [info]                   <height>270</height>
[2024-03-11 00:11:52.794] [info]                   <sbs-key>false [true|false]</sbs-key>
[2024-03-11 00:11:52.794] [info]                   <colour-space>RGB [RGB|datavideo-full|datavideo-limited] (Enables colour space convertion for DataVideo TC-100 / TC-200)</colour-space>
[2024-03-11 00:11:52.794] [info]                </screen>
[2024-03-11 00:11:52.794] [info]                <ndi>
[2024-03-11 00:11:52.794] [info]                   <name>CasparCG_1</name>
[2024-03-11 00:11:52.794] [info]                   <allow-fields>false</allow-fields>
[2024-03-11 00:11:52.794] [info]                </ndi>
[2024-03-11 00:11:52.794] [info]             </consumers>
[2024-03-11 00:11:52.794] [info]          </channel>
[2024-03-11 00:11:52.794] [info]          <channel>
[2024-03-11 00:11:52.794] [info]             <video-mode>1080p2500</video-mode>
[2024-03-11 00:11:52.794] [info]             <consumers>
[2024-03-11 00:11:52.794] [info]                --
[2024-03-11 00:11:52.794] [info]                <screen>
[2024-03-11 00:11:52.794] [info]                   <device>1</device>
[2024-03-11 00:11:52.794] [info]                   <aspect-ratio>default</aspect-ratio>
[2024-03-11 00:11:52.794] [info]                   <stretch>fill</stretch>
[2024-03-11 00:11:52.794] [info]                   <windowed>true</windowed>
[2024-03-11 00:11:52.794] [info]                   <key-only>false</key-only>
[2024-03-11 00:11:52.794] [info]                   <vsync>false</vsync>
[2024-03-11 00:11:52.794] [info]                   <borderless>false</borderless>
[2024-03-11 00:11:52.794] [info]                   <interactive>true</interactive>
[2024-03-11 00:11:52.794] [info]                   <always-on-top>false</always-on-top>
[2024-03-11 00:11:52.794] [info]                   <x>3300</x>
[2024-03-11 00:11:52.794] [info]                   <y>500</y>
[2024-03-11 00:11:52.794] [info]                   <width>480</width>
[2024-03-11 00:11:52.794] [info]                   <height>270</height>
[2024-03-11 00:11:52.794] [info]                   <sbs-key>false [true|false]</sbs-key>
[2024-03-11 00:11:52.794] [info]                   <colour-space>RGB [RGB|datavideo-full|datavideo-limited] (Enables colour space convertion for DataVideo TC-100 / TC-200)</colour-space>
[2024-03-11 00:11:52.794] [info]                </screen>
[2024-03-11 00:11:52.794] [info]                <ndi>
[2024-03-11 00:11:52.794] [info]                   <name>CasparCG_2</name>
[2024-03-11 00:11:52.794] [info]                   <allow-fields>false</allow-fields>
[2024-03-11 00:11:52.794] [info]                </ndi>
[2024-03-11 00:11:52.794] [info]             </consumers>
[2024-03-11 00:11:52.794] [info]          </channel>
[2024-03-11 00:11:52.794] [info]       </channels>
[2024-03-11 00:11:52.794] [info]       <controllers>
[2024-03-11 00:11:52.794] [info]          <tcp>
[2024-03-11 00:11:52.794] [info]             <port>5250</port>
[2024-03-11 00:11:52.794] [info]             <protocol>AMCP</protocol>
[2024-03-11 00:11:52.794] [info]          </tcp>
[2024-03-11 00:11:52.794] [info]       </controllers>
[2024-03-11 00:11:52.794] [info]       <amcp>
[2024-03-11 00:11:52.794] [info]          <media-server>
[2024-03-11 00:11:52.794] [info]             <host>localhost</host>
[2024-03-11 00:11:52.794] [info]             <port>8000</port>
[2024-03-11 00:11:52.794] [info]          </media-server>
[2024-03-11 00:11:52.794] [info]       </amcp>
[2024-03-11 00:11:52.794] [info]    </configuration>
[2024-03-11 00:11:52.794] [info]    -----------------------------------------
[2024-03-11 00:11:53.157] [info]    Initialized OpenGL Accelerated GPU Image Mixer for channel 1
[2024-03-11 00:11:53.158] [info]    video_channel[1|1080p2500] Successfully Initialized.
[2024-03-11 00:11:53.159] [info]    Initialized OpenGL Accelerated GPU Image Mixer for channel 2
[2024-03-11 00:11:53.159] [info]    video_channel[2|1080p2500] Successfully Initialized.
[2024-03-11 00:11:53.161] [info]    Screen consumer [1|1080p2500] Initialized.
[2024-03-11 00:11:53.170] [info]    Loaded "C:\Program Files\NDI\NDI 5 Tools\Runtime\Processing.NDI.Lib.x64.dll"
[2024-03-11 00:11:53.263] [info]    ndi_consumer[1|CasparCG_1] Initialized.
[2024-03-11 00:11:53.265] [info]    Screen consumer [2|1080p2500] Initialized.
[2024-03-11 00:11:53.275] [info]    ndi_consumer[2|CasparCG_2] Initialized.
[2024-03-11 00:11:53.275] [info]    Initialized channels.
[2024-03-11 00:11:53.287] [info]    Initialized controllers.
[2024-03-11 00:11:53.288] [info]    Initialized osc.
[2024-03-11 00:11:53.350] [info]    async_event_server[:5250] Accepted connection from 127.0.0.1 (1 connections).
[2024-03-11 00:11:53.351] [info]    Received message from 127.0.0.1: VERSION SERVER\r\n
[2024-03-11 00:11:53.351] [info]    Sent message to 127.0.0.1:201 VERSION OK\r\n2.3.2 4de6d18f Dev\r\n
[2024-03-11 00:11:53.351] [info]    Received message from 127.0.0.1: INFO\r\n
[2024-03-11 00:11:53.352] [info]    Sent message to 127.0.0.1:200 INFO OK\r\n1 1080p2500 PLAYING\r\n2 1080p2500 PLAYING\r\n\r\n
[2024-03-11 00:11:53.352] [info]    Received message from 127.0.0.1: CLS\r\n
[2024-03-11 00:11:53.353] [info]    Received message from 127.0.0.1: TLS\r\n
[2024-03-11 00:11:53.354] [info]    Received message from 127.0.0.1: DATA LIST\r\n
[2024-03-11 00:11:53.356] [info]    Received message from 127.0.0.1: THUMBNAIL LIST\r\n
[2024-03-11 00:11:57.365] [error]   Failed to connect to media-scanner. Is it running?
[2024-03-11 00:11:57.365] [error]   Reason: Connection refused
[2024-03-11 00:11:57.365] [info]    Sent message to 127.0.0.1:501 CLS FAILED\r\n
[2024-03-11 00:12:00.397] [info]    Sent message to 127.0.0.1:200 TLS OK\r\nGITM D2 V3\r\n\r\n
[2024-03-11 00:12:00.398] [info]    Sent message to 127.0.0.1:200 DATA LIST OK\r\n\r\n
[2024-03-11 00:12:00.405] [info]    Sent message to 127.0.0.1:200 THUMBNAIL LIST OK\r\n"BANNER V1" 20230215T235655 89747\r\n"BANNER V3" 20230215T235641 77918\r\n"EMPOWERMENT PROMO [TRAINING DAYS]" 20230215T235853 2252\r\n\r\n
[2024-03-11 00:12:08.538] [info]    Received message from 127.0.0.1: PLAY 1-10 "BANNER V1" CUT 0 Linear RIGHT\r\n
[2024-03-11 00:12:08.542] [info]    Sent message to 127.0.0.1:202 PLAY OK\r\n

I have been playing around with using CasparCG for video/title playout in my streaming rack. It was a while ago that I was experimenting, and I was experimenting using a laptop. I have a few events, where it would be useful, coming up, and so I am trying to move my Caspar CG set up from the Laptop to a mini-desktop inside the rack itself.

On the laptop, everything works perfectly, but on the mini-pc (it’s a minisforum UM350 - Ryzen 5 3550H with vega integrated gfx, 16GB ram), the server crashes the second I try to play out anything - I have tried with three different videos and one template, all of which work absolutely fine on the original laptop.

There is another thread talking on this issue, where it seems to be a recent AMD driver that is causing problems and rolling back fixed it for them. (The Laptop is running an nVidia GPU) I have tried installing the oldest driver still available from AMD’s website (22.11.2 - August 22) and still no dice.

An error in the log seems to mention media scanner not working, as far as I know it should be fine in the version I am using but I did download media scanner separately and tried running it (I tried starting it both before and after launching the server.) Again, it didn’t seem to make a difference.

I had changed the config when setting it up, but I did keep a copy of the original ‘stock’ config and swapped that back in, making no difference. I also tried removing my set up version and fresh installing the latest stable version of Caspar CG and just copying the template file across, changing nothing else and still nothing. This leads me to being pretty sure it is something to do with the GPU/drivers rather than caspar config. (Again, it is working just fine on another machine - without needing to launch scanner seperately.)

Someone on a thread suggested changing the amcp host to be 127.0.0.1 rather than localhost, but that didn’t work either.

Finally, I have also changed the ports, just in case that was causing an issue, but again, no dice.

A thread on github talking about a similar issue talks about RGB and RGBA, but I found only one reference to RGB in the config and changing it to RGBA made no difference. I feel like this was actually something to do with the driver, but it went over my head - so if anyone knows how to change this (or if this might be the problem) that would be great.

Any help would be appreciated!

didikunz · March 11, 2024, 8:31am

What are these !-- and -- marks about? In XML  ends it. I guess this messes up the config. At least I would get rid of it.

You run on Windows, do you? Have you tried to run both (scanner and CasparCG server) “as Administrator”? Usually Scanner.exe terminâtes, when it does not have admin privileges and Caspar does not run correctly. You can also use a batch file (or casparcg_auto_restart.bat) to start both programs. If you let that batch file run as Administrator both programs will also be run as admin.

andyw · March 11, 2024, 11:41am

In addition to the items @didikunz highlights there are four more badly formatted XML lines in the config file. Although these do not seem to be causing error reports in your system the lines should be removed or amended. Remove them if your system is not using a DataVideo TC-100 or TC-200 interface. The lines that need removing are the last two in both screen consumer settings:

<sbs-key>false [true|false]</sbs-key>
<colour-space>RGB [RGB|datavideo-full|datavideo-limited] (Enables colour space convertion for DataVideo TC-100 / TC-200)</colour-space>

The media scanner does not need to be running to play videos, but is needed by a client such as the SVT standard client which asks the server to return the lists of files. In turn the server asks for the media list and template list from the scanner. The scanner must run in the same folder as the server because the scanner reads the config file paths segment of the casparcg.config file to locate the media stores.

So you can play a media file by typing the play command in the server console window:

play 1-10 "BANNER V1"

You can test the media scanner connection and operation from a web browser. If you run the browser on the server machine enter the address http://localhost:8000/cls which should return a list of media files and their key properties or address http://localhost:8000/tls which should return a list of template files.

I note from the log report that the server is returning a template list in response to the Received message from 127.0.0.1: TLS\r\n suggesting that the scanner is accessible from the server.

The list of templates returned also implies the server has run at some time to create the thumbnails.

The only other slightly odd value I spotted is the <x> value in both screen consumer definitions. I assume you have a very large extended display as the value of 3300 if quite large. However I do not think this should cause a crash if you only have a full HD resolution display.

quoole · March 12, 2024, 6:15pm

Removed the extra stuff that you guys mentioned. The web address goes to a page not found error and entering the command you specified to playout the video also immediately crashes.

andyw · March 12, 2024, 9:41pm

This is a “Just in case” sort of suggestion. In your first posting the address of the config file shows as:

C:/Users\natha\Downloads\casparcg-server-v2.3.3-lts-stable\casparcg.config

I wonder if the dots in the version number are tripping the name parser when it trys to access your media? Try changing the folder name to use underscores rather than dots.

C:/Users/natha/Downloads/casparcg-server-v2_3_3-lts-stable

I assume that your media path would then become:

C:/Users/natha/Downloads/casparcg-server-v2.3.3-lts-stable/media/

As the web browser fails with a page not found error it suggests that the scanner is either not running or not receiving port 8000 traffic. You can ask windows to list the open TCP and UDP ports and which task owns the open port. Run “Command Prompt” as an administrator. Then enter the command:

netstat -ab

Look through the reports for a line

 TCP    [::]:8000

The line below should show [scanner.exe] as the owner.

If scanner.exe is running and owns TCP port 8000 and CasparCG opens, type the AMCP CLS command in the server window. When you press the enter key you should see activity in the scanner window

quoole · March 12, 2024, 9:59pm

I managed to find an older version of the driver on AMD’s website, and that seems to have worked!

For reference for anyone else with the same issue, I am using 19.9.2.

If anyone does have any further suggestions, that would be great, as obviously an outdated GFX driver is not ideal!