Succesfull CasparCG 2.2 linux build

linux

#1

We have been able to build CasparCG 2.2 in ubuntu 18.04.
We followed the following steps:

  1. We downloaded code from https://github.com/CasparCG/server/releases
  2. executed the following steps
  3. Install dependencies apt-get install git cmake build-essential g++ libglew-dev libfreeimage-dev libtbb-dev libsndfile1-dev libopenal-dev libjpeg-dev libfreetype6-dev libglfw3-dev libxcursor-dev libxinerama-dev libxi-dev libsfml-dev libvpx-dev libwebp-dev liblzma-dev libfdk-aac-dev libmp3lame-dev libopus-dev libtheora-dev libx264-dev libx265-dev libbz2-dev libssl-dev libcrypto++-dev librtmp-dev libgmp-dev libxcb-shm0-dev libass-dev libgconf2-dev
  4. Install Docker by following installation instructions from [Docker Docs][1]
  5. git clone --single-branch --branch master https://github.com/CasparCG/server casparcg-server-master
  6. cd casparcg-server-master
  7. Extract Boost, FFmpeg and CEF from the docker images via sudo ./tools/linux/extract-deps-from-docker. Alternatively these can be prepared manually by following the steps laid out in each Dockerfile
  8. mkdir build && cd build
  9. cmake ../src
  10. make -j8

When we tried to run the build we got an error message.missing “ErrNo.”
We included the follwoing to resolve the error.
In Casparcg server master folder inside src>tools there is a bin2c.cpp file.There we have added
#include<errno.h>
From staging folder we were able to start CasparCG 2.2 linux server successfully


#2

Hey @Syscaspar - thank you for putting this info up here. I have struggled with getting things working on Linux and the official documentation seems to be lacking a bit. (I’m looking at submitting some changes once I’ve got everything running correctly!)

A few questions, if you don’t mind:

  1. Do you have any updates to this method or is this still working well for you?
  2. Is this running on Ubuntu Desktop or Server?
  3. Are you using this on a headless server or with a GUI/desktop on the Caspar box?

Many thanks


#3

The steps above look to be directly copied from https://github.com/CasparCG/server/blob/master/BUILDING.md#development, with a fix applied to a file because it caused the build to fail.

What have you been struggling with for building? Those steps work for me, but only on ubuntu18.04. On any other version or distro, ‘Extract Boost, FFmpeg and CEF’ step has to be done differently.

Those steps are targetting Ubuntu desktop. Doing it on server should work too, but will likely need more dependencies installed. (Also expect it to install a lot more packages than those listed)


#4

Thanks @Julusian, I am indeed trying to install and run on a headless server, running in AWS ec2. (A g3.4xlarge instance)

Running CGC results in

[0618/203706.396046:ERROR:browser_main_loop.cc(1653)] Unable to open X display.

I’m making some slow progress in understanding the issues I’m facing but you’ll still need to bear with me! Here is what I know so far:

The output of readelf -d casparcg | grep NEEDED is

 0x0000000000000001 (NEEDED)             Shared library: [libtbb.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libtbbmalloc.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libtbbmalloc_proxy.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libsfml-graphics.so.2.4]
 0x0000000000000001 (NEEDED)             Shared library: [libsfml-window.so.2.4]
 0x0000000000000001 (NEEDED)             Shared library: [libsfml-system.so.2.4]
 0x0000000000000001 (NEEDED)             Shared library: [libGLEW.so.2.0]
 0x0000000000000001 (NEEDED)             Shared library: [libGL.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libvpx.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libwebpmux.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libwebp.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [liblzma.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libfdk-aac.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libmp3lame.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libopus.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libtheoraenc.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libtheoradec.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libvorbis.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libvorbisenc.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libx264.so.152]
 0x0000000000000001 (NEEDED)             Shared library: [libx265.so.146]
 0x0000000000000001 (NEEDED)             Shared library: [libbz2.so.1.0]
 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.1.1]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.1.1]
 0x0000000000000001 (NEEDED)             Shared library: [librtmp.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-shm.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-shape.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-xfixes.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libass.so.9]
 0x0000000000000001 (NEEDED)             Shared library: [libfreetype.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libicui18n.so.60]
 0x0000000000000001 (NEEDED)             Shared library: [libicuuc.so.60]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libopenal.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libcef.so]
 0x0000000000000001 (NEEDED)             Shared library: [libfreeimage.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

And so it looks like I might be able to somehow manually link CasparCG to the correct drivers and also to use an “OpenGL context”, as per the Nvidia blog, but I will need a bit of help to join up those dots (if indeed that is the correct path to go down).

Thanks as always


#5

Xvfb doesn’t support hardware acceleration so it’s no good as a dummy X server.

Ah. It’s been a while since I have tried any of this, so I was probably thinking of a 2.1 (cpu mixer) setup without a gpu then.

Maybe an X server (dummy or otherwise) isn’t necessary anymore as the Nvidia driver supports EGL

I looked into using EGL a while ago, but SFML which is used to get the OpenGL context only works with X11. I havent checked since then, so it might have changed in newer versions.


#6

Hi @peedrr have you made any progress on the headless server?, I’m looking to do the same but on google cloud… for now I’m stuck at the same point as you.

I found the same article and also This updated post on stackoverflow and this blog post from Nvidia but I don’t really understand anything at this point, hehe

btw for me was a lot easier to install and build Casparcg using the Building inside Docker procedure and then just use ./tools/linux/extract-from-docker.


#7

Not really, no. I’m hoping to take another bash at it one evening this week. Current thoughts are to implement VirtualGL as it seems to do what’s needed, plus it has a very short guide specifically for Nvidia Headless Servers which seems to fit the bill.

I’ll post my findings in here, but let me know if you get a chance to try this out before me. Would be good to pool our knowledge on here.

I’ve got to say, I’m slightly relieved to hear that I’m not alone in this struggle, although I’m finding it hard to believe that we’re trailblazers! Surely others in this community have already solved this problem. If anyone has any specific pointers, or knows of others who are running CasparCG headless on Linux, please shout! I’m keen to get a detailed how-to guide together.

@Julusian - has CPU Mixer development been abandoned? I always thought this sounded promising, especially for cloud stuff where GPU is still a bit of a novelty and often difficult to configure


#8

VirtualGL looks interesting. Not 100% sure if it will just work though, as caspar expects an x server to get the opengl context from, so it might still require running one manually. But VirtualGL looks like it could still be useful to provide access to an xserver/opengl over ssh.

has CPU Mixer development been abandoned?

All I know is that it has been removed from 2.2 as it was a burden and to allow some abstractions to be removed.
Personally I agree that it has its uses. Cloud instances is a good example, or even caspar channels which dont need any compositing. (eg a channel being a simple html -> decklink)


#9

I don’t fully understand but I was thinking the same… I did find this on StackOverflow about creating an OpenGL context without an X server.

NVIDIA recently published an article detailing how to use OpenGL on headless systems, which is a very similar use case as the question describes.

In summary:

Link to libOpenGL.so and libEGL.so instead of libGL.so. (Your linker options should therefore be -lOpenGL -lEGL

Call eglGetDisplay, then eglInitialize to initialize EGL.
Call eglChooseConfig with the config attribute EGL_SURFACE_TYPE followed with EGL_PBUFFER_BIT.
Call eglCreatePbufferSurface, then eglBindApi(EGL_OPENGL_API);, then eglCreateContext and eglMakeCurrent.

From that point on, do your OpenGL rendering as usual, and you can blit your pixel buffer surface wherever you like. This supplementary article from NVIDIA includes a basic example and an example for multiple GPUs. The PBuffer surface can also be replaced with a window surface or pixmap surface, according to the application needs.

The linked source URL (in my previous post) gives an example, the following snippet compiles an application with support for both EGL and GLX contexts.

find_package(OpenGL REQUIRED COMPONENTS OpenGL EGL GLX)
add_executable(your_binary_name main.c)
target_link_libraries(your_binary_name PRIVATE OpenGL::OpenGL OpenGL::EGL OpenGL::GLX)

What do you think @Julusian, Does any of this look like It could be helpful?


#10

Depends on what functionality you need.
Caspar currently has a hard dependency on a running X server because of the library used to setup OpenGL, but that can easily be replaced with EGL, a few messages back I linked to a very outdated branch where I was playing around with that. I think that would be a pretty easy port, so doing something like that is an option.
However, CEF/Chromium also requires a running X server. I did not look much into fixing that, and that is going to be a much more difficult task to fix. It might be possible with a recompilation of CEF. Ozone looks to be the required thing to configure https://bitbucket.org/chromiumembedded/cef/issues/2296/linux-add-ozone-mus-support-as-an https://chromium.googlesource.com/chromium/src/+/master/docs/ozone_overview.md.
I didn’t have a need to run caspar on a headless machine, I was only experimenting so that is where I stopped.

I don’t fully understand but I was thinking the same…

What I was trying to say is that Caspar needs an X server to setup OpenGL. But even with X running, it is not easily possible to run caspar over ssh, but VirtualGL looks like it might resolve that ssh problem.


#11

Got it, that makes total sense! thanks!


#12

hmm, i got casparcg branch 2.2.x built on ubuntustudio 18.04 (lowlatency kernel) but i just downloaded, unzipped &

cd server-2.2.x
./tools/linux/build-in-docker
./tools/linux/extract-from-docker
cd casparcg_server
nano casparcg.config (edit paths to casparcg root dir)
./run.sh

i may have had to Resolve Common Issues on Linux

…when i used

git clone --single-branch --branch _master_ https://github.com/CasparCG/server _casparcg-server-master_

i kept building the 2.0.1 source instead of 2.2.x

edit: unfortunately, media-scanner did not get built in docker, the server is throwing errors about it & no paths to media, videos, images, etc are seen in the client.

after building media-scanner, the executable has errors:

$ ./scanner 
pkg/prelude/bootstrap.js:1172
      throw error;
      ^

Error: Unexpected close tag
Line: 8
Column: 75
Char: >
    at error (/snapshot/media-scanner-master/node_modules/sax/lib/sax.js:651:10)
    at strictFail (/snapshot/media-scanner-master/node_modules/sax/lib/sax.js:677:7)
    at closeTag (/snapshot/media-scanner-master/node_modules/sax/lib/sax.js:871:9)
    at SAXParser.write (/snapshot/media-scanner-master/node_modules/sax/lib/sax.js:1436:13)
    at Parser.exports.Parser.Parser.parseString (/snapshot/media-scanner-master/node_modules/xml2js/lib/parser.js:322:31)
    at Parser.bind [as parseString] (/snapshot/media-scanner-master/node_modules/xml2js/lib/parser.js:5:59)
    at Object.<anonymous> (/snapshot/media-scanner-master/src/config.js:0:0)
    at Module._compile (pkg/prelude/bootstrap.js:1243:22)
    at Object.Module._extensions..js (module.js:644:10)
    at Module.load (module.js:552:32)

edit: media-scanner-thumbnail-fix throws the same error as master :confused:

has anyone successfully built media-scanner on linux?