Questions about the build system

General
1

With the help of this forum and the Gentoo admins I have been able to create an Ebuild file for the Gentoo Package Manager which allows to properly build and install CasparCG Server using the Gentoo Package Manager. However, that undertaking has raised some question on the Gentoo side regarding the build system and they asked me to submit some patches to CasparCG to make the build system more compatible. But I am wondering, why the build system might be as it is and before I start to actually take the effort and I would like to learn more about the why:

  1. Why is the idea to build everything inside a Docker image? What’s the reason for not running cmake/ninja on the host system?
  2. Why is there no CMakeLists.txt in the project’s root directory?
  3. Installation: The CMakeLists.txt lacks a build target to actually install the final binaries and auxiliary files onto the host system. Instead there are some shell scripts which try to extract the files from the Docker image. Why this approach?
2

Is there a link to something I can follow?
FYI, I have been making various simplications/changes to the build process in master(2.5), so make sure you look at the 2.4.x branch building guide for accurate description of arguments and things. server/BUILDING.md at v2.4.x · CasparCG/server · GitHub

  1. I think that came about as a way to easily build on a predictable target with predictable library versions without needing to actually install that particular os.
    At the time it was desired to have the software packaged as a single zip that could be extracted and run without needing to install any libraries on the system. But of course this adds other issues and quirks, so in 2.5/master I have start advising against that approach, and the auto-builds are packaging debs instead.

  2. I have a vague memory of the person doing the docker build script wanting everything put under a src directory for easier dockering (prior to this the contents of that was at the root). I wasnt involved in that, and haven’t questioned it since as it works.
    I suppose adding a file at the root level could basically just include the one in src, making it optional of which to use? (ie, not breaking any existing build scripts)

  3. Again this is primarily due to how the binaries used to be distributed as a zip file.
    Even now with the deb packaging, I based that upon prior work in debian instead of doing it via cmake. So I have simply had no need to learn how to do it.
    I expect that it is still the case that the majority of users are on windows (linux support was a much later addition)

All of these answers feel like historical baggage of decisions someone else made that I haven’t been confident enough to change.
I am very open to improving it, especially if it doesn’t break existing workflows/scripting

3

That is probably the easiest way forward. But I am too less of an CMake expert to say whether this works as expected. Maybe there are certain CMake settings (like project name, etc.) which must only be set once in the “main” CMakeLists.txt and hence prevent you from having two CMakeLists.txt which can be used interchangeably.

In the long run, there might be a situation which does not require the tools/ directory anymore and hence the content of src/ could become the root again. But generally, I agree that it is nice to have a directory which is the root for the actual C/C++ source files that make up the final binaries and keep that seperated from auxilary (script) files which are required to maintain the “infrastructure” of the project. Even without tools/ there would still be the necessity for .github/. Hence, keeping src/ seems a good thing.

Good to know. So let me just paraphrase what Gentoo expects from the build system:

  • CMakeLists.txt should define the typical standard targets, like all, install, doc etc., with all and install being the required minimum. Any other targets are purely optional:
    • all should build all artefacts of the project; for binaries this means to compile and link them
    • install should copy the built artifacts to the proper, final locations
  • The actual build system which CMake is going to use (i.e. traditional Make, Ninja, etc.) is irrelevant
  • All CMake targets but install should be runnable as a non-privileged user.
  • The CMake targets should act as if they were operating on the final host system.
    • For instance, the target install should attempt to copy the built artifacts to the proper locations on the actual system, i.e. executable should go into usr/bin, libraries to usr/lib64, etc.
    • In particular, CMake itself should not try to isolate the build or installation in some kind of enclave. Leave any kind of sandboxing/chrooting/containerization to the surrounding system.
      • For example, CMake supports the environment variable DESTDIR which causes CMake to actually install everything relatively to ${DESTDIR}, i.e. ${DESTDIR}/usr/bin, ${DESTDIR}/usr/lib64, etc. However, this is transparent for the project’s CMakeLists.txt. This is nothing CMakeLists.txt should be concerned with.
      • The Gentoo Emerge system sets the environment variable DESTDIR to a temporary location. Emerge then supervises and records everything what CMake does and creates an actual Gentoo package from the recorded actions. However, this only works, if the projects CMakeLists.txt is compliant with the standard approach.
  • The build targets should be the top-level entry points, i.e. the project should not expect that someone calls some scripts (e.g. like build-in-docker.sh) which then call CMake in a clever way. Instead, imagine a user which calls the build system in the traditional way manually, i.e. mkdir ../build; cd ../build; cmake ../<project-root>; make all; DESTDIR="/some/prefix/path/" make install.
  • Packaging: This is nothing Gentoo Linux needs, but I thought I mention this point, because you talked about Debian packages. There is also a standard target dist which can be used to create a packaged version of the installation. This is something you might use. (Gentoo doesn’t need it, because it creates a package from supervising the actions of CMake inside a sandbox.)

Thanks for the advice. Honestly, I do not feel like picking up the torch and changing/advancing/improving the build system. I am neither involved with the Gentoo Package system nor CapsarCG. Taking on activities in this field will probably lead to an situation in which soon necessary work will get left undone. I am already participating in other open source projects. I do not even use CasparCG myself. I only happen to create an Gentoo Ebuild for it as my sister wants to use CasparCG on her laptop. Hence, as I am not a user myself I would probably not even notice if anything breaks myself in a timely manner.