I’ve been experimenting with using CasparCG as an offline renderer, compositing video and HTML graphics to file without the real-time constraint.
I built an offline consumer that skips the sync clock so the pipeline runs as fast as the encoder allows, and an Ultralight HTML producer (WebKit-based, but less features than CEF) that renders exactly one frame per channel tick, synchronously. No wall-clock dependency. One tick, one frame.
Caveat: I only tested this in Docker on a Mac with software OpenGL, no GPU at all. Numbers should be much better on Linux with a real GPU.
Results at 720p25 on software GL:
Configuration
Speed
Frames
Deterministic
Video only (offline)
2.3x
501
+1
Video + Ultralight offline
1.5x
500
exact
Video + CEF offline
1.5x
500
no*
FFmpeg consumer (any)
0.9x
501
+1
* CEF overlay frames are duplicated/skipped since its compositor is wall-clock driven.
Changes to existing code are small: 44 lines across 8 files. Ultralight is opt-in via cmake. CEF is untouched except for one JS injection per tick.
This is an experiment, not production-ready. I’d love feedback:
Useful for your workflow?
Templates that might break under Ultralight vs CEF?
I will happily accept a PR for the offline consumer part upstream, some users have been asking for this for years, so a piece of the puzzle that starts in that direction would be good.
ultralight I am less sure on though, as the paid licensing of it makes it challenging to use, and any builds using it wont be GPL compliant (but that doesnt stop users/organisations building it for their own use).
On the offline consumer though, it looks like 90% of the code is the same as the current ffmpeg producer, the changes I can see from a diff are:
Removing a bunch of comments/TODO notes
Removing the realtime parameter/property (hardcoding to false)
Adding a new queue_depth property
Some minor logging/naming changes
The main part; changing how send() and has_synchronization_clock() work
So it looks to me that this could be achieved by possibly replacing the realtime boolean with an enum (file/stream/offline?), and then possibly either handling determining that inside the existing factory methods, or some new factory methods.
That will greatly reduce the maintenance compared to having a separate file with 90% duplication, and will ensure that any fixes made for one apply to the other too.
Definitely, you’re right, there’s a lot of duplication. And as such the offline consumer isn’t that big of a modification. It’s really just because my first intention was to reduce the blast radius in the existing code while testing and comparing.
But I could definitely try to merge them with an enum like you suggest, and add that to a PR.
I’m aware that Ultralight might be a difficult dependency, but as for now it seems like the only html-renderer that will work in lock-step. That’s why I hid it behind a feature flag that’s off by default. I just really enjoyed having a path where the html render is deterministic. As such I didn’t intend it to be invasive. Unfortunately I don’t think there are many other alternatives out there, as browser engines tend to be beasts, and not even Ultralight is feature-complete compared to webkit.
I’ll do a pr against the main repo, and then we can work from there!
Performance-wise, is Ultralight considered superior for the HTML producer because it bypasses the CPU-to-GPU IPC and memory-copying stages inherent in CEF?
The issue is that I haven’t found a way to make CEFs free-running thread sync up with the frame consumer. This leads to sync drift and for offline rendering it means that CEF will bot animate smoothly, as there’s no synchronous calls to upside and render the framebuffer of the html view.
I’m moving in a similar direction, but from a slightly different angle.
CasparCG Logic Without CasparCG
For the past six months, I’ve been creating templates for CasparCG. Ultimately, I realized that for simpler workflows, where a human acts as an intermediary for data entry, a full-fledged server might be overkill.
As a result, I developed Lottie Local Lower Thirds. It uses Playwright to render Lottie animations directly to ProRes 4444 with an alpha channel. No CasparCG installation or server setup is required. It’s designed as a portable tool for editors. The goal is to create a new template simply by replacing the JSON file.
But we’re still a long way off, and this is just an experiment and an attempt to simplify some parts of our work.
I just want to point out, that the OGraf initiative of the EBU contains a specification for “non-realtime” rendering of templates, that supports this. Maybe interesting to look at that, before a Caspar specific workflow is implemented.
Thanks for all the feedback. I’ve also had a look of Ograf, and it’s a super clever way of making web components announce their parameters. I haven’t read the full spec and just experimented with frame synced renders for html, but yes, it would make sense to go straight to ograf, if there’s interest. Currently I’m just doing this for fun, but if someone wants to sponsor a real contribution, I’d be happy to go that direction. Also - I would dare say - if html rendering is becoming a broadcast standard, maybe it’s time to get involved with webkit or chromium to solve deterministic framestepping there once and for all.