In this article we would like to publish an update regarding three key components of Cosmos Laundromat’s pipeline: BAM, Flamenco and Attract.
The development of BAM started during the beginning of the Gooseberry project, with the intention of investigating and developing the foundation for an asset management tool. While we could not cover the all the initial design specs of the project due to time constraints, we laid out a foundation for future work.
Done:
Todo:
Before continuing the work on BAM, it will be very interesting to check on the current Asset/Library management improvements being added in Blender, as well as trying some radical ideas, such as saving a blend file in an “exploded format”, with all data block stored independently.
Originally known as brender, Flamenco is the software we used to manage the two render farms here in Amsterdam and in Paris. It was mostly rewritten for the Gooseberry project and survived the production, with the server handling two managers and over 200 nodes during peak time.
Using a freshly designed software during production, facing new requirements (mainly multi-site rendering) provided us with continuous challenged. This will be more extensively presented in October at the Blender Conference in a dedicated talk.
Done
Todo
The Attract shot management software was developed during the production of Tears of Steel as a quick utility to keep track of the status of each shot. After that, it was further improved to support more complex scenarios, involving tasks, assignees, shots, sequences, and so on. In order to achieve the desired flexibility for a tool of this type, we had to rethink the entire system architecture. During the production of Cosmos Laundromat we laid the foundation of a modular and powerful system, leveraging more modern technologies and design patterns. The experience we gained developing such tool, allowed us to explore and experiment with the Python EVE framework, Flask and MongoDB. Most importantly, we investigated how to integrate such system with the Blender ID authentication system.
The results of these research are being merged with the Blender Cloud, which in the future will integrate Attract and Flamenco. There is no official roadmap yet, but we are actively working on a new Blender Cloud implementation, following the Attract design pattern.
If you have any question or remark, please drop us a comment!
Regarding the work on Flamenco, did you guys consider any of the other options that are already out there, or was the idea to build something completely in-house?
I’m thinking of something like HTCondor or Torque when I ask, though I don’t know how well either of those two tools would work for you particular environment.
Thank you for the update.
thanks for the update
Thanks too for the update :)
All of those projects are not well known, that is some kind of shadow work, but that remains very important for every studios and company working with Blender :)
Keep going guys ;)
Nice report, thanks.
TODO “Investigate higher performance C++ based server and manager”
I think you should consider Nodejs for this, because that way one can put one instance (or many) of the software to Amazon or some other cloud very easily and run it by themselves.