Hi there!
Pablo taught me a little bit of simple artist-friendly Python and how we lighters/compositors can utilize it for preparing the animated shots for the render farm – in other words: To Automate our Workflow. Hopefully it will benefit some of you, dear readers, as well :)
As an example, I will use the shot 1.2.1.E1, animated by Hjalti.
Thanks to the linking capabilities of Blender it’s easy for me to start to light the scene in a separate .comp file even if Hjalti is still animating. The latest animation changes from the .anim file will automatically get updated to my .comp file when I update my Gooseberry Repository in our SVN and reload the file in Blender. If you don’t know what the SVN is, I can try to search for some blog posts about it from the Mango, Durian, Peach and Orange blogs and edit it here if I find one. In the mean time, the web is your friend. There will also be a more detailed tutorial about the linking process later. – However, when Lukas and the fellows get the Alembic caching and related things to work reliably for production, we will do things in a totally different manner – that also could be an excellent post topic for the future. But for now, let’s just concentrate on how the things are done at this moment in Gooseberry.
So, when the hi-res versions of the environment set, lights, fog, random stones and bushes, Victor, Franck and the animation actions are linked into the .comp file, some of their properties are locked and can’t be edited because of how the linking in Blender currently functions. Also, there are a whole bunch of rendering related and other settings that need to be adjusted before sending the file to the farm. Some of the render settings could be individually put by hand, of course, but automating things is a lot more efficient (and fun :D) way to handle that. This is a perfect case for us to use some Python scripting.
With it we can do with a single button press:
Okay, let’s begin!
I have two datablocks in the Text Editor of Blender (Shift + F11):
The settings.py text datablock can be freely renamed to whatever, but the startup.py must have the .py ending. Otherwise it won’t get registered (in fact, the “Register” checkbox will be grayed out if the name doesn’t end with .py).
There are links at the end of this post to download the full example scripts.
As a side note, it must be mentioned that Pablo’s Amaranth Add-on has also been an invaluable tool for easing up the daily use of Blender in a production environment. Check out the list of its features here. One of my favorites is its Scene Debugging abilities, like finding the causes of missing textures. Amaranth is included with Blender if you build yourself or if you download Blender from buildbot.
Alright, I will now go through how to build the scripts and make some pointers on how you can easily add your own preferences into them:
I like to define some often adjusted values in the beginning of the script so that I don’t have to hunt for some tiny value all over the place (whether in the script or in the user interface). Later in the script these values will be called. Also, for those who don’t know, the lines marked with the #-symbol are comments. Feel free to write whatever you wish on those lines. They don’t get run with the script.
Professional programmers have some strict guidelines on what to comment in order to keep the code nice and tidy – but this tutorial is meant for artists, so:
# _____ _ # / ____| | | # | | __ ___ ___ ___ ___| |__ ___ _ __ _ __ _ _ # | | |_ |/ _ \ / _ \/ __|/ _ \ '_ \ / _ \ '__| '__| | | | # | |__| | (_) | (_) \__ \ __/ |_) | __/ | | | | |_| | # \_____|\___/ \___/|___/\___|_.__/ \___|_| |_| \__, | # __/ | # |___/ ############################################################## # # # SETTINGS # Samples = 50 Depth_of_field = 0.006
That kind of stuff can be easily generated via some ASCII generators.
The beginning of the script itself:
############################################################## # # # SCRIPT # import bpy Scene = bpy.context.scene
Most of the code we need in the script can be discovered straight from the interface of Blender. By hovering your mouse cursor over the Simplify field, for example, you can see the Python code for it. It’s not exactly the line you should be using in the script, but it gives a good hint:
Then, go to the Python Console (Shift + F4), start typing what we see on the tooltip: bpy.data.sce… and keep hitting CTRL+Space to take advantage of the autocomplete feature of the console (hitting multiple times gives you the available options). If you want to clear the line, you can use SHIFT+Enter. Eventually navigate to bpy.data.scenes[‘Scene’].render.use_simplify and hit Enter. The console should display either True or False, depending on what state the checkbox is in. Then, if it’s off, let’s try to turn it on from the Python Console: Press Up arrow key to recall the last command, and add “= True” (or 1) to the end of the line. Then, hit enter as hard as you can!
The checkbox for the Simplify should turn on. Nice!
This way, you can interactively fool around in Blender and find different kind of checkboxes, settings, knobs and whistles you want to include into your script in order to automate your life a little bit.
However, in the actual script you need to write the commands in this manner:
Scene.render.use_simplify = True Scene.render.simplify_subdivision = 0 Scene.use_unsimplify_render = True
Change the values in the script and run it from the ‘Run Script’ button in order to test how the code affects your render settings. Wonderful!
We can use this same process to add some more stuff into the script.
Okay, let’s continue.
# Render Display Mode: Scene.render.display_mode = 'SCREEN' Scene.render.use_lock_interface = True # Uncheck Post Processing: Scene.render.use_compositing = False Scene.render.use_sequencer = False
You can find the different values like the ‘SCREEN’ by changing the drop-down menu to the setting you want it to be and slithering with the Python Console to check the name of the setting. Just like we did with the Simplify:
The Depth_of_field variable is now fetching the value that we defined in the settings in the beginning of the script.
# Camera settings: Scene.camera.data.cycles.aperture_size = Depth_of_field Scene.camera.data.cycles.aperture_ratio = 2 Scene.camera.data.show_name = True
This is an example of adding a driver to a field by using Python. In here, we make different looking sampling noise pattern for each frame. (You can manually achieve this by typing #frame into the sampling seed field.)
Note: Here we’re assuming the scene doesn’t have any other drivers (scene level, character-rigs don’t count), so we’re accessing by just doing Scene.animation_data.drivers[0]. Increase the index number in case you have more drivers in the Scene: [1], [2], etc.
# Sampling seed to "#frame":
Scene.cycles.driver_add('seed')
Scene.animation_data.drivers[0].driver.expression = 'frame'
I use JPEG for saving my test renders. The setting will anyway get automatically overridden with PNG by our render farm software Flamenco.
# Output to JPEG 100% (for easy test render saving): Scene.render.image_settings.file_format = 'JPEG' Scene.render.image_settings.quality = 100
The render stamp values or strings are going to be written as metadata into the image, so we don’t need to enable the visible stamp that gets slapped on top of your precious image as pixels. (But all the other stamp checkboxes need to be enabled.) In the other startup.py script we are going to define what values or strings the stamp’s note field outputs (like the name of the rendernode or the amount of samples, for example).
In some cases you might actually want to see the stamp on your image by enabling the use_stamp. In that case, you might want to know how to place the render stamp outside the image frame.
# Disable burned-in stamp: Scene.render.use_stamp = False # Enable needed stamp values for metadata: Scene.render.use_stamp_time = True Scene.render.use_stamp_date = True Scene.render.use_stamp_render_time = True Scene.render.use_stamp_frame = True Scene.render.use_stamp_camera = True Scene.render.use_stamp_lens = True Scene.render.use_stamp_filename = True Scene.render.use_stamp_marker = True Scene.render.use_stamp_note = True
This are settings that are a good start for our shots, doesn’t mean they will benefit your own shots. Also, once we set them up, we go and tweak those that are needed. For example in a Laundromat shot where we have lots of glossy materials, it would look nicer if we had reflective caustics, but for most shots in the island we wouldn’t even notice so better disable it.
# Use Preview Range OFF: Scene.use_preview_range = False # Sampling: Scene.cycles.samples = Samples Scene.cycles.sample_clamp_direct = 11 Scene.cycles.sample_clamp_indirect = 5 Scene.cycles.volume_step_size = 1 Scene.cycles.volume_max_steps = 1024 # Light Paths: Scene.cycles.transparent_max_bounces = 8 Scene.cycles.transparent_min_bounces = 8 Scene.cycles.max_bounces = 2 Scene.cycles.min_bounces = 1 Scene.cycles.diffuse_bounces = 2 Scene.cycles.glossy_bounces = 2 Scene.cycles.transmission_bounces = 2 Scene.cycles.volume_bounces = 1 Scene.cycles.caustics_reflective = False Scene.cycles.caustics_refractive = False Scene.cycles.blur_glossy = 2
Adjusting the Cycles Hair subdivision settings for the linked hair. These settings are Scene level, so they don’t get transferred via linking. By tweaking them per shot, we can make Franck’s fur strands look nice and curly in closeup instead of low resolution for far away shots, for example.
# Hair settings: Scene.cycles_curves['primitive'] = 2 Scene.cycles_curves.subdivisions = 4 Scene.cycles_curves['shape'] = 0
This is a tad more complicated thing and I have no idea how it is built. It automatically goes into the Sequencer, selects all the movie strip files, deletes them and goes back to where you were. We want to remove the strips because when we send the file to the render farm, all the file paths inside the .blend will be automatically collected into one big .zip file by BAM – and the movie files certainly would unnecessarily boost the file size quite a bit.
In case you need more complicated scripts like this, you can always search from the web or, for example, participate to the community of the Blender Stack Exchange. Or go and find the nearest friendly and helpful Pablo in your neighborhood and ask for a little guidance.
# Remove Sequencer Strips:
class SequencerStripSlayer(bpy.types.Operator):
"""Delete All Strips"""
bl_idname = "sequencer.strip_slayer"
bl_label = "Sequencer Strip Slayer"
def execute(self, context):
# Save which editor are we in when we run the operator
where = bpy.context.area.type
# Temporary switch to the Sequencer to get context data
bpy.context.area.type = 'SEQUENCE_EDITOR'
# Build the context to override
override = {
"window": bpy.context.window,
"screen": bpy.context.screen,
"scene": context.scene,
"area": bpy.context.area,
"region": bpy.context.area.regions[0],
"blend_data": context.blend_data}
# Check if we have sequences at all
if context.scene.sequence_editor and context.scene.sequence_editor.sequences_all:
i = 0
# loop through all the strips
for s in context.scene.sequence_editor.sequences_all:
# select them!
s.select = True
# this is just to count them (for the report message)
i += 1
# DIE DIE DIE!
bpy.ops.sequencer.delete(override)
# Report, if we have more than 0
if i != 0:
self.report({"INFO"}, "BAM! {0} Strips Destroyed!".format(i))
else:
self.report({"INFO"}, "No strips to murder")
else:
self.report({"INFO"}, "No sequences")
# Go back to the area we were before
bpy.context.area.type = where
return {'FINISHED'}
The following bit is the end of this strip-removing script. It should be located at the end of the entire script. If you need to add more lines to the code for your settings, make sure you add them before this code:
def register():
bpy.utils.register_class(SequencerStripSlayer)
def unregister():
bpy.utils.unregister_class(SequencerStripSlayer)
if __name__ == "__main__":
register()
# Now actually run the operator. By commenting this line below, the delete operator will not run, but it will be registered. So you can find it via Space bar search and use it when needed.
bpy.ops.sequencer.strip_slayer()
# _____ _ # / ____| | | # | | __ ___ ___ ___ ___| |__ ___ _ __ _ __ _ _ # | | |_ |/ _ \ / _ \/ __|/ _ \ '_ \ / _ \ '__| '__| | | | # | |__| | (_) | (_) \__ \ __/ |_) | __/ | | | | |_| | # \_____|\___/ \___/|___/\___|_.__/ \___|_| |_| \__, | # __/ | # |___/ # # $$$$$$\ $$$$$$$$\ $$$$$$\ $$$$$$$\ $$$$$$$$\ $$\ $$\ $$$$$$$\ # $$ __$$\\__$$ __|$$ __$$\ $$ __$$\\__$$ __|$$ | $$ |$$ __$$\ # $$ / \__| $$ | $$ / $$ |$$ | $$ | $$ | $$ | $$ |$$ | $$ | # \$$$$$$\ $$ | $$$$$$$$ |$$$$$$$ | $$ | $$ | $$ |$$$$$$$ | # \____$$\ $$ | $$ __$$ |$$ __$$< $$ | $$ | $$ |$$ ____/ # $$\ $$ | $$ | $$ | $$ |$$ | $$ | $$ | $$ | $$ |$$ | # \$$$$$$ | $$ | $$ | $$ |$$ | $$ | $$ | \$$$$$$ |$$ | # \______/ \__| \__| \__|\__| \__| \__| \______/ \__| # ############################################################## # # # SETTINGS # # Define some names to the stamp note: Animator = 'Hjalti' Compositor = 'Manu'
Avoid overdoing the naming of your variables in the settings though, it might look messy in the code. Let’s keep it simple.
############################################################## # # # SCRIPT # import bpy import socket
The socket is needed to detect the rendernode’s name so that it can be printed into the note stamp which will be seen in the image’s metadata via some image viewer.
Like always, the best way to find out what to type into your script is to first mess around in the viewport and the Python Console for a bit:
As you can see in the example above, back in the days when we were creating the environment set for the scene called Meet Victor, we deliberately decided to leave some not-so-clearly named particle systems in there in order to illustrate for you guys now how difficult it is to find them from the Python Console if they don’t have some good and descriptive names. So, it’s always sensible to rename the particle settings, the particle system itself and also the particle system modifier in the modifier panel with the same name, if possible. In the future it might be that the whole particle system engine thingie gets easier to use, but I’m not the right person to speak about that. So, like always in life, let’s concentrate on what we have now and enjoy it :)
Below you can find the code bits I used in the above example. These depend so much from your own scene that they really need to be dug out on case-by-case basis. Sometimes you need to open up the linked file itself (which is super-easy and fast in Amaranth) in order to find the Python settings you are looking for by using the methods mentioned in the previous settings.py section of this post.
# Particle systems: bpy.data.objects['GEO-cliff_ground_high'].modifiers['ParticleSystem 3'].show_viewport = 1 bpy.data.particles['cliff_grass.001'].count = 12000 bpy.data.particles['cliff_grass.001'].hair_length = 200
Here’s also a couple more examples of how to not render a particle system or change the seed of it (in order to get different kind of randomization):
bpy.data.objects['GEO-cliff_ground_high'].modifiers['ParticleSystem 3'].show_render = 0 bpy.data.objects['GEO-cliff_ground_high'].particle_systems['island_plants_small_dead'].seed = 16
Below you can find the script to generate the note for the stamp metadata so that it includes relevant extra information about the .blend file. It’s run each time Blender starts in order to also collect some information about the specific computer the .blend is opened in, like the rendernode’s name.
In addition to that, the stamp note will include the samples amount, camera’s DOF amount, Blender version and the names of the fellows who worked on the shot (that were defined in the beginning of this script).
When the frame is rendered, you can view the metadata of the image in a feature-rich image viewer, like the (freeware for personal use) XnView MP by using the ExifTool that comes with it. For our Gooseberry project though (where only Open Source tools are in use), I really like the nomacs Image Lounge (Win/Mac/Linux). In the 2.4.5 development version they actually support the Blender’s PNG metadata! (Download here the Nomacs 2.4.5 .deb installer for a 64-bit Ubuntu based OS – but remember, it’s not an official stable release)
Blender’s PNG metadata in nomacs Image Lounge 2.4.5 development version in 64-bit Xubuntu Linux
You can also use Blender’s image viewer to display the metadata information.
# Stamp note script:
def stamp_set(scene):
render = scene.render
is_cycles = (render.engine == 'CYCLES')
render.use_stamp_note = True
render.stamp_note_text = (
"Samples: {samples} | "
"Aperture Radius: {aperture:.4f} | "
"Blender {ver} {branch} {hash} | "
"Animator: {anim} | "
"Compositor: {comp} | "
"Rendernode: {hostname}"
).format(
samples=scene.cycles.samples if is_cycles else render.antialiasing_samples,
aperture=scene.camera.data.cycles.aperture_size if is_cycles else "NONE",
ver=bpy.app.version_string,
branch=bpy.app.build_branch,
hash=bpy.app.build_hash,
hostname=socket.gethostname(),
anim=Animator,
comp=Compositor,
)
bpy.app.handlers.render_pre.append(stamp_set)
The following script will override a keyed value in, for example, one of Victor’s controls in the armature. Here it switches the keyed toggle for Victor’s hair particles OFF when the file is first loaded (even though it’s animated to be ON all the time).
# Override keyed PARTICLES_TOGGLE value: from bpy.app.handlers import persistent @persistent def load_handler(dummy): bpy.data.objects['victor_high_proxy.001'].pose.bones['properties']['PARTICLES_TOGGLE'] = 0 bpy.app.handlers.render_pre.append(load_handler)
Also, if you want to mute/unmute animation channels for, for example, handheld Camera’s noise modifier curves, use this:
bpy.data.objects['camera'].animation_data.action.fcurves[0].mute = True #(or False)
Do it to other channels by increasing the index [0], [1], [2], etc.
EDIT 2015-04-24:
Pablo made a nice node for Franck to control the amount of color variation that Andy had added to Franck’s wool shader. Here’s a quick demonstration on how you can use Python to control that node’s value with Python. It works very well for a linked character:
I also noticed Pablo’s famous trick of using simple planes to block light when he lights a scene. I just thought it had to be mentioned if someone finds it helpful:
Here you can download the finished script templates: settings.py, startup.py
I hope you have found this post useful for your projects :)
Cheers!!
mj
Gyus,
pinterest links does not works as supposed – “We’re having some trouble talking to gooseberry.blender.org. Please try again later! ”
Cheers
Jozef
Instead of locking the interface, why not run Blender in batch mode to do the render?
I wrote a pair of wrapper scripts for this: https://github.com/ldo/render-useful
Ah, yes we actually use both methods, UI and Terminal. Locking the UI just makes the scene less prone for crashing during rendering. And also, I have understood, it frees up memory a little bit. Terminal rendering is very useful when simulating how the scene works in the renderfarm.
I must admit the scripts you’ve made are complicated enough for me not to know how take advantage of them – but I’m sure other people will find them wonderful :) Thanks for providing them!
mj
Yes, the scripts look complex, because they have lots of options, and I keep adding more. :)
But you can do simple things with them, like stereo renders. And rendering animation sequences. And stereo animations. And renumbering animation frames to get around Blender’s 4-digit numbering limit.
I also added another script called ext-files, which reports on external files (fonts, images, libraries, sounds) linked to from a .blend file, and whether they can be found or not. Useful when you move .blend files around.
Thank you for this examples, the most interesting part for me was this sequencer magic. Could you please provide us also with your ext-files script? Just to be complete :)
cheers! Thank you!