Tools

Simple Python tips for Artists

15 April, 2015 |  5 Comments | by Manu Järvinen

Hi there!

Pablo taught me a little bit of simple artist-friendly Python and how we lighters/compositors can utilize it for preparing the animated shots for the render farm – in other words: To Automate our Workflow. Hopefully it will benefit some of you, dear readers, as well :)

A little bit about the lighting/compositing process

As an example, I will use the shot 1.2.1.E1, animated by Hjalti.svnThanks to the linking capabilities of Blender it’s easy for me to start to light the scene in a separate .comp file even if Hjalti is still animating. The latest animation changes from the .anim file will automatically get updated to my .comp file when I update my Gooseberry Repository in our SVN and reload the file in Blender. If you don’t know what the SVN is, I can try to search for some blog posts about it from the Mango, Durian, Peach and Orange blogs and edit it here if I find one. In the mean time, the web is your friend. There will also be a more detailed tutorial about the linking process later. – However, when Lukas and the fellows get the Alembic caching and related things to work reliably for production, we will do things in a totally different manner – that also could be an excellent post topic for the future. But for now, let’s just concentrate on how the things are done at this moment in Gooseberry.

So, when the hi-res versions of the environment set, lights, fog, random stones and bushes, Victor, Franck and the animation actions are linked into the .comp file, some of their properties are locked and can’t be edited because of how the linking in Blender currently functions. Also, there are a whole bunch of rendering related and other settings that need to be adjusted before sending the file to the farm. Some of the render settings could be individually put by hand, of course, but automating things is a lot more efficient (and fun :D) way to handle that. This is a perfect case for us to use some Python scripting.

With it we can do with a single button press:

  • set on/off state for various checkboxes in settings
  • set different values for all kinds of fields in settings
  • hide unnecessary particle systems from the linked environments or change their seed or particle size parameters (which naturally don’t affect the settings in the original source file that is being linked)
  • override Victor’s keyed parameters, like double the size of his head if needed
  • remove all the video file strips from the Video Sequencer that animators use as reference (so the file is lighter, simpler, and BAM doesn’t pack those unneded strips)
  • set render stamp metadata information to the image frames like the amount of samples, render time or the name of the rendernode
  • just about everything!

Okay, let’s begin!

Automate with Python

I have two datablocks in the Text Editor of Blender (Shift + F11):

  1. settings.py – This script is for some render settings, preferences, and some scene&viewport optimizing to prepare your scene. Execute it from the ‘Run Script’ button.
  2. startup.py – This script controls the visibility and attributes of some of the particle systems and also some renderstamp features. It will run every time the .blend is opened (important for the farm). To make it to do so, the option ‘Register’ needs to be checked.

The settings.py text datablock can be freely renamed to whatever, but the startup.py must have the .py ending. Otherwise it won’t get registered (in fact, the “Register” checkbox will be grayed out if the name doesn’t end with .py).

There are links at the end of this post to download the full example scripts.


As a side note, it must be mentioned that Pablo’s Amaranth Add-on has also been an invaluable tool for easing up the daily use of Blender in a production environment. Check out the list of its features here. One of my favorites is its Scene Debugging abilities, like finding the causes of missing textures. Amaranth is included with Blender if you build yourself or if you download Blender from buildbot.


Alright, I will now go through how to build the scripts and make some pointers on how you can easily add your own preferences into them:

 

settings.py:

 

1. Define some settings

I like to define some often adjusted values in the beginning of the script so that I don’t have to hunt for some tiny value all over the place (whether in the script or in the user interface). Later in the script these values will be called. Also, for those who don’t know,  the lines marked with the #-symbol are comments. Feel free to write whatever you wish on those lines. They don’t get run with the script.

Professional programmers have some strict guidelines on what to comment in order to keep the code nice and tidy – but this tutorial is meant for artists, so:


#      _____                      _                          
#     / ____|                    | |                         
#    | |  __  ___   ___  ___  ___| |__   ___ _ __ _ __ _   _
#    | | |_ |/ _ \ / _ \/ __|/ _ \ '_ \ / _ \ '__| '__| | | |
#    | |__| | (_) | (_) \__ \  __/ |_) |  __/ |  | |  | |_| |
#     \_____|\___/ \___/|___/\___|_.__/ \___|_|  |_|   \__, |
#                                                       __/ |
#                                                      |___/ 

##############################################################
#                                                            #
#                         SETTINGS                           #

Samples = 50
Depth_of_field = 0.006

That kind of stuff can be easily generated via some ASCII generators.


2. Import Blender Python, define the scene

The beginning of the script itself:

##############################################################
#                                                            #
#                          SCRIPT                            #

import bpy
Scene = bpy.context.scene

3. Enable Simplify, set its subdivision amount to zero and also enable Amaranth’s Unsimplify Render

Most of the code we need in the script can be discovered straight from the interface of Blender. By hovering your mouse cursor over the Simplify field, for example, you can see the Python code for it. It’s not exactly the line you should be using in the script, but it gives a good hint:

tooltip

 

Then, go to the Python Console (Shift + F4), start typing what we see on the tooltip: bpy.data.sce… and keep hitting CTRL+Space to take advantage of the autocomplete feature of the console (hitting multiple times gives you the available options). If you want to clear the line, you can use SHIFT+Enter. Eventually navigate to bpy.data.scenes[‘Scene’].render.use_simplify and hit Enter. The console should display either True or False, depending on what state the checkbox is in. Then, if it’s off, let’s try to turn it on from the Python Console: Press Up arrow key to recall the last command, and add = True” (or 1) to the end of the line. Then, hit enter as hard as you can!

The checkbox for the Simplify should turn on. Nice!

python_console

This way, you can interactively fool around in Blender and find different kind of checkboxes, settings, knobs and whistles you want to include into your script in order to automate your life a little bit.

However, in the actual script you need to write the commands in this manner:

Scene.render.use_simplify = True
Scene.render.simplify_subdivision = 0
Scene.use_unsimplify_render = True

Change the values in the script and run it from the ‘Run Script’ button in order to test how the code affects your render settings. Wonderful!

We can use this same process to add some more stuff into the script.

Okay, let’s continue.


4. Display rendered images fullscreen, lock interface during rendering , uncheck post processing stuff

display_post_processing

# Render Display Mode:
Scene.render.display_mode = 'SCREEN'
Scene.render.use_lock_interface = True

# Uncheck Post Processing:
Scene.render.use_compositing = False
Scene.render.use_sequencer = False

You can find the different values like the ‘SCREEN’ by changing the drop-down menu to the setting you want it to be and slithering with the Python Console to check the name of the setting. Just like we did with the Simplify:

value


5. Camera settings for the Scene Camera

The Depth_of_field variable is now fetching the value that we defined in the settings in the beginning of the script.


# Camera settings:
Scene.camera.data.cycles.aperture_size = Depth_of_field
Scene.camera.data.cycles.aperture_ratio = 2
Scene.camera.data.show_name = True


6. Putting the #frame driver to the sampling seed

This is an example of adding a driver to a field by using Python. In here, we make different looking sampling noise pattern for each frame. (You can manually achieve this by typing #frame into the sampling seed field.)

Note: Here we’re assuming the scene doesn’t have any other drivers (scene level, character-rigs don’t count), so we’re accessing by just doing Scene.animation_data.drivers[0]. Increase the index number in case you have more drivers in the Scene: [1], [2], etc.


# Sampling seed to "#frame":
Scene.cycles.driver_add('seed')
Scene.animation_data.drivers[0].driver.expression = 'frame'


7. Image format settings

I use JPEG for saving my test renders. The setting will anyway get automatically overridden with PNG by our render farm software Flamenco.


# Output to JPEG 100% (for easy test render saving):
Scene.render.image_settings.file_format = 'JPEG'
Scene.render.image_settings.quality = 100


8. Stamp metadata settings

The render stamp values or strings are going to be written as metadata into the image, so we don’t need to enable the visible stamp that gets slapped on top of your precious image as pixels. (But all the other stamp checkboxes need to be enabled.) In the other startup.py script we are going to define what values or strings the stamp’s note field outputs (like the name of the rendernode or the amount of samples, for example).

In some cases you might actually want to see the stamp on your image by enabling the use_stamp. In that case, you might want to know how to place the render stamp outside the image frame.


# Disable burned-in stamp:
Scene.render.use_stamp = False

# Enable needed stamp values for metadata:
Scene.render.use_stamp_time = True
Scene.render.use_stamp_date = True
Scene.render.use_stamp_render_time = True
Scene.render.use_stamp_frame = True
Scene.render.use_stamp_camera = True
Scene.render.use_stamp_lens = True
Scene.render.use_stamp_filename = True
Scene.render.use_stamp_marker = True
Scene.render.use_stamp_note = True


9. Preview range off, lighting settings

This are settings that are a good start for our shots, doesn’t mean they will benefit your own shots. Also, once we set them up, we go and tweak those that are needed. For example in a Laundromat shot where we have lots of glossy materials, it would look nicer if we had reflective caustics, but for most shots in the island we wouldn’t even notice so better disable it.


# Use Preview Range OFF:
Scene.use_preview_range = False

# Sampling:
Scene.cycles.samples = Samples
Scene.cycles.sample_clamp_direct = 11
Scene.cycles.sample_clamp_indirect = 5
Scene.cycles.volume_step_size = 1
Scene.cycles.volume_max_steps = 1024

# Light Paths:
Scene.cycles.transparent_max_bounces = 8
Scene.cycles.transparent_min_bounces = 8
Scene.cycles.max_bounces = 2
Scene.cycles.min_bounces = 1
Scene.cycles.diffuse_bounces = 2
Scene.cycles.glossy_bounces = 2
Scene.cycles.transmission_bounces = 2
Scene.cycles.volume_bounces = 1
Scene.cycles.caustics_reflective = False
Scene.cycles.caustics_refractive = False
Scene.cycles.blur_glossy = 2


10. Hair settings

Adjusting the Cycles Hair subdivision settings for the linked hair. These settings are Scene level, so they don’t get transferred via linking. By tweaking them per shot, we can make Franck’s fur strands look nice and curly in closeup instead of low resolution for far away shots, for example.


# Hair settings:
Scene.cycles_curves['primitive'] = 2
Scene.cycles_curves.subdivisions = 4
Scene.cycles_curves['shape'] = 0


11. Lastly, remove the strips from Sequencer

This is a tad more complicated thing and I have no idea how it is built. It automatically goes into the Sequencer, selects all the movie strip files, deletes them and goes back to where you were. We want to remove the strips because when we send the file to the render farm, all the file paths inside the .blend will be automatically collected into one big .zip file by BAM – and the movie files certainly would unnecessarily boost the file size quite a bit.

In case you need more complicated scripts like this, you can always search from the web or, for example, participate to the community of the Blender Stack Exchange. Or go and find the nearest friendly and helpful Pablo in your neighborhood and ask for a little guidance.


# Remove Sequencer Strips:
class SequencerStripSlayer(bpy.types.Operator):
    """Delete All Strips"""
    bl_idname = "sequencer.strip_slayer"
    bl_label = "Sequencer Strip Slayer"

    def execute(self, context):

        # Save which editor are we in when we run the operator
        where = bpy.context.area.type

        # Temporary switch to the Sequencer to get context data
        bpy.context.area.type = 'SEQUENCE_EDITOR'

        # Build the context to override
        override = {
            "window": bpy.context.window,
            "screen": bpy.context.screen,
            "scene": context.scene,
            "area": bpy.context.area,
            "region": bpy.context.area.regions[0],
            "blend_data": context.blend_data}

        # Check if we have sequences at all
        if context.scene.sequence_editor and context.scene.sequence_editor.sequences_all:

            i = 0

            # loop through all the strips
            for s in context.scene.sequence_editor.sequences_all:

                # select them!
                s.select = True

                # this is just to count them (for the report message)
                i += 1

            # DIE DIE DIE!
            bpy.ops.sequencer.delete(override)

            # Report, if we have more than 0
            if i != 0:
                self.report({"INFO"}, "BAM! {0} Strips Destroyed!".format(i))
            else:
                self.report({"INFO"}, "No strips to murder")
        else:
            self.report({"INFO"}, "No sequences")

        # Go back to the area we were before
        bpy.context.area.type = where

        return {'FINISHED'} 

The following bit is the end of this strip-removing script. It should be located at the end of the entire script. If you need to add more lines to the code for your settings, make sure you add them before this code:


def register():
    bpy.utils.register_class(SequencerStripSlayer)

def unregister():
    bpy.utils.unregister_class(SequencerStripSlayer)

if __name__ == "__main__":
    register()

# Now actually run the operator. By commenting this line below, the delete operator will not run, but it will be registered. So you can find it via Space bar search and use it when needed.
bpy.ops.sequencer.strip_slayer()


 

 

startup.py:

 

1. Define some settings again

#      _____                      _                          
#     / ____|                    | |                         
#    | |  __  ___   ___  ___  ___| |__   ___ _ __ _ __ _   _
#    | | |_ |/ _ \ / _ \/ __|/ _ \ '_ \ / _ \ '__| '__| | | |
#    | |__| | (_) | (_) \__ \  __/ |_) |  __/ |  | |  | |_| |
#     \_____|\___/ \___/|___/\___|_.__/ \___|_|  |_|   \__, |
#                                                       __/ |
#                                                      |___/
#
#  $$$$$$\ $$$$$$$$\  $$$$$$\  $$$$$$$\ $$$$$$$$\ $$\   $$\ $$$$$$$\  
# $$  __$$\\__$$  __|$$  __$$\ $$  __$$\\__$$  __|$$ |  $$ |$$  __$$\
# $$ /  \__|  $$ |   $$ /  $$ |$$ |  $$ |  $$ |   $$ |  $$ |$$ |  $$ |
# \$$$$$$\    $$ |   $$$$$$$$ |$$$$$$$  |  $$ |   $$ |  $$ |$$$$$$$  |
#  \____$$\   $$ |   $$  __$$ |$$  __$$<   $$ |   $$ |  $$ |$$  ____/
# $$\   $$ |  $$ |   $$ |  $$ |$$ |  $$ |  $$ |   $$ |  $$ |$$ |      
# \$$$$$$  |  $$ |   $$ |  $$ |$$ |  $$ |  $$ |   \$$$$$$  |$$ |      
#  \______/   \__|   \__|  \__|\__|  \__|  \__|    \______/ \__|      
#
##############################################################
#                                                            #
#                         SETTINGS                           #

# Define some names to the stamp note:
Animator = 'Hjalti'
Compositor = 'Manu'

Avoid overdoing the naming of your variables in the settings though, it might look messy in the code. Let’s keep it simple.


2. Import Blender Python and socket

##############################################################
#                                                            #
#                          SCRIPT                            #

import bpy
import socket

The socket is needed to detect the rendernode’s name so that it can be printed into the note stamp which will be seen in the image’s metadata via some image viewer.


3. Adjusting the linked particle systems

Like always, the best way to find out what to type into your script is to first mess around in the viewport and the Python Console for a bit:

particle_python

As you can see in the example above, back in the days when we were creating the environment set for the scene called Meet Victor, we deliberately decided to leave some not-so-clearly named particle systems in there in order to illustrate for you guys now how difficult it is to find them from the Python Console if they don’t have some good and descriptive names. So, it’s always sensible to rename the particle settings, the particle system itself and also the particle system modifier in the modifier panel with the same name, if possible. In the future it might be that the whole particle system engine thingie gets easier to use, but I’m not the right person to speak about that. So, like always in life, let’s concentrate on what we have now and enjoy it :)

Below you can find the code bits I used in the above example. These depend so much from your own scene that they really need to be dug out on case-by-case basis. Sometimes you need to open up the linked file itself (which is super-easy and fast in Amaranth) in order to find the Python settings you are looking for by using the methods mentioned in the previous settings.py section of this post.


# Particle systems:
bpy.data.objects['GEO-cliff_ground_high'].modifiers['ParticleSystem 3'].show_viewport = 1
bpy.data.particles['cliff_grass.001'].count = 12000
bpy.data.particles['cliff_grass.001'].hair_length = 200

Here’s also a couple more examples of how to not render a particle system or change the seed of it (in order to get different kind of randomization):

bpy.data.objects['GEO-cliff_ground_high'].modifiers['ParticleSystem 3'].show_render = 0
bpy.data.objects['GEO-cliff_ground_high'].particle_systems['island_plants_small_dead'].seed = 16


4. Stamp metadata note

Below you can find the script to generate the note for the stamp metadata so that it includes relevant extra information about the .blend file. It’s run each time Blender starts in order to also collect some information about the specific computer the .blend is opened in, like the rendernode’s name.

In addition to that, the stamp note will include the samples amount, camera’s DOF amount, Blender version and the names of the fellows who worked on the shot (that were defined in the beginning of this script).

When the frame is rendered, you can view the metadata of the image in a feature-rich image viewer, like the (freeware for personal use) XnView MP by using the ExifTool that comes with it. For our Gooseberry project though (where only Open Source tools are in use), I really like the nomacs Image Lounge (Win/Mac/Linux). In the 2.4.5 development version they actually support the Blender’s PNG metadata! (Download here the Nomacs 2.4.5 .deb installer for a 64-bit Ubuntu based OS – but remember, it’s not an official stable release)

Nomacs_image_viewer_png_metadata

Blender’s PNG metadata in nomacs Image Lounge 2.4.5 development version in 64-bit Xubuntu Linux

You can also use Blender’s image viewer to display the metadata information.


# Stamp note script:
def stamp_set(scene):

    render = scene.render
    is_cycles = (render.engine == 'CYCLES')

    render.use_stamp_note = True
    render.stamp_note_text = (
        "Samples: {samples}  |  "
        "Aperture Radius: {aperture:.4f}  |  "
        "Blender {ver} {branch} {hash}  |  "
        "Animator: {anim}  |  "
        "Compositor: {comp}  |  "
        "Rendernode: {hostname}"
    ).format(
        samples=scene.cycles.samples if is_cycles else render.antialiasing_samples,
        aperture=scene.camera.data.cycles.aperture_size if is_cycles else "NONE",
        ver=bpy.app.version_string,
        branch=bpy.app.build_branch,
        hash=bpy.app.build_hash,
        hostname=socket.gethostname(),
        anim=Animator,
        comp=Compositor,
        )

bpy.app.handlers.render_pre.append(stamp_set)


5. Override a keyed value

The following script will override a keyed value in, for example, one of Victor’s controls in the armature. Here it switches the keyed toggle for Victor’s hair particles OFF when the file is first loaded (even though it’s animated to be ON all the time).


# Override keyed PARTICLES_TOGGLE value:
from bpy.app.handlers import persistent

@persistent
def load_handler(dummy):
 bpy.data.objects['victor_high_proxy.001'].pose.bones['properties']['PARTICLES_TOGGLE'] = 0

bpy.app.handlers.render_pre.append(load_handler)

 

Also, if you want to mute/unmute animation channels for, for example, handheld Camera’s noise modifier curves, use this:

bpy.data.objects['camera'].animation_data.action.fcurves[0].mute = True
#(or False)

Do it to other channels by increasing the index [0], [1], [2], etc.


Resolving errors


 EDIT 2015-04-24:

Pablo made a nice node for Franck to control the amount of color variation that Andy had added to Franck’s wool shader. Here’s a quick demonstration on how you can use Python to control that node’s value with Python. It works very well for a linked character:

python_for_material_node_overriding

I also noticed Pablo’s famous trick of using simple planes to block light when he lights a scene. I just thought it had to be mentioned if someone finds it helpful:

pablos_sun_blocker_trick


 

Here you can download the finished script templates: settings.py, startup.py

I hope you have found this post useful for your projects :)

Cheers!!

mj

, ,



5 Responses

  1. Gyus,

    pinterest links does not works as supposed – “We’re having some trouble talking to gooseberry.blender.org. Please try again later! ”

    Cheers

    Jozef

  2. Lawrence D’Oliveiro says:

    Instead of locking the interface, why not run Blender in batch mode to do the render?

    I wrote a pair of wrapper scripts for this: https://github.com/ldo/render-useful

    • Ah, yes we actually use both methods, UI and Terminal. Locking the UI just makes the scene less prone for crashing during rendering. And also, I have understood, it frees up memory a little bit. Terminal rendering is very useful when simulating how the scene works in the renderfarm.

      I must admit the scripts you’ve made are complicated enough for me not to know how take advantage of them – but I’m sure other people will find them wonderful :) Thanks for providing them!

      mj

      • Lawrence D’Oliveiro says:

        Yes, the scripts look complex, because they have lots of options, and I keep adding more. :)

        But you can do simple things with them, like stereo renders. And rendering animation sequences. And stereo animations. And renumbering animation frames to get around Blender’s 4-digit numbering limit.

        I also added another script called ext-files, which reports on external files (fonts, images, libraries, sounds) linked to from a .blend file, and whether they can be found or not. Useful when you move .blend files around.

        • piotr says:

          Thank you for this examples, the most interesting part for me was this sequencer magic. Could you please provide us also with your ext-files script? Just to be complete :)
          cheers! Thank you!

Leave a Reply

Your email address will not be published. Required fields are marked *