By Jeremy Birn.

If you are rendering multiple shadows, then you don't want them all to be combined together in the alpha channel of your shadow pass: You want them each to be separately manipuable in the composite.

There is a better way to render a shadow pass in Maya, which avoids the limitiations of Maya's default shadow pass preset: turn Shadow Pass rendering off for your render layer, and create your own shadow pass layer by overriding the shadow colors on your lights. For the lights that will be used in your shadow pass, set the light's Color to black {0,0,0} and set the shadow color to a pure primary such as red, green, or blue.

To isolate just the cast shadows, without getting any attached shadows in the pass, the shadow-casting object needs its primary visibility turned off during the shadow pass render.

Use a different color for different shadows if you want them to be separatable in the composite. You may want to make a separate group of lights that are just visible in your shadow pass, and leave another group of lights in your scene that will be made visible for your hero render when you aren't rendering a shadow pass. Your shadow pass will look something like this.

Because the key light shadow is green, and the fill light shadow is red, they are easy to separate in the composite. In Shake, a reorder node is used to pull out rrrr (all red) and gggg (all green) from the shadow pass rendering. Gamma adjustment can be made to the shadow from the green channel, and a blur can be added to the red channel, separately adjusting each of the shadows in the shadow pass.

This simple example only uses two of the color channels. If you wanted to put another shadow or some occlusion into a third channel, you could get even more out of your shadow pass. This technique seems to work fine in the Maya Renderer or Mental Ray, with raytraced or depth mapped shadows. This technique does not necessarily work in all other software packages - although if you write your shaders correctly, a three-pack shadow pass should be achievable using arbitrary output variables in Renderman just as well as we set it up in Maya.

3dRender Home | Digital Lighting & Rendering

Copyright © 2005 by Jeremy Birn.