Arnold Brushstroke shader (c++/osl)

IMPRESSIONISM IN PATH TRACERS (ARNOLD C++/OSL)

For a while I've been thinking how we can use all the goodies from path tracers, such as bounced light, soft shadows and estabilished animation pipelines, but use it to make impressionistic images instead. Path traced images often suffer from perfectness. Reference paintings contain a lot of character that can be lost in the pathtracing process.

It turns out that we can actually do this in a temporally stable manner.

Note: I'd love to cg-supe a project with this art style/technique. Contact me if interested.

Buy these shaders

I've included all of the scene files (geo, textures, houdini point cloud setup + houdini shader setup, additional maya shader setup, custom c++/osl shaders). You will be able to render out this image yourself. The minimum required arnold core version is 7.1.1.0. The binaries are compiled for windows, osx and linux.

It will be possible to pick up the scene files and get going without understanding all the information at the bottom of this page. I just wanted to explain the workflow in case someone wants to replicate it in another renderer.

Key elements to simulate

1. Quantized patches of color, aka brushstrokes. Disregarding the mixing of colours, the basic principle of painting is to put some colour on your brush and put it on the canvas.
2. Break up object edges. Nothing shouts "hey i'm 3d" as much as a perfect object edge.
3. Correct layering of brushstrokes. Painters often start with big brushstrokes and detail selectively with small brushstrokes.
4. Correct orientation of the brushstrokes.

The answer to this ended up being a rendered point cloud as camera-facing cards. Ofcourse not straight out of the box, so let's dive into the details:

The shader setup, open the node editor in the maya scene to bring this up. The shader setup for Houdini is also included.

Quantized patches of colour

I developed a custom c++ shader (quantize) to sample a singular shading point on the underlying surface, for every shading point per card. This is possible because we feed the shader userdata per-point. This allows us to take the pathtraced final colour of an underlying object, and transform it to the brushstroke that is on top of it.

The reality is slightly more complex, but you can imagine the shader shooting a ray from the userdata::worldpos point, in the direction of the userdata::worldvel vector. Then it will return the radiance at the first hit point, which happens to be the point it was spawned at. Since all of the shadingpoints per card do this same lookup, we get the same colour for each shadingpoint on the card.

Breaking of object edges

This is a significant reason why cards are the right approach. You can really make the object edges look like brushstrokes. The problem actually becomes the inverse. For the big brush stroke layer (base coat), we don't want this to happen since it'd screw up the silhouette! I wrote another c++ shader to handle this (quantize_cut_edge).

Correct layering of brushstrokes

It's important to layer multiple sizes of brushstrokes. Start with a sparse pointcloud with large cards, and layer smaller sets on top to selectively add detail into places where your eye should focus.

Work from large to small strokes

Correct orientation of brushstrokes

The cards need to aligned to the surface, or a custom direction, in screenspace. To handle this I wrote some OSL (align_uvcoords_to_vec.osl) that takes care of everything. You input the position/direction userdata of the points, it outputs rotated UV coordinates that you can feed into an image.

To show the automatic surface alignment, I replaced the brushstrokes with some arrows.

See how in the above example I faded out the cards whose supplied vector is perpendicular to the viewing vector? The rotation there is a little bit too fast, so we mask it out. It turns out we can safely do this without much visual impact at all. I supplied another bit of osl for that (facingratio_cam.osl).

If you want to paint a custom direction for the strokes, that's possible too. For example, around the snout of the fox it was key I aligned the brushstrokes so that they follow the colour boundary. That edge needs to be sharp! The same shader is used, only with a different input vector. E.g in Houdini, it's very easy to paint these vectors.

The snout needed custom oriented vectors that followed the texture.

Point cloud setup

Note: The /setup/houdini/pointcloud_setup_with_shading.hip contains the whole setup.

Looking at the shading node graph you can see the whole workflow relies on 2 simple things.

- Each point holds a world space position (userdata_worldpos).
- Each point holds a vector, which is the surface normal at which the point originated. (userdata_worldvel)

There's also a (userdata_flow) node, but that's merely a custom vector that replaces (userdata_worldvel)

This is all the "custom setup" to make this work. It's rather straightforward. I used Houdini to generate the points. This can be done in pretty much any software though. I'm simply familiar with Houdini.

The basic idea is as follows: Generate points on a surface. Move the points away from the surface, e.g along the surface normal. Store the vector between this new point and the original point. Also store the new point position.

VEX (point wrangle)

To make the pointcloud render as cards, we need to set an arnold attribute (mode:quad).

The rendered cards are ONLY visible to camera rays. It doesn't cast shadows, nothing, nada. Just visible to primary rays.

The quantize, quantize_cut_edge shaders have a traceset parameter. Make sure to add that same trace set on any object that should _not_ be included in the sampling process. In this case, that's the points themselves.

At this point I moved away from houdini and did the shading in maya - but there's honestly no good reason for that. Just to show you that you can easily transfer this pointcloud between DCC's using .ass files.

Note: Render-settings wise the only thing that stands out is having to deal with a huge number of transparent surfaces. In this example I set the transparency depth to 700.

Note: I've found that the fastest way to work is to light your scene with the brushstrokes disabled. Then you can render out a low-res render, and project that on *all* the objects in the scene as emissive shader. This replaces the monte carlo simulation with a simple texture lookup, speeding up the rendering significantly. This makes it very fast to fine-tune the look of the brushstrokes.

Conclusion

The beauty of this technique is that you can start with extremely simple geometry and extremely simple shaders. I think the most exciting prospect of this journey is that it actually fits into current animation pipelines. It's temporally stable because we're working with 3D data.

Installation

Requirements

The minimum Arnold core version is 7.1.1.0.

MtoA

Option 1: Set the following environment variables, replacing $PATH_TO_FOLDER with the actual path on your machine

ARNOLD_PLUGIN_PATH = $PATH_TO_FOLDER/bin/$OS/7.1.1.0

Option 2: It’s also possible to copy the files into your MtoA install. Copy the files like this:

Files in /bin/7.1.1.0 go to $MTOA_LOCATION/shaders

HtoA

Step 1 - Option 1: Set the following environment variables, replacing $PATH_TO_FOLDER with the actual path on your machine and $OS with your operating system (windows/linux/osx).

ARNOLD_PLUGIN_PATH = $PATH_TO_FOLDER/bin/$OS/7.1.1.0

Step 1 - Option 2: It’s also possible to copy the files into your HtoA install. Copy the files like this:

Files in /bin/7.1.1.0 go to $HTOA_LOCATION/arnold/plugins

C4DToA

Files in /bin/$OPERATING_SYSTEM/7.1.1.0 go to $C4D_INSTALL/plugins/C4DtoA/shaders

More information can be found here.

MaxToA

Files in /bin/$OPERATING_SYSTEM/7.1.1.0 go to $3DSMAX_LOCATION/Plugins/MaxToA