HD Index Painting in Photoshop | Pixel Art
Dan Fessler is an artist and independent game developer currently working on Chasm. He writes about art, technology, and game dev. You can follow him on twitter @DanFessler
Building off from last post’s topic about Pixel Purism; how it’s more important where you arrived than how you got there. I wanted to share with you a method I’ve been developing with my environment work on Chasm for turning Photoshop into the most powerful pixel art tool using what I’m calling “HD index painting”. I’ve shown it to a number of people and each time the reception has been through the roof. If you’re unsure what index painting is, then I recommend you read the previous blog post as it defines it as well as several other terms that we will be expanding upon here.
If you remember from last time; Index painting involves using “dirty” tools (tools which lay multiple colors at once) within a pre-defined indexed palette. Photoshop doesn’t inherently allow for this behavior. If you index an image in Photoshop all of the “dirty” tools become locked or reduce their functionality to not be dirty. No more soft brushes, smudges, gradients, etc. The one way around this is to paint with dirty tools first, then index the image to either a generated palette or a pre-defined one (Kiwi has excellent tutorials on this method here). But with that method you never truly know what the end result is going to be until you index the image. A lot of what you thought would look nice when color-reduced ends up looking like a mess or gets lost entirely and requires you to spend lots of time cleaning up the image pixel-by-pixel. It’s a great way to get a head-start on a composition, but still cumbersome. But what if I told you there was a way to use soft brushes, smudges, gradients, and all your favorite dirty tools, but remain within an indexed palette? It’d blow your mind right? Well here’s how.
HD Index Painting
Why “HD?” The backbone of this method that makes all the magic happen is the concept of manipulating higher resolution data than the end result. Instead of painting directly with an indexed color palette, you’re always painting in HD which then procedurally gets mapped to the indexed palette. For every color ramp you define your source data is 8bit allowing for a possible 256 colors, with no limit on the number of ramps you define. Typically when dealing with pixel art you’d map that to 32 colors or less. What’s most important though is the indexing is non-destructive. All of the high resolution detail is retained allowing you to do things that no other index painting tool allows, essentially making this the most powerful “pure” pixel art tool in existence.
You can download the PSD used to create the following gifs HERE
So what can it do?:
Pixel-bush, AA-brush, Soft-brush
Fixed index Adjustments
Dynamic index Adjustments
The first thing you’ll want to do is make sure your eyedropper tool is set to sample “current and below.” This is essential if you plan to paint with colors sampled from the canvas as we’ll be adding a whole bunch of layers stacked on top of the paint layer which will alter the colors that you’ll want to paint with. Some versions of Photoshop might not support this, in which case you might have to pick by changing the value slider.
Second thing you'll want to do is make sure you have your dither patterns defined. Dither patterns can be anything you want, so get creative. To create a new pattern, simply paint the pattern in black and white within a square (usually no larger than 16x16 pixels. Once you're done painting, select the square with the selection tool and go to EDIT > DEFINE PATTERN. Now the pattern will be available to you in your pattern list for use with the paint bucket tool. The image above shows some examples of various patterns applied to a gradient. you can download the basic dither pattern I made here.
That's pretty much all the settings you'll need to change actually.
Basic Layer Stack
This is where all the magic happens. The idea behind the method is basically a stack of adjustment layers that get clipped to a HD-paint layer. Clipping the layers ensures that the stack *only* effects the pixels of that particular paint layer rather than the whole document. To clip/unclip layers, hold down the ALT or OPTION key directly in between two layers in the layers palette. You'll see your cursor change. Just left click and you're done! The following describes the layers in clipping order.
Multiple color ramps / Transparency
One downside to this method is it's a bit cumbersome when working with multiple color ramps. Unlike in other pixel art tools, You can't just start painting with a different ramp on the same layer, instead you have to create an entirely new layer stack and paint there. This means you need to be conscious of which color should be on top of the other. Additionally working with transparency is a bit of concern. When posterizing Photoshop doesn't threshold the alpha channel. This means that if you're using a soft brush on top of a transparent paint layer (necessary if you want to have multiple color ramps), then the result will be a non-indexed blend. To solve this, the best practice is to define the silhouette of what you're painting with the pixel brush first, then lock the alpha channel of the layer using the alpha-lock feature in the layer palette. This will ensure that when using soft-brushes it will only paint on top of the opaque pixels.
Advanced Layer Setup
This isn't the only way of doing things either! Try adding in additional paint layers if you dont want to commit to a new idea. Try adding adjustment layers in various parts of the stack. Depending on where you place it, it will have vastly different effects. If you place one right after your paint layers you'll be making a "dynamic index adjustment", but if you place one in-between two duplicated posterize layers you'll be making a "fixed index adjustment." Play around and have fun!
I'll leave you with a few examples of images I created with this method on Chasm. Unfortunately most of the work I've done with this method is still not public.
Conclusion:By far the most important thing I need to stress here is that this ISN’T end-all be-all solution for creating pixel art. It is simply a tool like any other. Sometimes I use it and sometimes I choose not to depending on the task at hand. It will not make you a better pixel artist, it will only make you a more efficient one as the pixel-level decisions that great pixel artists make can’t be taught to a computer. With that said, these tools can drastically increase your work efficiency. Tasks that used to take hours now takes only minutes allowing you to express yourself more freely and get to your final composition faster.
I hope this tutorial will be as helpful to you as it has been for me. Let me know your thoughts and feel free to ask me any questions. There's a lot more that I haven't covered on this topic, but the post was getting quite long as it is.