Premultiplied alpha

Sorry I've been gone

It's been a while since my last post about OpenGL and OpenTK Fun. This is because the challenge I described in that article, and ultimately resolving that issue, unstuck me from making real progress on my development and I've been putting much more effort into my game since then and am now much further along. That's all good news, but now I need to both make progress on the game and try to keep this blog going too.

I'll start today with a discussion about OpenGL premultiplied alpha textures, but before that I want to send a quick thank you out to a new code syntax highlighter by Alex Gorbatchev. It's JavaScript based, so I no longer need to manipulate my code entries before posting them in these articles. Also a thank you to Carter Cole and his blog entry describing how to easily setup the syntax highlighting in Blogger.

So, what is "premultiplied alpha" and why do I care?

As for what it is, there are plenty of pages out there that can describe it much better that me, so I'm not even going to try. I found a good succinct description on Martin Stone's blog, which points to a more detailed description on Tom Forsyth's blog. Please check those pages out if you want to learn more.

As for why I care, that I can describe in more detail. Initially I knew almost nothing about premultiplied alpha. I had seen the term a few times, for example when compiling my app there is some log message mentioning that it is precompiling my PNG textures, but I never tried to understand that more since everything was working fine. The reason I started to care more, and look into it more, are due to a couple things.

First, there was always something bothering me about a portion of my OpenGL code. From the beginning when I got the code working on both iOS and in Windows I found that I had to have a platform specific check for texture blending:

// on iPhone this is needed to apply the color adjustment (on PC this is our normal setting)
if(colorAdjustment.Alpha != 1)
{
    GL.BlendFunc(BlendingFactorSrc.SrcAlpha, BlendingFactorDest.OneMinusSrcAlpha);
}
#if MONOTOUCH
else
{
    GL.BlendFunc(BlendingFactorSrc.One, BlendingFactorDest.OneMinusSrcAlpha);
}
#endif

I put this code into place after trial and error. The #if MONOTOUCH section only compiles and runs on the MonoTouch framework, which means only on iOS. What I didn't understand was, given that OpenGL is supposed to be a consistent interface across platforms, why did I need to have this condition depending on the platform? All other code and image assets related to OpenGL textures and blending was the same between the two platforms, so why was this needed?

Well, the answer goes back to what I mentioned above about where I had previously heard about premultiplied alpha; the iOS compilation log message. What that message means is that my assumption that I'm using the same image assets (PNG images in my case) is not true. Although I have the same PNGs referenced in both the iOS project and Windows project, when the iOS version gets built the PNGs are adjusted to have alpha premultiplied.

So, why does that require the adjustment in GL.BlendFunc? Well, first we need to know what GL.BlendFunc does. The first parameter is how to use the incoming (source) colors when blending them with the existing (destination) pixels. The second parameter is how to adjust those destination pixels. There is ample documentation about this on many sites, so I won't go into all of the parameter options, but I will discuss the two that I was using. The first version, GL.BlendFunc(BlendingFactorSrc.SrcAlpha, BlendingFactorDest.OneMinusSrcAlpha); says two things:

1. For the source color, take the alpha (or transparency) component and multiply it by each other component. For example, if the RGB color is (1, 0.8, 0.5)       then after the multiplication it would become (0.5, 0.4, 0.25)      . This is the same color, but darkened.
2. For the destination color, take the alpha component of the source, subtract that from one, and multiply that by each component of the destination. In this case the alpha is 0.5, so subtracting that from 1 is also 0.5, which would be multiplied by the destination color. For example, if the existing destination RGB color was (0.2, 1, 1)       then after multiplying it would become (0.1, 0.5, 0.5)      . Again, the same color, but darkened.

After completing the above calculations on the source and destination colors then are blended by adding them together. That is (0.5, 0.4, 0.25) + (0.1, 0.5, 0.5) = (0.6, 0.9, 0.75)      . Looking at the two original colors you can see that this works and the resulting blended color is correct.

Ok then, what's up with the second version: GL.BlendFunc(BlendingFactorSrc.One, BlendingFactorDest.OneMinusSrcAlpha);? How does this change things? I won't go through all the steps in detail, but starting with the same two first colors you would get a final function of (1, 0.8, 0.5) + (0.1, 0.5, 0.5) = (1.1, 1.3, 1.0) or pure white, since each component exceeds the maximum value of one and is therefore limited to one. Clearly that is too bright and doesn't work work. So, why would you want to do that? Well, that's where premultiplied alpha comes in. The first version used BlendingFactorSrc.SrcAlpha, which multiplies the alpha by the color. And what do you think premultiplied alpha does? It did that exact same calculation when the texture asset was created (or built, in this case). This means that we don't need to do it again now while blending. Instead we use the color as is, which is what BlendingFactorSrc.One does.

So, final question, why do this premultiplied alpha in the first place? I'll quote Tom Forsyth from his blog post (referenced above) for a pretty good explanation. For a much more in depth discussion please read his whole post.

"Normal" alpha-blending munges together two physically separate effects - the amount of light this layer of rendering lets through from behind, and the amount of light this layer adds to the image. Instead, it keeps the two related - you can't have a surface that adds a bunch of light to a scene without it also masking off what is behind it. Physically, this makes no sense - just because you add a bunch of photons into a scene, doesn't mean all the other photons are blocked. Premultiplied alpha fixes this by keeping the two concepts separate - the blocking is done by the alpha channel, the addition of light is done by the colour channel. This is not just a neat concept, it's really useful in practice.

Doing premultiplied alpha on Windows

Ok, so now I understand what's going on, but how do I fix it for Windows since I don't have a build step modifying my PNG files? I did some research and it seems there are some tools that will convert a normal PNG to a premultiplied alpha PNG, specifically it seems GIMP will do this, although I didn't try it myself. I didn't really want to have to convert my existing assets each time I modified them, or complicate my Windows build process by using addition tools, so I made a code change to do the alpha multiplication at the time I load my PNGs. That's a somewhat wasteful operation, but it only happens once, and considering I don't have very many textures I felt it was a good and simple solution.

Below is the code that does this. You'll notice some strange BGRA to RGBA conversion as well. This is due to using the Bitmap class and it's how Windows works.

public static Texture LoadTexture(string fileName, Func<byte[], int, int, Texture> textureCreator)
{
    if(textureCreator == null)
    {
        throw new ArgumentNullException("textureCreator");
    }

    using(Bitmap image = new Bitmap(fileName))
    {
        BitmapData bitmapData = image.LockBits(
            new Rectangle(0, 0, image.Width, image.Height),
            ImageLockMode.ReadOnly,
            PixelFormat.Format32bppArgb
        );
        image.UnlockBits(bitmapData);
        IntPtr rawData = bitmapData.Scan0;

        Texture texture;
        unsafe
        {
            int length = image.Width * image.Height * 4;
            byte[] data = new byte[length];
            Marshal.Copy(rawData, data, 0, length);

            for(int i = 0; i < length; i += 4)
            {
                float alpha = (float)data[i + 3] / 255;
                byte r = data[i + 2];
                byte g = data[i + 1];
                byte b = data[i + 0];
                data[i + 0] = MultiplyByAlpha(alpha, r);
                data[i + 1] = MultiplyByAlpha(alpha, g);
                data[i + 2] = MultiplyByAlpha(alpha, b);
                data[i + 3] = (byte)(alpha * 255);
            }

            texture = textureCreator(data, image.Width, image.Height);
        }

        return texture;
    }
}

private static byte MultiplyByAlpha(float alpha, byte channel)
{
    return (byte)(channel * alpha);
}

After putting this code in place, so that my textures turn into premultiplied alpha data as I load them, I was able to simplify my original platform dependent code into just this:

GL.BlendFunc(colorAdjustment.Alpha != 1 ? BlendingFactorSrc.SrcAlpha : BlendingFactorSrc.One, BlendingFactorDest.OneMinusSrcAlpha);

Removing that platform specific code is a very minor improvement, but more than that I'm happy to understand all of this quite a bit more.

Next: Simple Solutions.