Alpha layering

In a recent post I discussed premultiplied alpha in textures. After that I thought I had my alphas under control, but it turns out I was wrong.

Keeping score

Early in my game development I came up with a slightly interesting way to render text. In the game, text is used only to show the game score and is generally quite large on the screen. The score advances every frame, or 1/60^th of a second. Because that advancement would actually change too quickly for the eye to see I actually truncate the last digit and only render the remaining ones. So if the score advances frame-by-frame as 137 → 138 → 139 → 140 → 141, I would actually render it as 13 → 13 → 13 → 14 → 14. Even doing this the rendered score visibly advances every 10 frames, or every 1/6^th of a second. This is still pretty rapid, and I wanted to visually smooth that quickly changing score when it's rendered on the screen.

50 shades of grey

The idea I settled on was to keep track of the last N instances of the score each time a frame in rendered. In the above example, if N was 5, that would be the list of numbers shown. One frame later the list would change to 13 → 13 → 14 → 14 → 14, and 7 frames later it would become 14 → 14 → 14 → 14 → 15. After keeping track of these last N scores I then actually rendered them all with the oldest score being very transparent and each successive score being more and more opaque. This gives an effect like the one shown in the image to the right. Actually that image is showing the score rendered as it is also being rotated in order to make the separate layers easier to see, in most cases the score is not rotating, so each layer is stacked directly on top of the previous one.

This layered transparency effect worked out well and I've had it in place since quite early in the game development. The way I actually implemented this was to take my desired alpha and figure out how many fractional shades of that I would need such that when I layered them all together I would get my desired alpha. For example, for the score when it is shown during a game it is dimmed so as not to distract from game play and has an intended alpha of 0.25. After the game when showing the final score it is brightened to full opacity, or alpha 1.0. If I have 5 layers then I decided to break the desired alpha into 15 fractions 1 + 2 + 3 + 4 + 5. This is also \( \frac{N \times (N+1)}{2} \) or \( \frac{N^2 + N}{2} \). So, for the dimmed alpha of 0.25 I calculated the fractional alpha as \( \frac{0.25}{15} = 0.0167 \). Then I rendered the oldest score layer with an alpha of 0.0167, the second oldest as \( 0.0167 \times 2 = 0.0333 \), etc, up to the newest score as \( 0.0167 \times 5 = 0.0833 \). When all of these were layered on top of one another it produced the desired 0.25 alpha...or so I thought.

One problem I periodically ran into was that if I adjusted a parameter related to the transparency, like making the text slightly lighter or darker, or make the number of layers more or fewer, then the resulting aggregated color often ended up being not what I expected. Recently, and after improving my understanding of alpha blending as described in my previous post, I finally spent the time to dig into this and understand this properly. The problem was that my simple math was incorrect and does not represent how layered alphas work.

In the case of my text, I'm layering a pure white opaque texture onto some existing background. For the texture, being pure opaque white, each white pixel is (1, 1, 1, 1). Because I want to dim this, as described above, I adjust the alpha 0.0167 to get (1, 1, 1, 0.0167). To figure out how this blends with the background I return to the alpha formula I described in my previous post, which said regarding GL.BlendFunc(BlendingFactorSrc.One, BlendingFactorDest.OneMinusSrcAlpha);: for the source color use it as is; and for the destination color take the alpha component of the source, subtract that from one, and multiply that by each component of the destination.

Diminishing returns

Let's do the math. The source is (1, 1, 1, 0.0167) and to keep things simple the destination is black, or (0, 0, 1, 1). So, use the source component as is means just keep (1, 1, 1, 0.0167). Next, for the destination color, take the alpha component of the source, subtract that from one, and multiply that by each component of the destination means we take the source alpha, 0.0167, subtract that from 1, \( 1 - 0.0167 = 0.9833 \), and multiply that by the destination, which gives us (0, 0, 0, 0.9833). Now, the actual screen doesn't have an alpha component, it just shows colors, so to get a real color we multiply the alpha by each component and then add the results, or: (1, 1, 1, 0.0167) becomes (0.0167, 0.0167, 0.0167) and (0, 0, 0, 0.98333) becomes (0, 0, 0) and adding those two gives us (0.0167, 0.0167, 0.0167) or   X   . That's very close to black, but that's ok, it's just 1 of 15 shades we're applying. Let's do another.

The source is the same, (1, 1, 1, 0.0167), or multiplied out as (0.0167, 0.0167, 0.0167). The destination is different this time, since we layering on top of what we just calculated, which is (0.0167, 0.0167, 0.0167, 0.9833), or multiplied out as (0.0164, 0.0164, 0.0164). Adding these gives us (0.0331, 0.0331, 0.0331) or   X   . Hmm, that still looks awfully black, but it's just two of 15 shades.

Now, for those of you paying attention, this actually isn't the shade we'd have at this point. I'm supposed to take the fractional alpha for the oldest text plus two of that fraction for the next oldest, which means at this point I should have three shades layered on one another, which would be a tad brighter black. But, actually, even that isn't true, and this is where we start getting to the heart of my problem. You do not get to the same color if you apply the alpha layer three times in succession compared to doing it once and then a second layer at twice the alpha. Let me explain.
If I continued the above calculations for one more layer at the same fractional alpha I would end with a final color of (0.0492, 0.0492, 0.0492). On the other hand, if I took my first layer, (0.0167, 0.0167, 0.0167), and layered on top of it a double fractional shade (1, 1, 1, 0.0334), then I would result with (0.0497, 0.0497, 0.0497).

Now, 0.04918 and 0.04973 are not that different, but if I did this for all 15 layers the difference increases. Here's the complete table:

	Shade Applied		Cumulative Total		Calculated Color
Layer	Single Layers	Increasing Shades	Single Layers	Increasing Shades	Single Layers	Increasing Shades
1	0.0167	0.0167	0.0167	0.0167	0.0167	0.0167
2	0.0167	0.0333	0.0333	0.0500	0.0331	0.0494
3	0.0167	0.0500	0.0500	0.1000	0.0492	0.0970
4	0.0167	0.0667	0.0667	0.1667	0.0650	0.1572
5	0.0167	0.0833	0.0833	0.2500	0.0806	0.2274
6	0.0167		0.1000		0.0959
7	0.0167		0.1167		0.1110
8	0.0167		0.1333		0.1258
9	0.0167		0.1500		0.1404
10	0.0167		0.1667		0.1547
11	0.0167		0.1833		0.1688
12	0.0167		0.2000		0.1826
13	0.0167		0.2167		0.1963
14	0.0167		0.2333		0.2097
15	0.0167		0.2500		0.2228

This shows two methods of adding the layers. Single Layers means layering the fractional alpha multiple times. Increasing Shades means doing that, but increasing the fraction for each layer. I've highlighted the interesting numbers. The Cumulative Total colums just show that in both methods the resulting simple-math total is what we're trying to get to, 0.25. The Calculated Color column shows what we actually end up with, which in both cases is less than our target, or 0.2228 in the case of Single Layers or 0.2274 in the case of Increasing Shades.

The reason both numbers don't add up to what we want is that each successive layer causes less of a change than the previous one. This is a case of diminishing returns. Because of this we need to do some math to calculate the fractional shade such that it takes into account these diminishing returns.

	Calculated Color
Layer	Single Layers	Increasing Shades
1	0.0667	0.0667
2	0.1289	0.1911
3	0.1870	0.3529
4	0.2412	0.5255
5	0.2918	0.6836
6	0.3390
7	0.3830
8	0.4242
9	0.4626
10	0.4984
11	0.5318
12	0.5630
13	0.5922
14	0.6194
15	0.6447

Warning! Math ahead!

I'm going to shift our target to 1 instead of 0.25. This will make the diminishing returns problem more clear. Doing this, and following the same flawed layering strategy described above, the Calculated Color from the above table becomes as shown on the right. You can now see the totals are quite far for our target of 1. In fact, because of diminishing returns, no matter how many layers we add we'll never actually get to 100% opaque. If I extended the above to 100 layers the Single Layers color would be 0.999.

So, what I needed to do was to determine a different fractional alpha to use when layering that would add up to my desired target. To do this I converted my Single Layers iterative algorithm into a formula. First, the iterative version of this is below, where fractional is the fraction of my intended alpha, or \( \frac{1}{15} = 0.067 \).

$$ f(n) = (1 - fractional) \times f(n-1) + fractional $$

This calculates the final color for n where n is the number of layers. To convert this to a formula that does not reference itself I started by listing the successive values of this formula in Google Spreadsheets, just like the above tables, and then I played with it until I came up with the following equivalent formula.

$$ color = 1 - (1 - fractional)^{layers} $$

We can check this by plugging in our fractional alpha of \( \frac{1}{15} = 0.067 \) and our 15 layers as \( 1 - (1 - 0.067)^{15} = 0.645 \), which gives us the same value. So far so good, but what we want is a formula into which we supply the color and the number of layers and it gives us the fractional alpha. That means we want to get the fractional part of the formula on one side by itself. Time to remember how do to formula manipulation. First, get the exponent expression on a side by itself.

$$ 1 - color = (1 - fractional)^{layers} $$

Next, reverse the exponential equation.

$$ (1 - color)^{ \frac{1}{layers} } = 1 - fractional $$

Finally, get fractional by itself and positive.

$$ fractional = 1 - (1 - color)^{ \frac{1}{layers} } $$

Great, now we just plug in our target color, which was fully opaque white or 1, and keeping the layers as 15, we get...oops...hmm, \( 1 - 1 = 0 \) and \( 0^{ \frac{1}{15} } = 0 \), and \( 1 - 0 = 1 \), so the fractional is 1. Ack, we have a problem. This actually makes sense. Remember when I said that with the diminishing returns problem we can add the fractional alpha as many times as we want and never actually get to 100% opaque, well this is telling us the same thing. Actually, if the fractional alpha starts out as 1 then we'll get to 100% opaque, but then we don't get the successive alpha blending effect that we want.

Ok, time to cheat, although I want the final alpha to be 1 I'll settle with it being a single shade off of 1. Since there are 256 possible alpha shades I'll settle with \( \frac{255}{256} = 0.996 \). And plugging that into the formula I get \( 1 - (1 - 0.996)^{ \frac{1}{15} } = 0.308 \). If I layer that 15 times I get this succession of colors: 0.308 → 0.521 → 0.669 → 0.771 → 0.841 → 0.890 → 0.924 → 0.947 → 0.964 → 0.975 → 0.983 → 0.988 → 0.992 → 0.994 → 0.996.

This works and I've implemented this in code now, but it's not actually exactly what I want. This is the Single Layers algorithm, where really I want the Increasing Shades algorithm. The formula for that is more complex, though:

$$ f(n) = \left[ \left(1 - \frac{ n^2 + n }{2} \times fractional \right) \times f(n-1) \right] + \frac{ n^2 + n }{2} \times fractional $$

I've tried to convert this into a single formula that does not reference \( f(n-1) \), but have not been successful yet. For now I'll either stick with the previous formula or use the one above and calculate it recursively.

Next: Who stole my pixels?