Phunky Physics

In my previous post I discussed some issues I ran into running my .NET code in the MonoTouch environment. In this post I'm switching gears a bit and will discuss an issue I've been struggling with for a while related physics simulation in my game.

Before Diagonal

After Diagonal

90° in the Shade

The issue came up because I was trying to make some of my physics interactions behave different from how they do in reality. You would think that bending reality or completely changing how physics works should be possible in a computer simulation's implementation, but I found out that changing how things work quickly leads to unintended problems. I wonder if this says something about the necessity of how physics actually works in real life.

If you look at the "Before Diagonal" image on the right, you see a white ball heading towards a collision with a blue diagonal. Then, in "After Diagonal", you can see the white ball after it has bounced off of the blue diagonal. For the purposes of my game the white ball must always be exactly halfway between a row or column of purple stars. Before the bounce it is travelling horizontally, and is halfway vertically in a row. After it bounces it is travelling vertically, and is halfway horizontally in a column. The problem is that in an actual 90° bounce like this, with a ball that has a non-zero radius, it would not bounce from halfway-vertical-in-row to halfway-horizontal-in-column like this. The lower part of leading edge of the ball would hit the diagonal first, before the halfway point, and then the ball would bounce upwards, but to the left of the halfway-horizontal-in-column point.

I initially tried to resolve this problem by simply teleporting the ball from the one halfway point to the next halfway point when the bounce occurred. This looked fine, and seemed to work ok, until I added interactions between multiple balls. The problem was that if there was already another ball in the place to where the first ball is teleported, then they would become overlapped and not properly bounce off of one another.

I solved this problem by not doing the teleportation and having the ball bounce as if it has a zero radius. This works fine for my purposes, but has another small problem. For the skin shown in the images to the right, the ball is fuzzy and the diagonal is lightning-like, so having the ball overlap the diagonal when it bounces looks fine. For an alternative skin, though, where the ball is completely solid and the diagonal is more physical-looking, it will not look ok. My plan to solve this is to have the interaction calculation done as if the radius is zero, but have the rendering adjust the ball's visible position so it doesn't overlap the diagonal.

Before Collision

After Collision

Collision!

Billiards this is Not

After resolving the above issue I then started working on ball-to-ball collisions. When the two balls are on the same row or column this is easy. The issue is when they are moving perpendicular to one another.

If you look an the "Before Collision" image to the right you can see that the tow balls are heading towards one another and will collide in the lower-left area of the image. If these were actual balls, like billiard balls, then the result of the collision is that the one moving down would head left and the one moving left would head down. The angle each ball changes to would depend on their speed and mass. My problem with implementing this type of collision is the same as that I described above related to bouncing off of diagonals. Basically, I want the balls to remain exactly in the halfway point between a row or a column. In this scenario, though, that would not happen.

For ball-to-ball collisions I could not use the same strategy I used above for diagonal collisions; calculating as if the ball had a zero radius. I actually initially tried this, and it worked ok for two balls, but once more than two were involved it became possible that all of the balls come to be in exactly the same position and could never resolve their collisions. Also, adjusting the rendering of the overlapped balls so that the don't appear to be overlapping is more difficult.

The solution I eventually came up with was to have the balls simply bounce and reverse directions entirely (see "After Collision" image). As I said, in real physics, that would not happen, and I wanted to make it look reasonable. To attempt do that I added a graphic element that is shown when the collision occurs, and implies that something more than just a normal real-life, ball-to-ball collision is happening. Currently I'm using the graphic shown in the "Collision!" image, but this is tentative, and also can be customized for each skin. Even though the physics of this collision isn't even close to real physics, it actually works pretty well in the game.

Summary

Resolving both of the problems described above took quite a while. I experimented with a number of different solutions before settling on the current one. I also tried to get things working in the early stages when I had the balls teleporting during a diagonal bounce. This seemed we work for some cases, but was never reliable. I find it interesting that even the small adjustment to physical rules I was trying to make in having the balls bounce and maintain their position halfway in a row or column caused problems that were basically impossible to solve. Of course the other side if this is that even my final solution goes against how physics works in real life.

One other thing that makes me feel pretty comfortable with my current solution is that it ended up in much cleaner code. In other strategies I always had a number of special cases, that made the code messy. Once I changed to my current solution they went away.

Next time: More Optimizations.

More MonoTouch Gotchas

In my previous post I took a break about discussing my adventure developing for the iPhone with .NET and discussed some issues I've had blogging with Blogger. In this post I get back to the main of subject and describe an odd issue I've encountered with MonoTouch.

The issue is due to the way that MonoTouch compiles to native iPhone binaries. It does not support JIT compilation, which a "normal" .NET environment does support. In most cases this is fine, but sometimes the compiler cannot fully detect the types that are needed and therefore doesn't compile them into the binary. Then, during runtime, when it detects it needs a type that was not compiled it crashes.

The code below is a pretty simple example of this issue:

/// <summary>Initializes a new instance of the <see cref="Shape"/> class.</summary>
/// <param name="points">The points.</param>
public Shape(Vector2d[] points)
     : this(points.Select(point => new Vector2((float)point.X, (float)point.Y)).ToArray())
{
}

This is a very simple alternate constructor for my Shape class. It is a convenience constructor to allow passing in Vector2d values (these are double based vectors) instead of Vector2 values (these are float based). The LINQ expression just enumerates over the array and converts the values.

So, why doesn't this work? Well, the error I get is something like:

Attempting to JIT compile method
'System.Linq.Enumerable/PredicateOf`1<OpenTK.Vector2d>:.cctor
()' while running with --aot-only.

I believe what this is saying is that the constructor for System.Linq.Enumerable/PredicateOf`1<OpenTK.Vector2d> is not found, and since JIT isn't supported it can't compile it at runtime. The reason it isn't compiled initially is because the compiler doesn't recognize that the LINQ expression will ultimately need this constructor. The way around this is it reference the needed type explicitly in code. Unfortunately, I didn't figure out a way to do that. Fortunately, I found a different workaround, although it does include a little bit of unnecessary overhead:

/// <summary>Initializes a new instance of the <see cref="Shape"/> class.</summary>

/// <param name="points">The points.</param>

public Shape(Vector2d[] points)

     : this(new List<Vector2d>(points).Select(point => new Vector2((float)point.X, (float)point.Y)).ToArray())

{

}

In this case, the LINQ expression is over the List<Vector2d>, which works fine. I believe the reason this works is because List<Vector2d> explicitly implements System.Linq.Enumerable/PredicateOf`1<OpenTK.Vector2d>, where the array does not.

I need to play around with this a bit more to see how I can fix it without the List wrapper. The problem is this only shows up when I deploy to the iPhone, so the build/run/test cycle is a bit slower. For now my workaround works ok, so improving this is not a high priority.

Next time: Phunky Physics.

Fighting with Blogger

In my previous post I described some work I spent optimizing my .NET code to make it perform well on an iPhone. I expect I'll have a follow up post when I do more optimizations in the future, but next I want to take a slight detour from coding to discuss a small battle I had with getting Blogger to work well with the type of info I'm posting.

Before getting into that discussion I want briefly give an update on the game. These blog posts are lagging actual development by a couple months. Basically it takes me a while to finish a blog entry, so I try to queue them up and revise them when I have time. So, in reality the game is coming along pretty well. It is still slow going because I can only put in a few hours here and there, but it is progressing nicely.

Now, on to fighting with Blogger...

So, as mentioned above, I wanted to be able to post source code that looks good and has syntax highlighting. Initially I tried a simple solution that I know worked to some degrees from my previous experience, simply copy the code from Visual Studio (VS) and paste into Blogger. Oops, no, that doesn't quite work. If you do that no formatting is included. But, if you copy and paste into Word then it is included. Hmm, ok, how about copying and pasting into Word, then re-copying from Word and pasting into Blogger? Yep, that works, and gives output like below:

/// <summary>
/// Binds the texture.
/// </summary>
/// <param name="textureData">The texture data.</param>
/// <param name="width">The width.</param>
/// <param name="height">The height.</param>
/// <returns>The texture's bound id.</returns>
private static uint BindTexture(byte[] textureData, int width, int height)
{
     uint textureId = 0;
     GL.GenTextures(1, ref textureId);
     GL.BindTexture(All.Texture2D, textureId);
     All format = (System.Environment.OSVersion.Platform == PlatformID.Win32NT) ? All.Bgra : All.Rgba; // useless comment placed here just to make the line very long
     GL.TexImage2D(All.Texture2D, 0, (int)format, width, height, 0, format, All.UnsignedByte, textureData);
     textureData = null;
     GL.TexParameter(All.Texture2D, All.TextureMinFilter, (float)All.Linear);
     GL.TexParameter(All.Texture2D, All.TextureMagFilter, (float)All.Linear);
     GL.Enable(All.Texture2D);
     return textureId;
}

Actually, the above is wrapped in a pre, which I have styled to make the grey background and such. But, there's still a problem, the long lines wrap and make it harder to read. So, what I really want is for the code area to scroll horizontally when the lines are too long. Fortunately, a bit of CSS can do that:

.csharpcode
{
  font-family: Consolas, "Courier New", Courier, Monospace;
  margin: 0em;
  padding: 0.5em;
  border: solid 0.1em #000000;
  background: #222222;
  white-space: pre;
  white-space: nowrap;
  overflow: auto;
  overflow-y: hidden;
}

So, if I change the simple pre to pre class="csharpcode" then I get the following instead. Notice the horizontal scrollbar and the long lines no longer wrap.

/// <summary>
/// Binds the texture.
/// </summary>
/// <param name="textureData">The texture data.</param>
/// <param name="width">The width.</param>
/// <param name="height">The height.</param>
/// <returns>The texture's bound id.</returns>
private static uint BindTexture(byte[] textureData, int width, int height)
{
     uint textureId = 0;
     GL.GenTextures(1, ref textureId);
     GL.BindTexture(All.Texture2D, textureId);
     All format = (System.Environment.OSVersion.Platform == PlatformID.Win32NT) ? All.Bgra : All.Rgba; // useless comment placed here just to make the line very long
     GL.TexImage2D(All.Texture2D, 0, (int)format, width, height, 0, format, All.UnsignedByte, textureData);
     textureData = null;
     GL.TexParameter(All.Texture2D, All.TextureMinFilter, (float)All.Linear);
     GL.TexParameter(All.Texture2D, All.TextureMagFilter, (float)All.Linear);
     GL.Enable(All.Texture2D);
     return textureId;
}

The only remaining problem is that this HTML that comes from VS via Word is very verbose. Just the very first line, a comment, contains all of this HTML:

<div class="MsoNormal" style="line-height: normal; margin: 0in 0in 0pt; mso-layout-grid-align: none;"><span style="color: grey; font-family: Consolas; font-size: 12pt;">///</span><span style="color: green; font-family: Consolas; font-size: 12pt;"> </span><span style="color: grey; font-family: Consolas; font-size: 12pt;">&lt;summary&gt;</span><span style="font-family: Consolas; font-size: 12pt;"></span></div>

So, the entire code block ends up being quite a lot of HTML.

To address this issue I actually jumped through a number of other hoops to create less verbose HTML. In fact, the earlier pages in this blog use that simpler HTML. But, going forward, I've decided to use the above method. It is easier and creates more nicely formatted output. It is a bit more verbose, but then if you look at all of the other CSS that Blogger includes the pages are pretty heavy to start with.

I actually might go back and reformat the previous pages at some point, so the above statement might become inaccurate.

Anyway, I guess my fight with Blogger is a lot less interesting that I thought. As I was figuring out how to do all of the above it was annoyingly painful. But not that I have a solution it is pretty simple.

Next time: More MonoTouch Gotchas.

Optimizing .NET on the iPhone

Early Splash Screen

After working exclusively on my PC for a while adding functionality I decided to deploy to my iPhone to make sure everything still works well. When I did that I was happy to see that it did run correctly, but not so happy to see pretty poor performance.

In the game's splash screen I was only getting about 10 FPS, even though I'm shooting for 30. The splash screen is ironically more computationally intensive than a real game screen because of the grid resolution. For a real game screen the board is around 10x15 squares, depending on the level. For the splash screen, currently, it is 20x30 squares. That a difference from 176 Poles on the game screen to 651 on the splash screen. Clearly I needed to either optimize a lot or reduce the complexity of the splash screen. On the other hand, if I could get the splash screen performing well, then that would be a good baseline such that if it worked well on a particular device then the real game screens should be just fine.

Because of my platform independent abstraction efforts I described in an earlier post, I have access to the sophisticated profiling tools in Visual Studio (VS) and can quickly find the areas of code that are taking the most time. I realize that Xcode on Mac includes the Instruments tool, but so far I haven't been able to get it to work well with my MonoTouch generated code so that no symbol names are displayed. Only function addresses are shown, so it is nearly useless for this type of investigation. I've reached out to the MonoTouch community and hopefully there is a way to get symbols displayed, but for now I'll rely on VS.

Too Many Objects

One area I suspected might become a problem even when I was first creating it was my Triangle object. It contained 3 Vector2 objects, one for each vertex of the triangle. It also has a number of manipulation methods (rotate, scale, etc) which calculate new vertices and replace the old ones with the new ones.

The problem is that the game does these triangle manipulations for almost every single graphic element for every frame. In the case of the splash screen thats 651 x 3 x 30 = 58590 new Vertex2 objects created every second, and that's just for the poles on the screen, there are many other elements as well. Although the .NET memory management system is quite efficient, this is still a lot of overhead that is completely unnecessary if I change how my Triangle works. Which is exactly what I did...

Instead of using Vertex2 objects for my triangle's points, I changed them to 6 float values. The 6 values are the X and Y values for each of the 3 vertices. I used float because that is what I later pass to OpenGL and want to avoid the overhead of casting.

Unfortunately, the result of this rewrite was that there was not measurable performance gain in either CPU usage or memory allocation. How can that be? Well, it turns out that Vector2 is a struct, not an object. This means that it is not allocated and garbage collected in the same way and creating new instances of it is quite light weight. Although I should have checked that Vertex2 is a class before embarking on this optimization it didn't take that much time and I learned a bit.

Array Bounds Checking

OpenGL requires an array of float values in GL.VertexPointer and GL.TexCoordPointer. To populate this array I go through each of my graphic elements and add its vertices to the end of the array. In .NET there is automatic bounds checking performed on arrays to ensure you do not access memory outside of the array. This bounds checking is great for safety, but there is a performance penalty associated with it.

To see if it was significant for me, I just wrapped the area of code that populates the array with an unsafe { ... } block. After doing this that function went from about 26% total CPU usage down to about 23%. It's only a 3% improvement, but it's a very simple one. Since I carefully calculate my array index in this routine I'm confident the lack of bounds checking will be ok in this case.

Too Many Function Calls

Initially when I populated the float arrays for OpenGL I was calling a property for each vertex in my triangle one at a time. I changed this so that I pass in an array and starting index into the triangle and it populates it. This reduced the number of property accesses by 6 times.

Unfortunately I didn't profile this change carefully so I can't say how much of an affect it had, but I imagine it would have been measurable reducing more than 60,000 property calls to closer to 10,000 method calls.

Too Many Calculations

My triangle object remembers its original state (position, rotation, scale, etc) so that I can manipulate it based on that state. A side affect of this is that when I want to get the actual vertices I need to calculate them based on its original state and the current transformations. Initially I was doing this every time I needed the vertices, even if its state or transformations hadn't changed.

To fix this I added a boolean flag, isDirty that I use to keep track of it state. If its state changes then I set the flag to true and then next time I need the vertices I calculate them only if that flag is set to true and then set it to false.

This optimization reduced these calculations from about 29% of CPU time to 11%. Now, finally, a nice gain.

Too Many Draws

After all of the above investigations and optimizations, it turned out the biggest improvement I made was from a stupid mistake.

I refactored the class that acts as the main view manager in order to better support different views; a game view and a splash screen view in this case. In doing that refactor I reduced the two-phase update-then-draw functionality into one-phase update-and-draw. In my case there's no real benefit to either case, and the single phase seemed a bit simpler. When I did this, however, the main render loop, which is one of the platform-dependent classes, still called both phases. As part of the refactor the update method became renamed to the draw one. The consequence of this was that on the iPhone version of the render loop, the draw method was called twice, once when it used to be for update and then again for the draw. Since I did the refactor on my PC I tested it there and must have fixed this problem for the PC version of the render loop. When I went to test the iPhone version, however, it was still doing this double work.

Anyway, when I found the problem and removed the redundancy there was a significant performance improvement. That, along with my other optimizations, means the game is back to running my targeted 30 FPS, even in the more complex splash screen. When it's doing that it's eating about 70% CPU, and I'd really like to bring that down for battery life and to better support older devices. I do have one more significant idea based on data from the VS profiler, but I haven't gotten to it. If it works well I'm hoping for about a 10% gain in efficiency. If and when I do another pass as optimizations I'll write about them here.

Next time: Fighting with Blogger.

Problems Deploying to iPhone

Sorry for the brief hiatus, I was on a business trip to Beijing for 1 week, but I'm back now and hope to get back into the swing of developing.

In my last post I discussed how I got OpenAL working on Windows. After that I wanted to confirm that all my platform independent abstractions worked correctly on the iPhone but going back to my Mac and building and deploying there. Unfortunately, when I did that, I ran into a few issues. This post discusses that experience.

The first error I received was mtouch failed with no output (1). After a bit of searching on the web I found a few hints about not allowing spaces in the output assembly name. I changed some settings to correct this, but the problem persisted.

I looked at the command being from by MonoTouch and noticed other areas in the command were not escaping or quoting some paths. Initially I was accessing my project over a Samba share directly to my Windows Documents directory. This meant that I was opening the project on my Mac from Documents\visual studio 2010\Projects\Zoing. Notice the spaces in "visual studio 2010".

To fix this I changed how I mounted the Samba share to directly mount the Zoing directory, thus skipping the areas with spaces. After doing this I could deploy to my iPhone. Joy!

Well, no, actually not. As soon as I ran the program I got this:

Tue Sep 28 23:46:25 unknown kernel[0] <Debug>: launchd[9744] Builtin profile: container (sandbox) Tue Sep 28 23:46:25 unknown kernel[0] <Debug>: launchd[9744] Container: /private/var/mobile/Applications/478863D8-A953-4D5A-84FD-AF8C096DC363 [69] (sandbox) Tue Sep 28 23:46:27 unknown UIKitApplication:launcher[0x114c][9744] <Notice>: Unhandled Exception: System.TypeInitializationException: An exception was thrown by the type initializer for System.Collections.Generic.EqualityComparer`1 ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ExecutionEngineException: Attempting to JIT compile method 'System.Collections.Generic.GenericEqualityComparer`1<OpenTK.Vector2>:.ctor ()' while running with --aot-only.
Tue Sep 28 23:46:27 unknown UIKitApplication:launcher[0x114c][9744] <Notice>:   at System.Reflection.MonoCMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 
Tue Sep 28 23:46:27 unknown UIKitApplication:launcher[0x114c][9744] <Notice>:   --- End of inner exception stack trace ---

The highlighted areas point to the important part. Basically, for some reason, MonoTouch, on the actual iPhone (it works fine in the simulator) does not support default EqualityComparers properly. Fortunately, I can pass in a IEqualityComparer<T> to the Dictionary constructor, like so:

_diagonals = new Dictionary<Vector2, Diagonal>(VectorComparer.Singleton);

And, my VectorComparer implementation is:

private class VectorComparer : IEqualityComparer<Vector2>
{
    private static VectorComparer _singleton = new VectorComparer();
    public static VectorComparer Singleton
    {
        get
        {
            return _singleton;
        }
    }
    public bool Equals(Vector2 x, Vector2 y)
    {
        return x.Equals(y);
    }
    public int GetHashCode(Vector2 obj)
    {
        return obj.GetHashCode();
    }
}

Currently, for my purposes, this implementation is fine because I know I'm only putting in Vector2 objects with values that are precise as a float. If I expect to want to compare Vector2 objects where, because of math rounding, the values may not be so precise I could adjust my comparisons to have some level of necessary precision to consider the values equal.

Next time: Optimizing .NET on the iPhone.

OpenAL on Windows

In my previous post I discussed the graphics related word  I had to do to get my iPhone targeted game running on my PC. This was relatively easy to do and has improved my productivity significantly. Of course, my game contains sound as well, and it is important to be able to develop those aspects of it on my PC as well. This post discusses that.

From the beginning I decided to use OpenAL for sound in my game. The main reason is it provides relatively simple access to 3D positioned audio, which is something I wanted (actually, I really just am working 2D space). Getting this working on the iPhone initially in native Objective-C was pretty straight forward, but, as with OpenGL, OpenAL is not natively available on Windows. The OpenTK library that MonoTouch relies on has built in support for OpenAL, but it still needs an underlying library.

I searched around a bit and quickly found Creative Labs' OpenAL library for Windows. This installed easily, but I still needed to implement the layer to reads WAV files and converts them into a format that OpenAL understands.

In the iPhone version I do this to initialize OpenAL:

/// <summary>
/// As per the SDK:
/// <br/>
/// Your application must call this function before making any other Audio Session Services calls.
/// You may activate and deactivate your audio session as needed (see AudioSessionSetActive),
/// but should initialize it only once.
/// </summary>
public SoundManager()
{
    // setup our audio session
    AudioSession.Initialize();
    AudioSession.Category = AudioSessionCategory.AmbientSound;
    AudioSession.SetActive(true);
    AudioSession.Interrupted += HandleAudioSessionInterrupted;
    AudioSession.Resumed += HandleAudioSessionResumed;
    // TODO: Check if BGM is playing
    // UInt32 size = sizeof(iPodIsPlaying);
    // result = AudioSessionGetProperty(kAudioSessionProperty_OtherAudioIsPlaying, &size, &iPodIsPlaying);
    // if the iPod is playing, use the ambient category to mix with it
    // otherwise, use solo ambient to get the hardware for playing the app background track
    // UInt32 category = (iPodIsPlaying) ? kAudioSessionCategory_AmbientSound : kAudioSessionCategory_SoloAmbientSound;
    OpenALManager = new OpenALManager();
}

And then this to load CAF sound samples:

private static AudioFile GetAudioFile(string filename)
{
    if(!File.Exists(filename)) {
        throw new FileNotFoundException("Could not find sound file.", filename);
    }
    AudioFileType fileType;
    switch(Path.GetExtension(filename).ToUpper()) {
        case ".CAF":
            fileType = AudioFileType.CAF;
            break;
        default:
            throw new NotSupportedException("Audio files of type " + Path.GetExtension(filename) + " are not supported.");
    }
    using(CFUrl cfUrl = CFUrl.FromFile(filename)) {
        return AudioFile.Open(cfUrl, AudioFilePermission.Read, fileType);
    }
}
private static AudioData GetAudioData(AudioFile audioFile)
{
    // Set the client format to 16 bit signed integer (native-endian) data
    // Maintain the channel count and sample rate of the original source format
    AudioStreamBasicDescription outputFormat = new AudioStreamBasicDescription();
    outputFormat.SampleRate = audioFile.StreamBasicDescription.SampleRate;
    outputFormat.ChannelsPerFrame = audioFile.StreamBasicDescription.ChannelsPerFrame;
    outputFormat.Format = AudioFormatType.LinearPCM;
    outputFormat.BytesPerPacket = 2 * audioFile.StreamBasicDescription.ChannelsPerFrame;
    outputFormat.FramesPerPacket = 1;
    outputFormat.BytesPerFrame = 2 * audioFile.StreamBasicDescription.ChannelsPerFrame;
    outputFormat.BitsPerChannel = 16;
    outputFormat.FormatFlags = AudioFormatFlags.IsPacked | AudioFormatFlags.IsSignedInteger;
    // Set the desired client (output) data format
    bool hadError = audioFile.SetProperty(AudioFileProperty.DataFormat, Marshal.SizeOf(outputFormat), GCHandle.ToIntPtr(GCHandle.Alloc(outputFormat)));
    if(hadError) {
        throw new InvalidOperationException("Could not set output format.");
    }
    byte[] data = new byte[audioFile.Length];
    audioFile.Read(0, data, 0, (int)audioFile.Length, false);
    AudioData audioData = new AudioData(data, outputFormat.ChannelsPerFrame == 1 ? ALFormat.Mono16 : ALFormat.Stereo16, outputFormat.SampleRate);
    return audioData;
}

For the PC I had to do something similar, except for WAV files, not CAF:

using System;
using System.IO;
using OpenTK.Audio;
using OpenTK.Audio.OpenAL;
using Qythyx.OpenTKTools.Sound;
namespace Qythyx.Launcher_PC
{
    internal sealed class WaveReader
    {
        private static ALFormat GetALFormat(WaveData data)
        {
            switch(data.Channels)
            {
                case 1:
                if(data.BitsPerSample == 8)
                {
                return ALFormat.Mono8;
                }
                else if(data.BitsPerSample == 16)
                {
                return ALFormat.Mono16;
                }
                break;
                case 2:
                if(data.BitsPerSample == 8)
                {
                return ALFormat.Stereo8;
                }
                else if(data.BitsPerSample == 16)
                {
                return ALFormat.Stereo16;
                }
                break;
            }
            throw new AudioException("Unsupported audio format. Channels = " + data.Channels + ", Bits per Sample = " + data.BitsPerSample);
        }
        /// <summary>Reads and decodes the sound file.</summary>
        /// <param name="filename">The WAVE filename.</param>
        /// <returns>An <see cref="AudioData"/> object that contains the decoded data.</returns>
        public static AudioData ReadWave(string filename)
        {
            WaveData data = ReadWaveData(filename);
            return new AudioData(data.Data, GetALFormat(data), data.SampleRate);
        }
        private struct WaveData
        {
            public int RiffChunckSize;
            public int FormatChunkSize;
            public short AudioFormat;
            public short Channels;
            public int SampleRate;
            public int ByteRate;
            public short BlockAlign;
            public short BitsPerSample;
            public int DataChunkSize;
            public byte[] Data;
        }
        // Read the WAVE/RIFF headers.
        private static WaveData ReadWaveData(string filename)
        {
            using(Stream stream = new FileStream(filename, FileMode.Open, FileAccess.Read))
            {
                using(BinaryReader reader = new BinaryReader(stream))
                {
                WaveData data = new WaveData();
                // RIFF header
                if(new string(reader.ReadChars(4))!= "RIFF")
                {
                throw new FormatException("File is not recognized as valid WAVE format. Can't find RIFF signature.");
                }
                data.RiffChunckSize = reader.ReadInt32();
                if(new string(reader.ReadChars(4)) != "WAVE")
                {
                throw new FormatException("File is not recognized as valid WAVE format. Can't find expected WAVE format.");
                }
                // WAVE header
                if(new string(reader.ReadChars(4)) != "fmt ")
                {
                throw new FormatException("File is not recognized as valid WAVE format. Can't find 'fmt' marker.");
                }
                data.FormatChunkSize = reader.ReadInt32();
                data.AudioFormat = reader.ReadInt16();
                data.Channels = reader.ReadInt16();
                data.SampleRate = reader.ReadInt32();
                data.ByteRate = reader.ReadInt32();
                data.BlockAlign = reader.ReadInt16();
                data.BitsPerSample = reader.ReadInt16();
                while(reader.PeekChar() == 0)
                {
                reader.Read();
                }
                if(new string(reader.ReadChars(4)) != "data")
                {
                throw new FormatException("File is not recognized as valid WAVE format. Can't find data marker.");
                }
                data.DataChunkSize = reader.ReadInt32();
                data.Data = reader.ReadBytes((int)stream.Length);
                return data;
                }
            }
        }
    }
}

Well, after this OpenAL was happy on my PC and I've now got sound and graphics and I can develop happily in Visual Studio. Woohoo!

Next time: Problems deploying to iPhone.

OpenGL ES 1.1 on Windows

Wrong Colors

Correct Colors

In my previous post I discussed how I separated my .NET solution into multiple projects with the goal of having the bulk of my code 100% platform independent. The initial goal of this was to allow me to easily develop with in Visual Studio (VS), but a second and important benefit is it sets me up well to port my game to other .NET platforms like Android and Windows Mobile 7. I also mentioned that the graphics framework I'm targeting is OpenGL ES version 1.1. Although iPhone 4 supports OpenGL ES 2.0, older iPhones do not and I want to support them as well.

To get the full benefits of developing in VS I not only want to be able to compile, but also run my game. To do that I need to be able to render using Open GL ES 1.1. The problem is that this is a version of OpenGL that is for mobile devices. Fortunately, there are a few emulators available for the PC.

After some web searching I found an emulator by from Mali, but when I tried to use it I kept getting this error: [Error] Failed to create EGL window surface, error 12293. I did a lot of searching and investigation to fix it, but was never able to. I did see one person describe that they were successful, but their suggestions weren't working for me, so I eventually gave up and tried to find another emulator.

Next I found this one from Khronos. It still had a few minor issues, but when I followed the steps described in this thread I got it to work. The thread said that they also managed to get the Mali emulator working, and I don't know why it didn't work for me.

One interesting thing I found with this emulator was that it isn't 100% compatible with the OpenGL ES 1.1 that runs on the iPhone. Specifically the internalFormat argument passed to GL.TexImage2D seems different. In my original iPhone version I passed All.Rgba for this, but when I ran this on the PC my textures' colors were strange. After a lot of investigation I finally discovered that changing this to All.Bgra fixes the problem. I currently have a hack to set this depending on the OS.

bool isWindows = System.Environment.OSVersion.Platform == PlatformID.Win32NT;
All format = isWindows ? All.Bgra : All.Rgba;
GL.TexImage2D(All.Texture2D, 0, (int)format, width, height, 0, format, All.UnsignedByte, textureData);

I'm not enough of a OpenGL expert to know which is actually correct. It will also be interesting to see how other platforms like Android and Windows Mobile 7 handle this. For now I'm just pretty happy that this was the only platform-dependent adjustment I've had to make.

Next time: OpenAL on Windows.

Divide and Conquer

In my previous post I discussed how I found a way to edit my iOS MonoDevelop (MD) projects in Visual Studio (VS) on my PC, but that I could not compile there and wanted to fix that. In this entry to discuss my solution.

At it's heart the problem is that by default iOS projects reference the MonoTouch (MT) library that wraps the standard iOS libraries (Cocoa, Core*, etc). These projects also have some special configuration options for iOS devices that VS doesn't like. Furthermore, if you are creating a typical iOS application then you probably need to interact with the MT libraries a lot. On the other hand, if you can divide your project into we separated layers, then all but the device specific functionality should be able to exist in standard .NET code and be fully compilable in any platform. That is exactly what I did.

Initially I had a single .NET project with 3 main components:

1. OpenGL setup - A modified version of the default OpenGL related classes provided when you create a new MT iOS OpenGL solution. The modifications are relatively minor, although they do include also setting up OpenAL.
2. Graphics and Sound utilities - Classes I'd written to provide some more object oriented constructs for base graphics and sound functionality. For example, I created a simple Texture class that was simply a set of coordinates that defined a triangle in the OpenGL world along with its matching triangular coordinates of the texture to show there. Another example is an AudioSample class that encapsulates an OpenAL audio sample that can be played, stopped, etc.
3. Game logic - The actual game related components of my game.

So, what I did was to separate these 3 components into separate projects with only the first one referring to the MT library. In addition I created a new version of the first component that was compatible with the PC. If you're paying attention you might wonder about how I could create a PC project that relies on OpenGL ES (ES is the version of OpenGL that is for mobile devices) and on OpenAL, neither of which are standard functionality in Windows. I'll go into some details of this in future posts, for now I'll continue the discussion of the project reorganization.

I should mention that this project separation includes the unfortunate requirement of having parallel project (.csproj) files, one for MD and one for VS. This means that anytime I add a new file to my VS project I need to eventually add it to the MD one as well. Fortunately these new files quickly show up as missing when I compile, so it hasn't been too much of a burden yet. I do hope that MT supports VS better in the future, though.

On the coding side, one of the challenges I faced was that I didn't want to have anything in my first 2 projects that was specific to the game I'm writing. All of that should be only in the 3rd project. At the same time, the loading of data into my Texture and AudioSample classes is platform specific. For example, on the iPhone it is recommended that you have you audio files in CAF format. On the PC WAV is more typical. Also, UI interaction is platform dependant. Specifically the iPhone provides multi-touch events and the PC mouse events. Fortunately, it turned out to be relatively easy to handle all of this.

For the Texture data loading I simply pass a Func<string, Texture> to my main game class' constructor. This takes a string (the filename of the texture data) and returns a fully instantiated, platform independent Texture object.

Similarly, for the AudioSample I pass a Func<string, AudioData> that does that same but returns a platform independent AudioData object.

Finally, for the UI interaction I just have few methods on my main game object that are called for touches/mouse event. At this point I don't have any interaction that is iOS specific (i.e., multi-touch) that is not possible on the PC. If I get to the point that this is required then I'll have to come up with a work-around for the PC side.

One thing that's implied in all of this is that I did need to implement the texture loader and sample loader twice, once for each platform. This is true, but was relatively simple. One other huge advantages of dividing my projects in this way now is that I'm setting myself up well to support Android and Windows Mobile 7 devices later. Although, I should mention, I am still writing my game classes with a certain amount of reliance on OpenGL, so porting this to another graphics framework would probably be a bit more work (Does Windows Mobile 7 have OpenGL or some mobile DirectX?).

Next time: OpenGL ES 1.1 on Windows.

P.S. One other note on this...
My initial version written in MD on the Mac used standard .NET System.Drawing.PointF structs to keep track of coordinates. Unfortunately, I ran into a problem with this when I separated my projects. Apparently MT does not include a full implementation of System.Drawing and only bundles a few parts inside the main MT library. This means that when I separated my platform independent project for which I did not want a reference to MT they also lot their reference to PointF. My solution was to change to use OpenGL's Vector2 struct, which is basically the same, although it is one of the reliances on OpenGL that I mentioned above that I would prefer not to have.

MonoDevelop isn't bad, but I really prefer VisualStudio

In my last entry I described how I decided to do my iOS development on MonoTouch. Now I discuss how despite the increased efficiency of a familiar language (C#), I still struggled with the MonoDevelop IDE.

First, let me say that my dissatisfaction with MonoDevelop (MD) is in due to comparing it to Visual Studio (VS). According to Wikipedia, VS started in 1997. Clearly it isn't fair to compare the maturity and depth of features of a product that has been under development for more than 13 years to one that is only a few years old. Also, I'm sure the size of the development teams are greatly unequal. Despite that, the reality for me was that I found MD falling short of what I was used to wanted.

I noticed on the MonoTouch Roadmap page that they plan to extend Mono for Visual Studio to "allow developers to use Visual Studio to develop iPhone applications", but since no timeline is given I have no idea when this will happen. So, I tried to do something on my own.
MD saves projects in the same format as VS, so my first thought was to just open the project there and see what happens. I tried this, but VS complained of some error. I did a search and found a page suggesting a solution called VSMTouch.

The solution was to convert the MD projects so that VS can open them. Although the format of the files are the same, the MD projects refer to certain iPhone specific libraries, which don't exist on a Windows PC, of course. What this means is that even using VSMTouch will create a project that can be open in VS, but can't be compiled.

Well, although that  allows me to edit in my familiar environment, I still was not satisfied. VS has some great features that require the ability to compile (e.g., FxCop and built-in unit test framework). Also, I knew that I would want the ability to run my game relatively often as I develop it to test new things. Additionally, in the long term, MT plans to support Android, and Windows Mobile 7 will support .NET out of the box, so there are strong incentives for me to be able to develop more fully across platfroms.

Next time: Divide and Conquer.

iPhone Does .NET

In my last entry I described how I began the process of developing a game for iOS. Now, I'm going to continue that story with what's changed since 1 and ½ years ago when I started.

When I was developing initially I was reading a number of news feeds related to iPhone development to try to learn as I went. I remember at that time Novell announce a Mono based platform for iPhone development. This was very intriguing to me at the time, since Mono is an opensource implementation of the .NET framework, with which I have a lot of experience; far more than C, C++, and Objective-C, which are the default languages for iPhone development. I didn't really look into it at the time as it was pre-release and, if I remember correctly, advertised as being around $1000 for a license.

When I recently returned to developing my game, however, I again quickly became frustrated by the Xcode development environment and the learning curve involved for me. Around this time Apple announced greatly relaxed development restrictions and I remembered the Novell announcement and decided to see how the maturity of the platform was now. As it turns out, MonoTouch (the name of the Novell platform) is still undergoing rapid development and is at the point where it is quite usable. Also, the price either changed or I misremembered and is down to $399 for individuals.

Novell has a free version of MonoTouch that has only the limitation that you can run your program only in the iPhone emulator and not on an actual device, so I downloaded that and started porting my game over from the old C & C++ & Objective-C version I'd previously worked on. I should mention that MonoTouch includes a version of MonoDevelop, an opensource .NET IDE, that integrates well for developing for iPhone.

After a bit of work I got enough of my game running to see that it would work on MonoTouch and that it seemed to run fast enough on .NET on the emulator on my Mac. But, I was a bit worried how well it would run on an actual iPhone.

I debated for a while, but then noticed that Novell was having a sale for MonoTouch of 15% off in celebration of 1 year of MonoTouch, so I jumped in and signed up. I then deployed to my iPhone and was pleasantly surprised at the performance. Everything looked fine.

Admittedly $399 is still a bit of an investment, but I'd already paid for a used iMac (about $300 or $400, I don't remember) and 2 iOS Developer licenses ($99 each year), so I decided to double-down and improve the chances I actually finish the game this time.

Next time: MonoDevelop isn't bad, but I really prefer VisualStudio.

Zoing: The Beginning...

About 1 and ½ years ago I decided I wanted to write an iPhone game. I've had the game idea for a long time and actually implemented a mostly-working version on my Amiga 20 years ago or so, but with the App Store and the iPhone ecosystem, I felt there was finally a way to get the game out there and into peoples hands.

I bought a used iMac from a coworker, signed up as an iPhone OS (now iOS) developer, and began. I immediately ran into a bunch of issues that slowed me down. I hadn't used a Mac in many years, I was new to OpenGL and found most documentation targeted the full OpenGL API, not the more limited mobile version (OpenGL ES), and I found the Apple Xcode development environment unintuitive and feature-limited (sorry, Apple, but despite you repeating that it is a great IDE, it absolutely pales in comparison to Visual Studio).

But, I kept at it. I figured out how to get around the Mac, I found a few OpenGL ES examples to get me started, and I was able to set aside the features I prefer in Visual Studio and use Xcode well enough. Within a month or so (I have a day job too, so I can't put too much time into this), I got enough running to show the main idea of the game. Heck, I even got OpenAL (the audio system to play sounds in the game) included.

Then I changed jobs, and I became very busy at my new job and had no time to work on my game. Then time passed, and my iOS Developer subscription expired.

But, now I'm back. I've renewed as an iOS Developer, and I'm determined to get the game done and released.

Next time: iPhone does .NET.

Adding PDF's to your iPhone (or iPad) from anywhere

So, you're someplace away from your iTunes account and want to add a PDF to your iPhone's or iPad's iBooks collection. How can you do this?

Well, the first way is you can email it to yourself. The problem with this is that your email account (either sender or receiver) may have size limitations that prevent sending large PDFs. If the PDF is too big then here's another idea...

You can use some online storage service, copy the PDF there, open it is Safari, then choose the "Open in iBooks" option. This will both open it in iBooks and copy it into your library.

I poked around on the net a bit and found that Microsoft's Windows Live SkyDrive works pretty well. You get 25GB free, although there is a 50MB per file max upload size.