OpenGL and OpenTK Fun

I've been working on updating my OpenGL code to use OpenGL ES 2.0 (instead of 1.1). I've run into a number of problems doing this, many of which are exasperated because I don't if my problems are related to my OpenGL code or the underlying libraries I'm using (OpenTK and the PowerVR OpenGL ES implementation for Windows).

To try to diagnose the problems I tried to start with a simple OpenGL ES 2.0 example. Unfortunately, all of the examples I found were either for native OpenGL ES (not via .NET) or specifically from Xamarin for iOS or Android, where I wanted a simple OpenTK example. The OpenTK distribution actually includes one very simple example (just showing an empty window with a colored fill), but as I mentioned in my previous post, the latest official OpenTK release is from 2010 and has some issues.

I searched around and found what appears to be the most active development branch of OpenTK on GitHub at https://github.com/andykorth/opentk. I downloaded that branch and tried to run the same example, but continued to run into problems.

At first I got a System.PlatformNotSupportedException.

System.PlatformNotSupportedException at OpenTK.Platform.Factory.UnsupportedPlatform.CreateGLContext(GraphicsMode mode, IWindowInfo window, IGraphicsContext shareContext, Boolean directRendering, Int32 major, Int32 minor, GraphicsContextFlags flags) in opentk\Source\OpenTK\Platform\Factory.cs:line 171
   at OpenTK.Graphics.GraphicsContext..ctor(GraphicsMode mode, IWindowInfo window, Int32 major, Int32 minor, GraphicsContextFlags flags) in opentk\Source\OpenTK\Graphics\GraphicsContext.cs:line 134
   at OpenTK.GameWindow..ctor(Int32 width, Int32 height, GraphicsMode mode, String title, GameWindowFlags options, DisplayDevice device, Int32 major, Int32 minor, GraphicsContextFlags flags, IGraphicsContext sharedContext) in opentk\Source\OpenTK\GameWindow.cs:line 222
   at OpenTK.GameWindow..ctor(Int32 width, Int32 height, GraphicsMode mode, String title, GameWindowFlags options, DisplayDevice device, Int32 major, Int32 minor, GraphicsContextFlags flags) in opentk\Source\OpenTK\GameWindow.cs:line 180
   at Examples.Tutorial.SimpleES20Window..ctor(GraphicsContextFlags flags) in opentk\Source\Examples\OpenGLES\2.0\SimpleWindow20.cs:line 26
   at Examples.Tutorial.SimpleES20Window.Main() in opentk\Source\Examples\OpenGLES\2.0\SimpleWindow20.cs:line 112
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

This led me down a little side road of trying to confirm that I had the PowerVR OpenGL ES library properly installed. After confirming it was, as far as I could tell, I stepped through the code in the debugger and got to the Factory constructor in opentk\Source\OpenTK\Platform\Factory.cs.

While stepping through I kept finding that Egl.Egl.IsSupported was false. That code for IsSupported is below.

Oh great, a swallowed exception, maybe that would tell me something helpful. Stepping through the code again and examining the exception showed me a System.BadImageFormatException.

System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at OpenTK.Platform.Egl.Egl.GetCurrentContext()
   at OpenTK.Platform.Egl.Egl.get_IsSupported() in opentk\Source\OpenTK\Platform\Egl\Egl.cs:line 316 

I'm sure some web searching would find it, but I've run into this problem in the past and remembered it had to do with incompatible binary formats. For example, if you have a .NET program compiled for x64 and try to load an 32bit x86 native DLL. I checked the OpenTK solution's build configuration, and indeed, it was set to "AnyCPU". I changed it to x86 and tried again and the above exception went away, but it still didn't work.

Next I got an entirely unhelpful System.Collections.Generic.KeyNotFoundException. With the below stack-trace.

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at OpenTK.Graphics.GraphicsContext.get_CurrentContext() in opentk\Source\OpenTK\Graphics\GraphicsContext.cs:line 344
   at OpenTK.Graphics.GraphicsContext.LoadAll() in opentk\Source\OpenTK\Graphics\GraphicsContext.cs:line 512
   at OpenTK.GameWindow..ctor(Int32 width, Int32 height, GraphicsMode mode, String title, GameWindowFlags options, DisplayDevice device, Int32 major, Int32 minor, GraphicsContextFlags flags, IGraphicsContext sharedContext) in opentk\Source\OpenTK\GameWindow.cs:line 220
   at OpenTK.GameWindow..ctor(Int32 width, Int32 height, GraphicsMode mode, String title, GameWindowFlags options, DisplayDevice device, Int32 major, Int32 minor, GraphicsContextFlags flags) in opentk\Source\OpenTK\GameWindow.cs:line 180
   at Examples.Tutorial.SimpleES20Window..ctor(GraphicsContextFlags flags) in opentk\Source\Examples\OpenGLES\2.0\SimpleWindow20.cs:line 26
   at Examples.Tutorial.SimpleES20Window.Main() in opentk\Source\Examples\OpenGLES\2.0\SimpleWindow20.cs:line 112
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Well, for a pretty low-level media platform this isn't very encouraging as it doesn't describe anything about the real problem and is clearly happening deep in the library. After a lot of debugging I finally found the below bit of code in opentk\Source\OpenTK\Graphics\GraphicsContext.cs.

I missed it the first few times I was stepping through the code, but the comment says it all: Note: this property will not function correctly when both desktop and EGL contexts are available in the same process. This scenario is very unlikely to appear in practice. In my case, since I'm running on Windows, I do actually have both desktop and EGL available. I didn't feel like digging into the code to fix this so that both contexts could actually exist simultaneously, but I remembered the Factory class that's I'd looked into above and made an easier fix. I did this in opentk\Source\OpenTK\Platform\Factory.cs in the constructor. My updated version is below.

You can compare to the original version above, but basically I made it such that the library first tries to see if EGL is available, and if so, uses it. If it is not available then the desktop version is instantiated. You can see that the Default context isn't instantiated unless the EGL contexts don't work.

After all of this I could run the sample program and was rewarded with a 800x600 blue window. Woohoo! Ok, it's not that exciting, but at least it was running and I can now try to get my OpenGL ES 2.0 code working.

Next: Premultiplied alpha

I'm Back

After more than 2.5 years I'm back. I'm going to try hard to get back into spending regular time on my game and actually get it published. So far I've spent about 15 hours just getting it back to the point it was when I last left it. A lot has changed since then, which is why it took so long to just get it working at all. Here're a few things that have changed and some details about what I've had to do to fix them.

1. I've changed my home computer. I used to have a tower Windows PC and a Macbook Air for doing the iOS-specific development portions. I now have just a Macbook Pro Retina, which I run VMWare Fusion and Windows 8. I made this migration a while ago, but finally getting my development environment working well took some work.
2. OpenTK seems to have made some changes, which I don't fully understand since their release date is 2010-10-06 and should have been before I took a hiatus. See more details below.
3. In the interim 2.5 years both XCode and Xamarin MonoDevelop have evolved and new versions have been released that required more updates. Xamarin's product has been renamed to Xamarin Studio and now has some interesting integration with Visual Studio, which hopefully will make development a bit easier. See more detail below.

OpenTK Issues

When I tried to run my game on Windows using OpenTK I got a PInvokeStackImbalance exception saying: This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature. Since I was running against the release build of OpenTK, which doesn't contain debug symbols this error occurred in my GLView class' constructor. It is an implementation of OpenTK's GameWindow. After compiling a debug build of OpenTK, I eventually traced this to the function GetProcAddress in Egl.cs, which is a PInvoke call.

The problem is the EntryPoint should be eglGetProcAddress, not eglCopyBuffers. After changing this and rebuilding OpenTK it worked fine. Doing some searching on the net shows this has been fixed in GitHub, but no new official build containing the fix has been released. It was an easy fix, but a bit disappointing that OpenTK's current official release has such a basic bug in it.

Beyond this, it seems that the latest official OpenTK release differs from what is released with MonoTouch. I'm still not sure of the extend of these differences, but I may write more on this later once I get a better understanding.

Xamarin and Visual Studio

As I discussed in a previous post, I've been using Visual Studio as my primary development environment and only switching to MonoDevelop Xamarin Studio when I wanted to actually deploy my code to my iPhone. Apparently others also are interested in doing this because Xamarin has integrated their product with Visual Studio; you can read more about this on their site. Because they're now providing .NET DLLs that can be used in Visual Studio to compile iOS .NET apps, it means I can simplify one of the issues I had with maintaining and building my code.

I have my game broken into 4 projects, each of which builds into a separate binary. One of these, which I call "Launcher", is the most device dependent, and references device specific libraries. This project can't actually be shared across platforms because the code is too different, so I have both a Launcher.iOS project and a Launcher.PC (in the future I hope to add Launcher.Android and Launcher.WinPhone). For the other three projects I also need separate project files, but 100% of the code is shared. This leaves me with this folder hierarchy:

• Zoing
• Launcher.iOS
  • Launcher.iOS.csproj (project file that references iOS specific libraries)
  • ...iOS specific code files...
• Launcher.PC
  • Launcher.PC.csproj (project file that references PC specific libraries)
  • ...PC specific code files...
• MediaFramework
  • MediaFramework.iOS.csproj (project files that references MonoTouch .NET libraries)
  • MediaFramework.PC.csproj (project files that reference Windows .NET libraries)
  • ...device independent C# files included in both project files...
• OpenTKConnector
  • OpenTKConnector.iOS.csproj (project files that references MonoTouch .NET libraries)
  • OpenTKConnector.PC.csproj (project files that reference Windows .NET libraries)
  • ...device independent C# files included in both project files...
• Zoing
  • Zoing.iOS.csproj (project files that references MonoTouch .NET libraries)
  • Zoing.PC.csproj (project files that reference Windows .NET libraries)
  • ...device independent C# files included in both project files...

MediaFramework is a set of device and framework independent classes that define the building blocks for my game. This includes things like: Audio/AudioSample, Device/Orientation, Graphics/Texture, Graphics/TextureArea, Graphics/MutableShape, Graphics/Shape, Graphics/Size, etc.

OpenTKConnector is a concrete implementation of the MediaFramework abstract classes using OpenTK (OpenAL and OpenGL). For example there is OpenGLES11/OpenGLES11Texture and OpenGLES20/OpenGLES20Texture, both of which implement MediaFramework/Graphics/Texture. Similarly there is OpenAL/OpenALAudioSample, which implements MediaFramework/Audio/AudioSample.

The benefit of having this separation is that my Zoing game just needs to know about the abstract classes in MediaFramework and complete ignores the platform details. For example, AudioSample has a Play method, which plays the sample. That is common for any platforms I will eventually support.

Anyway, restructuring my solution to have the project files arranged as above took a bit of time. I had to recreate the iOS project files to reference the MonoTouch library properly. I also had to move these files around, which is made a little more difficult with version control. Some of this meant examining and tweaking the project files' XML by hand.

This reminds me of one small bug in Visual Studio 2010 that caused me some confusion until I understood it. When you unload a project, then edit the project's .csproj XML file, then reload the project, it seems that Visual Studio doesn't recognize the manual changes you made. To fix this simple unload it again, then reload it again. This is a bit annoying, but once you know about it the workaround is simple.

Next: OpenGL and OpenTK Fun