Two weeks ago, I wrote about my efforts to port You Have to Win the Game to Mac and Linux, primarily focusing on scheduling and finances. I was also writing in the context of being right in the middle of the task with an unknown amount of work ahead of me. It wasn’t until a few hours after I posted that blog that I was finally able to run an in-development version of the game on a non-Windows platform and reach the title screen without segfaulting immediately.
Over the last two weeks, I’ve finished the remaining work, releasing the Linux port last Wednesday and the Mac port three days later. They aren’t perfect by any means. There are a number of known bugs in each, several wholly omitted systems like patching and telemetry, and a few disparities among all three platforms that I’d like to unify at some point. I’m not done yet, but I do feel pretty good about the builds I’ve shipped and the potential ease with which these changes will allow me to ship future titles on all three platforms simultaneously.
In contrast to my previous blog, this time I’m going to cover the entire porting process from inception to deployment from a day-to-day diary point of view. As this was my first real experience with non-Windows development to speak of (aside from dabbling a bit in Xbox 360 and PlayStation 3 as my job required on past titles), I can bring a first-timer’s perspective, and if you’re a developer who’s been considering taking your games from Windows to Mac and Linux, maybe you can learn from my mistakes. More than a postmortem or a how-to guide, though, I really want to put into words, as objectively as possible, just what goes into this sort of thing.
So why cross-platform? Why Mac and Linux? I knew that I wanted to remain PC-centric; mobile is a different world completely and one that I’m not ready to delve into yet. I like consoles and I do at least as much gaming on console as PC, but typically those have baggage like certification requirements associated with them, and I don’t want to deal with that sort of thing by myself unless I absolutely have to. Targeting the trio of PC operating systems (Windows, Mac, and Linux) feels like an easy choice given these conditions.
I’ve historically always been a Windows user and a Windows developer. The last time I touched Linux prior to this stint was in undergrad, early 2005 at the latest. I’d never owned a Mac despite my affinity for iOS devices. But my own personal preferences notwithstanding, are these platforms worth developing for? According to current Steam survey data, Windows represents about 95% of gamer machines, with the remainder being about 4:1 Mac:Linux. I don’t think it would be unfair to label these as niche markets, so why spend the time and effort targeting them? For me, this came down to three factors: inclusiveness, expectation, and opportunity.
When I say inclusiveness, I’m not strictly talking about shipping on as many platforms as possible. Inclusiveness is about empowering players to play the way they want to play. This is why I make my controls completely bindable (a harder problem than the average player imagines, I would wager). It’s why I implemented configurable FOV in Borderlands 2. What I didn’t expect — and didn’t realize for a very long time — is that although it was never my intent to deliberately exclude anyone, my choice of development platform and tools was having that effect. If you ever want to find out which of your friends are Mac users, release a game only for Windows.
The second reason I chose to target these platforms is consumer expectations. Last summer, I left my job at Gearbox Software to form Minor Key Games with my brother David. We released Eldritch last October, and I’ve had some share of its praise thrown my way, but it is completely David’s game. We made the decision when forming this company that, at least for our first few titles, we would work independently on solo projects but release them under a shared banner. This felt like a good way to maintain autonomy but also have a stronger company identity and be able to share word-of-mouth promotion. At the time, we were both developing exclusively for Windows, which made it easier to maintain compatibility parity and the illusion of a team dynamic. Shortly before Eldritch‘s release, however, David (by his own admission) got bored and decided to port that game to Mac and Linux. That put a burden on me because it created an expectation that Minor Key Games is a company that ships on all three platforms. To target Windows exclusively would look like a failure or a betrayal, or at least a step backwards, and I felt that it could hurt my reputation as a developer personally. While I could take a similar approach of waiting until I were nearing completion on a game before porting it, that would put me in an uncomfortable position of not knowing until the last minute whether that game would port well to other platforms. It was for this reason that I decided to attempt porting a finished game in order to more easily validate the results. You Have to Win the Game being the most prominent game I’ve developed to date and one that touches most parts of my engine, it seemed like an obvious candidate.
Finally, there’s the matter of opportunity. David was quick to remind me, when I questioned the practicality of targeting Mac and Linux, that the Humble Indie Bundles only include games that are available on all three platforms. In truth, I’m not really counting on any monetary gains from this effort. I suspect good will and hopefully recouping some of the development costs are the most I can hope for. But I won’t deny that having a game available on more than one platform can only be a good thing, especially when those platforms are moderately starved for entertainment.
So there were clearly a number of good reasons for this undertaking, but I was still on the fence because of the nature of the work involved. Some of these may seem like very silly or arbitrary complaints, but this is where my head was at.
I think OpenGL is a terrible API. I know a lot of developers prefer one API over the other because it’s the one they learned first. That wasn’t my experience. I learned both OpenGL and Direct3D at the same time as part of the curriculum at the Guildhall at SMU. One of them appealed to my programmer sensibilities. One did not. I know the OpenGL vs. Direct3D debate has been played to death, and I don’t intend to revive it. I bring this up only because it’s important to my personal narrative. After I graduated from the Guildhall, I nuked all my old GL code and built a D3D-facing renderer from the ground up, and it was glorious. I wasn’t looking forward to going back to what I saw as an inferior API.
I was mildly worried about the development time and cost, since I have no real income to speak of. I knew the only easily quantifiable cost would be the Mac hardware; the bigger question mark was how much of my savings I would burn through in this time, and when or whether I could recoup those costs. That was largely the focus of my last blog, so I won’t go into too much depth here. In retrospect, the development time was not longer than expected, and as I’ll describe later, the timing in relation to Steam Dev Days turned out to be serendipitous as well.
Finally, there was an unknowable amount of unknowns. It’s one thing to be told ad nauseam that SDL will solve all your problems; it’s another to actually integrate it into your code, and I have the world’s worst luck with external libraries. It’s one of the reasons I prefer to roll my own tech whenever I’m able. I gain an intimate understanding of the problem, I feel accomplished for having developed my own solution or implementation, and I don’t have to spend hours wrangling linker errors and fueling my latent alcoholism. I don’t usually experience impostor syndrome, but when I’m staring down a block of code that looks completely valid but just won’t link for no good reason, I do find myself wondering why no one else ever seems to have these problems.
The aggregate of all those thoughts is the mindset I was in when I finally made the decision to just go ahead and bite the porting bullet in early October 2013. What follows is a day-to-day account of the actual porting process, made possible by my insistence on documenting every pitfall via tweet.
Friday, October 11: Began porting. I correctly assumed that transitioning from Direct3D to OpenGL would be the largest challenge, so I made it my first goal. To this end, my very first step was to remove Direct3D calls from my renderer. Once that were done, I could begin rebuilding its functionality in OpenGL.
Sunday, October 13: Got the game running with D3D stripped out. The game would launch and shut down correctly, and it was still completely playable, albeit with a blank screen.
Monday, October 14: The next big step was figuring out what to do about my shader code. I use HLSL, but beyond that, I use effect files. These contain HLSL vertex and pixel shader code, but they also specify constants, texture samplers, and render state information. I knew that a piecemeal approach of rewriting all the shaders in YHtWtG in GLSL would just create more problems down the road whenever I had to port another game. I needed a better solution. I began developing an HLSL-to-GLSL converter.
Thursday, October 17: At this point, I felt like I had mostly finished my GLSL converter, though I hadn’t actually been able to try it out yet. As I would find out a few months later at Steam Dev Days, my solution very closely mirrored Valve’s. I did some textual parsing of HLSL files, mostly allowing function contents to remain the same while converting parameters with semantics to attributes and varyings as appropriate and replacing function calls where necessary, e.g., lerp() to mix(). For some cases not easily handled by my converter, I made small edits to the HLSL source that facilitated compatibility without changing the meaning. In order to ensure that my converter produced valid GLSL code, I wrote a small console application to compile and link the output. If this tool failed, I would delete the output of my converter but also the compiled HLSL .fxo file, which would prompt my build tools to attempt compilation and conversion again.
Friday, October 18: I actually began compiling and linking GLSL code. To my surprise, my first attempt worked with no problems. Later attempts did expose bugs, but I was off to an encouraging start.
Monday, October 21: At this point, I had vertex streams wrangled and was finally able to get something recognizable as YHtWtG content on screen for the first time, ten days after I had begun. As exciting as that was, it also revealed just how much was still broken.
Tuesday, October 22: I got most sampler and render state data hooked up. Render target textures were still broken at this point, and I spent a frustrating and fruitless night trying to make them work to no avail. Going to bed on an unsolved problem is never a fun feeling.
Wednesday, October 23: After a very long day of frustrating development, solving multiple overlapping bugs (which I colorfully described on Twitter as a “human centipede of bugs”) and dealing with bad debugging tools, I finally got render target textures working. The game still appeared visually corrupted, but it was clear this was a separate problem.
Thursday, October 24: I tracked down the visual corruption I was seeing to a bug that I was amazed D3D had been able to handle gracefully: I was erroneously allocating an array of three-component vectors for the texture coordinates on the level’s tile buffer, instead of the expected two-component vectors. After the inclusion of a few other bug fixes such as inverting the vertical coordinates on render target textures, I called OpenGL done.
At this point, I had spent a full two weeks on the OpenGL implementation alone. I turned my attention next to input, audio, windowing, and networking.
Friday, October 25: As I had done with the rendering subsystem, I began by completely stripping out DirectX and related Windows libraries. I made the decision to port input first, as I would need it to test other changes. However, I soon discovered that in order to utilize SDL’s input functions, I would first need to create a window with SDL, so my order of operations shifted slightly.
Wednesday, October 30: I finished porting my input system to SDL from a hybrid of DirectInput, XInput, and Raw Input. This left audio and networking as the major subsystems. As my networking library isn’t critical for gameplay in YHtWtG, only for patching and telemetry, I chose to focus on audio next.
Friday, November 1: I replaced DirectSound with a combination of SDL and a custom audio renderer. I have a dependency in my engine on libvorbisfile for playing .ogg sounds, but I did not port this yet, because YHtWtG does not depend on it and because I wanted to minimize the number of external libraries I’d have to wrangle on each platform.
It only took one week to port my input and audio systems to SDL after the two weeks that the renderer alone had taken. At this point, I had a nearly completely functional version of YHtWtG that would run in Windows without any DirectX libraries. It was time to start actually bringing this version to other platforms. As I didn’t have a Mac, I was compelled to start with Linux.
Saturday, November 2: I installed Ubuntu 12.04 LTS. I was thrilled to discover it didn’t promptly brick my system. These are things I worry about. I was admittedly kind of curious to see what the state of the art of the Linux world looked like; my last experience had been with some versions of GNOME and KDE in undergrad which were largely responsible for my lingering distaste for the OS. I could ramble for a while about my experiences with Ubuntu and Unity, but the diplomatic tl;dr is that it didn’t change my opinion. The Ubuntu Software Center is pretty nice, I guess.
Sunday, November 3: I ported my networking library to BSD sockets, which turned out to be a remarkably trivial process. I knew that Winsock was supposed to be pretty close to BSD sockets, but I didn’t realize how nearly 1:1 the translation would be.
Monday, November 4: This was not a good day. This was a very, very bad day. It was the sort of day that ends with me screaming at the walls and crying in the shower. I spent many long and relentlessly frustrating hours trying to compile on an unfamiliar OS, with an unfamiliar compiler, in an unfamiliar IDE. I made some truly gross hacks to appease the demons that live inside the GCC linker, and at the end of the day, I had a program that compiled and segfaulted instantly. Then I got to experience the joys of trying to debug in this environment, and that was the last straw. Exhausted, discouraged, and with no clear end in sight, I canceled the port effort after twenty-five days.
Two months later, I attended Steam Dev Days with a looming sense of dread. I had seen the schedule; I was fully prepared for two days of “RAH RAH LINUX, RAH RAH OPENGL” evangelism. And it kind of was, but not how I expected. The tone was very much one of, “Yes, we know this sucks. It sucks for us, too. But we can make it better.” Hearing about Valve’s efforts to convert HLSL to GLSL was validating and suggested that maybe I had been on the right track. We were told plainly that OpenGL and SDL would get you 90% of the way there (which wasn’t even remotely true — in my case it was closer to 50%), and I wondered if maybe I had stopped just short of success.
On top of that, I had been in a rut since aborting the Linux version, making constant but plodding progress on game systems without much enthusiasm or excitement. While not a fun task in and of itself, I knew that porting YHtWtG would afford me a change of pace and allow me to get back to a project that I cared deeply about, even if only for a little while.
David and I discussed this over Dev Days, and I decided that I would once again make porting YHtWtG to Mac and Linux my highest priority. And once again, I documented the process on Twitter so I have a clear timeline of how things went down. This time, I wasn’t dealing with porting subsystems to alternative APIs; that work was already done. Instead, I only had to make that code compile on other platforms. This turned out to be an ordeal in itself, however.
Saturday, January 18: I resumed Linux development, starting from the point I had left it. The code was hacked to pieces in order to compile, and it segfaulted immediately on launch. As I learned how to work around the idiosyncrasies of the Code::Blocks debugger, I discovered the reason for the segfault was not a surprising one; I was simply looking for my content packages in the wrong location. As such, dealing with paths became my first order of business.
Monday, January 20: I continued to fix bugs, slowly making it further through the application startup process before segfaulting. Most of these were caused by compiling a 64-bit build on Linux. I briefly attempted compiling a 32-bit version for parity with Windows, but this turned out to be another can of worms, and I felt it would be better in the long run to deal with 64-bit portability issues rather than hide them. In the evening, I posted my previous blog about the feasibility of returning to porting effort and soliciting feedback. Shortly thereafter, I finally was able to run the game and reach the main loop without crashing.
Wednesday, January 22: I fixed a bug with textures appearing corrupted and wired keyboard input back up. At this point, the game was playable enough that I could reasonably begin testing actual gameplay features. I noticed an odd collision bug in which the player would collide at the intersection point between two surfaces that were flush with each other, but I didn’t investigate it at this time.
Friday, January 24: Hooked audio up again. This caused some concern at first because the audio was stuttering badly, but this turned out to be due to using the wrong type in a sizeof() operator. This mistake had been benign in 32-bit builds, but the change to 64-bit caused a disparity between the expected type and the given size which introduced audio stutter. I also tracked down a font rendering bug to an abs() call which needed to be fabs() instead. I propagated that change throughout my codebase and was not entirely surprised to see it also fixed the collision bug from before.
Sunday, January 26: After fixing the remainder of the known critical bugs, I sent David a build. It took a few iterations and realizing I had left a hardcoded path in my code, but eventually one of these builds worked. I now had a Linux build I felt confident I could get some practical test coverage on without worrying too much about immediate segfaults. Anticipating the end of Linux development, I purchased a Mac Mini.
Monday, January 27: I put out a request for Linux testers on Twitter and sent builds to the volunteers.
Wednesday, January 29: After gathering feedback from the testers, I fixed all the medium and high priority issues and released a first version.
It had now taken eleven days to wrangle a Linux build into shape on top of the twenty-five days I had spent porting subsystems. Mac was up next, but before I started down that road, I wanted to deal with merging Linux-specific changes back into my root repository and reconciling any non-port engine work that I had done in the root during the two month interim.
Thursday, January 30: This was a long day full of nothing but merges. I bought the Linux changes back into a branch that contained a working no-DirectX Windows build, and then I brought all the recent changes from the root into this Windows+Linux no-DirectX branch. This branch would then serve as the basis for Mac development, and once that task were complete, I would take all these changes back into the root.
Friday, January 31: Mac development was fast but crunchy. In one day, I synced my repository, got a project set up in Xcode, installed library dependencies, got the game running on my machine, fixed a GLSL bug that had never shown up on Windows or Linux, and hacked my way around an SDL bug. By the time I went to bed, I had published a WIP version via Twitter in the hopes of getting some test coverage by morning.
Saturday, February 1: As it turned out, I had overestimated some of what Xcode would do for me, and the binaries I had published were dependent on GLEW and SDL dynamic libraries I hadn’t shipped. I was also targeting OS X 10.9 Mavericks exclusively without realizing it. I lowered the supported version as far as I practically could (down to 10.6 Snow Leopard) and linked GLEW and SDL statically. I had to leave signing the app for another day, but that wasn’t a deal breaker. I didn’t have a whole lot of confidence in the build I released that evening, but aside from some known issues with changing windowing mode and resolution, it sounds like it’s been pretty stable.
It only took me two days to take YHtWtG from Linux to Mac, as most of the hard problems had already been solved by that point. All said and done, it was thirty-nine days from start to finish to bring YHtWtG to these platforms, future bug fixes and improvements notwithstanding.
Though I’ve been touting this effort as bringing YHtWtG to Mac and Linux, what this really represents is a porting of my entire game framework (or at least the critical parts) to those platforms. You Have to Win the Game is simply the test case I’ve been using to prove the technology. There will undoubtedly be more work in the future to maintain and improve these versions, but I have at least of modicum of confidence that this work will facilitate being able to ship future titles cross-platform on day one.
That’s a lot of words, and I feel like I’ve barely scratched the surface of what I dealt with in this process. It was massive in scope, and it was also without a doubt the most frustrating work I’ve ever taken on in my personal codebase. I say this not to scare off anyone who might be considering port work or to guilt anyone who ever asked for a Mac or Linux version, only to illuminate what goes into this stuff. It’s easy to say game development is hard, and it’s also easy to shrug off that statement.
It’s gonna be nice to be able to actually work on a game again soon.