Here’s demo of the new 60beat GamePad with my friend Mike‘s game Caster HD. I added support to both Caster/Caster HD and the update has been submitted to the App Store for approval. It shouldn’t be too long before the update actually hits the App Store. (UPDATE: Both updates are now live on the App Store!)

In the video I demo the controls starting with the cursor. The Mac/Win/Linux version of Caster has game pad support for the in-game cursor and so it was easiest to just hook into that. You don’t need to worry about switching between touch and game pad controls. Caster has no on-screen control overlays and both control styles will work concurrently so you just plug in and go! No setting switch required.

After getting into the game I demonstrate the settings for the camera movement which is controlled by the right analog stick. The start button will pause the game and give you access to the options menu. There you can tweak the camera sensitivity and invert the axis if you want.

A lot of the other controls are doubled up so you can use whichever you prefer. The left analog stick or the directional pad control walking movement. Button 3 or R2 will jump. Button 4 or the R1 will fire. Buttons 1, 2 and L2 will rotate through the different weapons. By holding down L1 you initiate a dash when moving.

(While I’ve got you here, be sure to check out my new 3D physics shooter Invader Zurp if you haven’t already. It is a ton of fun and there is nothing else like in on the App Store. )

The “.DS_Store” file is an abomination and must be stopped. You know what I’m talking about. I regularly rant about how this annoying file gets in your way, dirties things up and just screws with your stuff in general. Today I decided to do something about it. Before we get to that, lets quickly review what it is and why it sucks.

What Is It?

The .DS_Store is a Finder metadata file created primarily by Mac OS X’s Finder.app. Because of the dot (“.”) prefix it is typically not visible in many file browsers and most Mac OS X users are probably not aware of it. It is regularly created when the Finder accesses filesystem directories. It contains directory information about icon locations, view options, silkscreen configuration and the like. The functionality that it provides is moderately useful, but becoming less and less relevant over time. In any case, a long time ago the horrible decision was made to store that Finder metadata in an actual file (.DS_Store) in the filesystem within the relevant directory. We have been paying for it ever since. Over time more and more metadata relating to files and the filesystem has been added to Mac OS X, but thankfully those have been stored in saner places (extended attributes, etc). For the time being though .DS_Store is still here with us and still causing trouble. What’s so harmful about the file you might ask?

Cross-OS Cleanliness

The .DS_Store file is only useful if you are accessing the filesystem with the Finder. What if you aren’t using the Finder though? What if you aren’t even using Mac OS X at all? At best the files then become wasted space that is ignored. At worst they are horrible clutter that dirties up your file browsing experience. It doesn’t take much online searching to find countless forum posts from Windows users asking what these files are or the innumerable removal scripts that Linux users have authored to clean them from their directories. On Mac OS X, if you insert a USB flash drive, mount a network volume or zip up a directory to give to a friend, more likely than not, .DS_Store files have hopped on for the ride. Whoever has to interact with those directories next that doesn’t use a Mac is going to have to deal with those in some way even if it just having to consciously ignore them. Eventually a power-user facing flag was added that would disable .DS_Store file creation on network volumes (“defaults write com.apple.desktopservices DSDontWriteNetworkStores true“) which was a good move. There are still however, plenty of reasons why Mac OS X users hate having them on their own systems.

Source Control Gunk

All the programmers in the room, raise your hand if you have ever added “.DS_Store” to your .gitignore file or equivalent source control ignore configuration? What’s that? That guy in the back who didn’t raise his hand? Oh, he was sleeping…there’s his hand. The contents of the .DS_Store file is always changing and so even if you did happen to check the .DS_Store files into your repository (furthering it’s unholy propagation) it is always going to be showing up as a changed file and gunking up your commits with changes and data that have absolutely nothing at all to do with your code. I have to wonder how many global man hours have been wasted in configuring source repos to ignore these files or cleaning them out of the repos after someone YET AGAIN added them in their latest commit.

Permissions Playtime

Because the .DS_Store file is so inconspicuous and usually innocuous, a lot of the time we simply forget it is there. Unfortunately when doing things like mass permission changes and the like, the .DS_Store file will unintentionally come along for the ride. Suppose you tighten up the permissions on a directory that has a .DS_Store in it. After a while you decide to go through an manually clean it or lower the permission levels on the contents again. You had better remember that you also raised the permissions on the .DS_Store file as well or you might be puzzled why you don’t have permission to delete the directory or why the Finder isn’t saving your directory options. It is no show stopper. Any competent command line monkey can figure it out and fix it. But again, the few minutes it takes to fix it should never have been necessary. Technology is supposed to save time, not destroy it.

So What Do We Do About It?

Those were just three examples that I thought of off the top of my head. Leave your own .DS_Store horror/annoyance stories in the comments and I will add any more that don’t generally fall under one of those three examples. So they are a time consuming annoyance, easy enough to whine about. But what to DO about it? The other day I again ranted on twitter about them and it was suggested that maybe someone could write a daemon that would listen to fsevents and nuke them on creation. I kept thinking that the evil should really be stopped at the source. The Finder. I wondered aloud what it would take to patch the Finder so that it wouldn’t even create them anymore at all. I decided that the creation of .DS_Store files needed to be stopped. Like Jack Ryan in Patriot Games I decided that “I will make it my mission in life”. For an afternoon at least. So in the interest of cleanliness I came up with a dirty hack. I call it DeathToDSStore. I have documented my investigation and development process below.

Who’s Behind This!? I Want A Name!

I initially suspected that it was solely the Finder.app that was responsible for .DS_Store creation. After a little investigation it appeared like that was pretty much true but there might be other things out there that also create them. My first lead was the DSDontWriteNetworkStores setting that causes the Finder to not create the files on network mounts. I needed to find out where the code that interacted with that setting lives. Examining /System/Library/CoreServices/Finder.app with strings came up empty. It must be in a framework somewhere. Poking around /System/Library/PrivateFrameworks revealed the culprit: DesktopServicesPriv.framework. Ok so, .DS_Store authoring functionality is very likely in this framework. Clients of this framework in /System/Library/CoreServices include Dock.app, FileSyncAgent.app, TimeMachine.menu, backupd and the Finder.app. For the time being I was just going to concentrate on the main offender.

AKA Property Stores

I started perusing DesktopServicesPriv.framework‘s symbols using nm (piped into c++filt) to see what kind of things I had to work with. Hmm, FSVolumeInfo::ShouldWriteNetworkPropertyStores()? That sounds like it could be directly connected to DSDontWriteNetworkStores. Perusing around a little more it appears like .DS_Store files are called “Property Stores” internally. There is even a C++ HFSPlusPropertyStore class present. I had just found what I was looking for. Time to start poking at things.

I Wanna Be A mach_star

I pulled out rentzsch‘s awesome mach_star injection/overriding code and started overriding various routines in that class. For some reason I wasn’t able to use dlsym() to get the address of any C++ member functions. I tried all sorts of permutations of the mangled name that nm was giving me for stuff but nothing seemed to work. “Well, I guess I could always just plug the raw address in.” I thought. So I used vmmap Finder to get the base address of where the DesktopServicesPriv.framework‘s __TEXT section was residing in memory. If ASLR (Address Space Layout Randomization) is in play then this value can be different every launch. (EDIT UPDATE: comex reminded me about the _dyld family of routines and so this is now obtained manually at runtime instead.)

nm gave me the offsets of the functions I was targeting and so it was simple enough to just add them together to get the location of the functions I needed to override. It would obviously be a lot cleaner to be able to determine the address from within the binary at runtime but I couldn’t convince dlsym to give me anything and this was working (unfortunately it adds an external dependencies on Apple’s Developer Tools (nm) and BSD.pkg(c++filt)).

Gotcha!

After a bit of trial and error I found that overriding HFSPlusPropertyStore::FlushChanges() with a function that simply did nothing, successfully prevented the creation of .DS_Store files on both Snow Leopard and Lion. On first inspection it doesn’t appear that overriding this function has any unintended behaviors and appears to accomplish the objective of halting .DS_Store creation quite nicely! Or about as nicely as you can when you are doing something like live patching the Finder.

(Side note: I noticed that I am not the only one using mach_star to live patch the Finder. The Dropbox daemon is doing it as well every time the Finder launches)

There’s An App For That

So I packaged my work up into an app (you can also run it through the command line under sudo with the “-silent” option) and published the source on github. In the near future I am going to add a launchd option so that it will run every time the user logs in or any time the Finder is run (EDIT UPDATE: The option to install a Launch Agent has now been added). Now that I have a solution I am going to put my money where my mouth is and start running without .DS_Store file creation full time. Hopefully I wont run into any major issues (I will report them here if I do) and I am pretty sure that I wont miss their functionality (At least by having new ones created. Existing disk images and CDs with custom layouts will still be read just fine). So far the result is pretty positive! Is this safe? Eh…. The safety/cleanliness/forward compatibility of patching OS components has been discussed and debated ad nauseum. You probably already understand the risks involved. Things look pretty stable but I make no promises. Go ahead and try out the DeathToDSStore.app or build your own from source if you want and let me know how it works out for you. Remember that you need to have the Developer Tools installed, BSD.pkg installed and admin authorization. Death to .DS_Store!

Last September I wrote a blog post introducing Invader Zurp which revealed a little of the back story on how I came upon this new game idea after it’s first 2 months in development. Fast forward to 3 months later and Invader Zurp had just hit the App Store! I thought it would be useful to sit down and review the last 5 months of development, kind of plan out where I want to go from here and go over the events and insights that I thought were most influential during development.

The Story So Far…

So to recap the original blog post a bit, it was the middle of the summer (2011) and I had been working my brains out on Cannonade for the previous 6 months. I was a little discouraged at that point because progress wasn’t coming quite as quickly as I had hoped. Reception from my testers (just friends and family at that point) hadn’t been as positive as I had wanted either. I still had a very clear vision of what I wanted Cannonade to be and still believed that there is a ton of untapped potential for multiplayer-only games on iOS. But there was only so much I that could do as a one-man team and testing a multiplayer game can be quite time consuming. I took the family on vacation in early July and was able to step away from things for a while. It was then that I got an idea for a single player experience that distilled the core gameplay mechanic of Cannonade down to it’s essence. Thus Invader Zurp was born. Within two weeks I had modularized the Cannonade game engine, re-written the graphics sub-system in OpenGL ES 2.0 and had a working prototype. And it was fun! I found myself on very long “testing” sessions playing even after I had verified my fixes. I seeded the first alpha version in early September and wrote the introductory blog post. Then began the journey of finishing the game and kicking the darn thing out the door.

Feedback Lesson 1: Make Written Feedback Easy

I got a lot of really great feedback from my testers all along the way. I learned two very valuable things about feedback through my experiences with Invader Zurp. The first was that you really need to make it dumb easy for people to give you WRITTEN feedback. A lot of my friends would give me awesome feedback when I saw them in person but for the life of me I wouldn’t be able to remember it even the next day. I got on the beta list for the TestFlight SDK and start integrating various features into Invader Zurp. One of the most important functionality that I saw in the SDK was a one-line (of code) feedback dialog. I added an obnoxious red “Feedback” button into the app and made it pervasive except when the user was actively playing the game. This button brought up the TestFlight feedback dialog where they could write to their hearts content. The floodgates opened. I started getting a lot of feedback. Most of it was pretty short, just stuff people had noticed in the moment. But it was those little things that people would not have remembered to tell me later that really helped raise the polish level in a lot of places.

Feedback Lesson 2: Take Individual Opinions As Part Of A Whole

The second important thing I learned about feedback was that any individual person’s opinion needs to be treated as a part of a larger set of feedback. I had friends tell me very confidently that things needed to be this way or that way. My friends are really smart and know what they are talking about and so I would start to worry that maybe I was on the wrong track and I really should change course (sometimes drastically). However, once I took that individual nugget of feedback and put it up against all the other feedback I was getting I was able to remove my more emotional reactions to it and look things more objectively. This meant that even if my friends were right and I was wrong, I could deal with it in a better way by understanding how it fit in to the entire picture of collective user reaction. I could see patterns emerge across multiple testers that indicated that a particular piece was either very wrong or very right. I wasn’t doing game design by committee though. I still took minority advice sometimes when I felt it was the right thing to do and ultimately I had to veto some majority feedback when I truly felt that it didn’t mesh with the overlying game vision. I was willing to try anything though and I did in fact implement a lot of crazy feedback that my testers gave me just so that I could see what it felt like in action. An idea on paper and the same idea in implementation are often completely different things. I mean, just look at the “it’s just a big iPod Touch” iPad.

Building A Game For Me

The overall gameplay concept of Invader Zurp was originally inspired a lot by Ikaruga. Ikaruga is a game that you can completely play through in 25 minutes. But the REAL game is not simply completing the levels from start to finish, but rather it’s mastering the intricate chaining system and racking up huge scores. I thought that this was a really deep gameplay mechanic. One that really wouldn’t be appreciated unless the player invested some time into it. However, this deeper gameplay and Ikaruga as a whole is not regarded as being a very casual friendly game (casual gaming being what the iOS App Store seems to reward a lot). Treasure (the developer) is well known for catering towards the more “hardcore” elements in gaming. I initially designed Invader Zurp in a similar manner to Ikaruga in that you could easily burn through all 100 waves of structures in about 20 minutes. The REAL game and the mechanic that would add substantive depth would exist in a system that rewards you for making tricky mid-air block hits. Anyone could shoot the blocks and destroy the towers but it took real skill to hit things in such a way that the blocks would pop up into the air and you needed to be lighting fast in order to target them before they hit the ground. Doing so would reward the player with a lot of points. Coupled with this is an automatic upgrading system that directly ties to your score, the maximum number of missiles that you have to shoot, how fast the missiles regenerate after being shot, how fast your shields regenerate and how powerful your missiles are. The idea was to have a nice smooth power ramp up. You start out with only a few, slowly regenerating missiles and as you get more points, you gradually upgrade to become an uber powerful, unstoppable destruction machine. The thing is that you need to make sure you are scoring a lot of points so that your upgrade curve stays ahead of the difficulty curve. The waves coming at you steadily increase in turret number and rate of fire and so if you don’t make sure to rack up tons of points along the way you will eventually not be able to keep up with the onslaught. Learning the point-heavy, mid-air shot skill is really important because it compounds on itself very quickly. The more points you get, the more upgrades. The more upgrades you get, the higher potential for you to be able to milk points and stay in the game. At this point in development the player always started the game with the same upgrade levels every time. Because of this you could very clearly observe your own skill increase by seeing how far you got before death or how high a score you were able to rack up before finishing the game. It was a very pure game concept that I enjoyed quite thoroughly.

Building A Game For The Masses

However, like Ikaruga, this mechanic’s hardcore simplicity didn’t appeal to everyone. Many people thought it was too simple and they said they didn’t feel like they had a reason to play it again once they got a little taste of it. They kept wanting me to put in more stuff and add more to complexity to the gameplay. I didn’t want to complicate things up though. I wanted to keep things “Fruit Ninja” simple and I felt that every added gameplay intricacy would need to be baked in for a long time to see if it really worked in the game (idea vs implementation). I had a long talk with my friend Reed Olsen about it one day. We talked a lot about what kind of games are successful right now on the App Store and what kinds of psychology they were tapping into to breed their success. I didn’t want to simply follow a pattern and become a “me-too” game, but there were many clear examples of gaming hooks that were extremely effective in getting people to play and continue playing that I couldn’t ignore. I thought long and hard about these and tried to find a way to integrate some without losing the gameplay essence I had already refined. After a long period of agonizing I decided to test out an in-game currency system. This added a very valuable psychological hook of giving the player the feeling of progress every time they played the game. Regardless of whether they did poorly or totally rocked it, their total accrued money would still go up. They would be farther than they were last time. They would feel accomplishment. I was wary of adding this element to the game mainly because I didn’t want grinding to ultimately overcome skill and I was also keenly aware that in-game currency systems (especially the ways in which they are implemented in freemium games) are distasteful to many in the gaming community right now. But I decided that I was going to be willing to give anything a try. I allowed the player to purchase upgrades outside of the gameplay that would increase the base level that their in-game upgrade levels started at. This meant that I had to adjust the difficulty curve up so that the player would need to save currency up and purchase upgrades to (realistically) be able to finish the game. No longer would a very skilled player be able to just waltz in and finish the game with just the initial upgrade levels. They would need to put in some time and work to get accomplish that goal. In practice I found that this worked fairly well at providing a hook for the less “hardcore” players and didn’t detract as much from the pure gameplay mechanic as I thought it would. I eventually had to bump down the maximum missile regeneration level because once the player had max’d out all their upgrade levels they basically became an unstoppable block-killing machine. Skill kind of got thrown out the window for the high end players at that point. After making some small adjustments to the max upgrade levels I found a nice sweet spot that made the player feel like through their efforts they had grown into a powerful, planet ravaging machine but still left room for the high end players to compete against each other with their pure skill.

Further Motivation

Another gameplay addition I developed grew out of a conversation with my friend Mike Smith. Again, he wanted some additional gameplay element to increase the complexity and I was resistant. But in the course of our chat we came up with an idea that I thought meshed really well with the existing mechanic and wasn’t confusing or overwhelming. The idea was suggested that I hide three special blocks in every wave and tie some sort of bonus to hitting all of them. This was really appealing to me because it was super easy to implement and addressed a major complaint that a lot of people had up to that point which was that they always felt like they were out of missiles. I decided to put three shiny gold blocks into every wave and give the player a full missile reload if they were able to hit all three. This meant that now there was now an opportunity for a full missile reload at pretty much every moment. The player now had to balance whether they were going to take the risk of spending ammo to go after the gold blocks or write them off. It was a great improvement on the gameplay and also allowed for another progress-based psychological hook. I could keep track of how many blocks the player got in each wave and record those in a running stats menu outside of gameplay. That way the player can see the progress they are making towards getting all the gold blocks in every wave. This kind of optional “collect” mechanic is used to great effect in tons of games these days (in my mind most notably in the Mario franchise). I then layered on an additional mechanic of keeping track of how long the player can maintain getting all three blocks in the waves in a row and giving a bonus missile damage multiplier as their streak increases. This seemed like a very easy element to simply layer on that would add depth without adding complexity and confusion. I then did a similar thing with enemy turret destruction streaks and started keeping track of those stats as well. The feeling of having your accomplishments recorded can be pretty powerful and very motivating to people with completionist mindsets which brings me to the next element I layered on.

Achievement Unlocked: Make 50 Achievements, Ship Game

Achievements are a no-brainer. They are super easy to implement and you get a lot of psychological mileage out of them. I found that coming up with crazy achievements and writing their humorous descriptions was some of the most fun I had during the development of Cannonade and Invader Zurp. It is well documented that having a lot of easy early game achievements can do a lot to keep people invested in your game and motivate them to keep playing. Coming up with a nice curve of achievements for the beginners, intermediate, advanced and insane was pretty fun and actually not that hard. I didn’t need to worry about giving any skill level too many. I could just make whatever achievement popped into my head. I loved coming up with really random ones that players would accidentally run into during the course of the game. “Huh? What was that achievement? Eh whatever, cool!“. I was really proud of a lot of the locked/unlocked achievement descriptions that I came up with. Unfortunately Game Center has visually small spaces in their app to display those descriptions and so anything more than just a basic description gets truncated. So when perusing achievements, be sure to do it in the in-game menu where you can view the “Director’s Cut” achievement descriptions.

Future Plans

I had a lot of other cool ideas that I wanted to implement in Invader Zurp but I knew that I needed to pare things down and determine my exact 1.0 feature set or else I would never get around to shipping (“Sometimes you just need to shoot the engineers and ship the darn thing“). I had a lot of ideas like:

• In-App purchase to allow people to buy in-app currency (kind of a given these days on the App Store).
• I also wanted to rip out and incorporate the Cannonade level editor (what I used to make all the levels) into Invader Zurp so that the players could design and ulpoad their own building designs and have them trickle down into the game. It would be really cool to see a different set of towers every day, each with a little “Designed by GamerX” credit under their announcement title.
• I want to have alternate missile effect sets like psychedelic rocket contrails or themed missile explosions like a burst of hearts when missiles explode for Valentine’s day.
• I wanted to have a lot of analytics hooks and an App Store release “Feedback” button that the players could use to interact with me.
• I also wanted more Zurp character art, iCloud save game support, Game Center avatars, Twitter integration and a lot of other little things.

Many of these will find their way into a future update. Just need to prioritize them correctly.

Submission Deathmarch

About a month before release I was talking to my friend Mason Sheffield about what I felt I needed for a 1.0 release and how far off I thought I was. He challenged me to get 1.0 on to the App Store by Christmas (2011 ). I thought was a really unrealistic target. I looked at what still needed to be done and saw an unsurmountable mountain. But December 25/26th is the biggest day of the year for the App Store and it was also ahead of my own in-head release target. I decided that this challenge was just the kind of motivation I needed and it put a fire under me. The App Store review team was going to be shutting down on December 22nd and I thought that during this season there would be a lot of other developers also trying to make the deadline in order to get on to the App Store before Christmas. I conservatively estimated that a review would take two weeks and so that put my submission deadline at December 8th. So for the last month of 1.0 development I worked like crazy and started aggressively cutting things right and left. About two weeks before the deadline I hit a moment when I realized “Hey, this thing is actually coming together! I might actually make it!”. I was quickly heading toward the light at the end of the tunnel. I was seeding every two or three days (much faster than my usual goal of once a week) and felt like I was running a marathon. Soon enough though the day of truth arrived. I actually decided then to give myself two extra days to finalize a couple of things and squash as many bugs as I could before submission. On December 10th I finally submitted Invader Zurp version 1.0 to the App Store. It was amazing how many truly egregious bugs I found on the day of submission and was thankfully able to squash them right before submission. I figured that I would probably get approved (fingers crossed) just in time for December 25th and that would give me enough time to make a website, gameplay trailer and find reviewers to give promo codes to. To my surprise, only four days later the iTunes Connect app on my phone buzzes to tell me that Invader Zurp is “In Review” and only 40 minutes later it changed to “Ready For Sale”. This was a nice surprise and actually caught me a little off guard! So now that the game is finally for sale the time has come to start promoting it and preparing for the first update. I definitely want to be in the league of other great games that have quality content/features in every update. I want my users know that when they get an update from me it’s going to be something new and good to explore. Aright, enough writing, back to work!

(Interesting side note: I found out during app validation that you can’t submit an app with an unresolvable symlink in it. I was doing this intentionally though. Email me if you want to know why.)

Aright it’s here! Invader Zurp, the project I have worked so hard on for the last 5 months is now for sale on the App Store! I want to thank all my wonderful testers for all their time they put into playing it and even more for the thoughtful and productive feedback that they so lavishly furnished on me. I couldn’t have done it without you!

Check out the gameplay trailer here:

In developing Cannonade and Invader Zurp I have invested a fair amount of time becoming familiar with the Bullet Physics Library and trying to milk every bit of performance out of it as I can. Realistic physics simulation plays a crucial part in both games and is also the performance bottleneck in the majority of gameplay scenarios with both. When trying to optimize for performance I generally see myself using two kinds of approaches. One is a higher level algorithmic approach that tries to see ways to create less work or avoid work in order to keep things going fast. Once I have nailed down as best I can, the minimum set of work that I really cannot avoid doing, then comes the work of getting down and dirty and speeding up the routines that actually do that work. When I initially approached the problem of speeding up Bullet, I first simply treated it as a black box (work that I wouldn’t be able to avoid) and explored what kinds of compiler configurations I could leverage to create the fastest possible execution of the physics simulation work. Later, after I had nailed down the gameplay mechanic for Invader Zurp I was able to start specifically attacking the set of physics simulation work needed for the game and whittled it down to a much smaller amount using some simplifications, accuracy compromises and psychology.

Work, work… More work?

So a basic description of the gameplay in Invader Zurp is that you, the player are flying in 1st person over a planet’s surface towards structures made up of variously shaped and sized blocks. The player taps the screen to fire missiles at the structures which then explode. This explosion destroys some blocks and sends other blocks flying through the air. There are also some special blocks (called turrets) that shoot missiles back at the player which must be intercepted or else the player takes damage. In the next couple of sections I am going to step through the procedures that I go through in order to make sure that my physics simulation work is as small as possible while still giving the player a good gameplay experience.

Less Stuff, Less Work

First things first. The player is constantly moving forward and “blocky” structures are being brought into the simulation at the horizon regularly as the player plays the game. Obviously more blocks means more work and so minimizing the amount of blocks in the simulation is imperative. Because the direction the player is moving never changes, it only seems natural that once a block has moved behind the player’s viewpoint that it should be removed from the simulation. In this game a block behind the player can no longer be interacted with (usually) and so it is simple enough to just remove them from the simulation. I also thought about removing blocks once they have left the player’s view frustum but I decided that I should still keep those blocks in play in case a missile has already targeted it and is currently homing in on it. The player might be really disappointed to have successfully targeted a special bonus block only to have it lifted from their grasp as it leaves their view. In addition to removing blocks behind the player viewpoint I also remove blocks that have been pushed too far into the background.

Shh… Don’t move. It’s CPU Load’s Based On Movement

Another very basic performance concept in physics simulation is that if objects are not moving (or are moving so slowly as to not be perceived as moving) we can skip over a large part of the simulation for them. You can set a movement threshold that causes blocks to become inactive from a lot of the simulation once their velocity falls below it. This functionality is built in to Bullet (btRigidBody::setSleepingThresholds()) and most other physics engines. In Invader Zurp I chose to bump up that threshold higher than normal so that blocks would become inactive sooner. I also set the friction coefficients of the blocks and the ground such that blocks would slide around less and fall under the inactivity threshold quicker (btRigidBody::setFriction()). Because the player is constantly moving there is not a lot of perceived difference to the player between a block that is sliding around a little bit and one that is stationary. Thus, aggressively changing these values didn’t result in a very different experience for the player. Also, when new block structures are added to the simulation, every block is always initially flagged as inactive (btRigidBody::setActivationState(ISLAND_SLEEPING)). This is because in Invader Zurp it is assumed that the structures are all initially standing still. This does have the interesting side effect of allowing for structurally unsound buildings to appear to be holding up. Blocks can be suspended in air and blocks can also occupy the same space at the same time indefinitely. Of course the instant a moving block collides with it the entire thing falls apart but I decided that the charm of having interesting (but structurally insufficient or conflicting) building designs outweighed the realistic depiction of the structure.

Accuracy Is In The Eye Of The Beholder

There is an interesting problem in traditional realtime physics simulation called “tunneling” which I describe in depth in another blog post. In that post I describe some techniques to deal with the problem of relatively small objects tunneling through others and different ways of dealing with it. In Invader Zurp the fastest moving objects are (usually) the missiles that the player shoots at the oncoming structures and incoming enemy missiles. The missiles travel quite quickly (especially when they are completely leveled up) and at a normal simulation granularity they can easily tunnel through the blocks. I considered special-casing them and simulating their movement and collision myself but I eventually decided to have them remain as Bullet simulation objects and keep (most) collision detection for them inside the physics engine. Taking some cues from the solution I came up with for Cannonade (“Convex Sweep Collision Based Time Slicing“, discussed in the tunneling blog post) I decided to start from there. To ensure that the missiles do not tunnel through blocks even at their high rate of speed I decided to increase the granularity of the simulation high enough so that they could not completely tunnel through an individual block. Because doing so increases the needed computational power substantially, I had to make sure that I only did it when absolutely necessary. So every frame I would spend some CPU cycles performing convex sweep collision queries (btCollisionWorld::convexSweepTest) for the fast moving missiles to see if they are likely to hit a block within the next simulation frame. If it seems like a collision is likely then I see what the fastest velocity is of any of the missiles that will likely hit a block during that frame. I use that velocity to determine just how much more accuracy (and in turn granularity) my simulation needs to ensure that the fastest moving missile will not tunnel. The cost of performing convex sweep collision queries is expensive but in the long run they save much more time by making sure that the simulation runs at lower accuracy level when missiles aren’t going to tunnel. Those convex sweep queries can still get quit expensive though. I started looking at ways to avoid performing it. I looked very closely at the specifics of the missile’s role and player perception in my game and got an insight. When you tap on the screen an algorithm determines which block or enemy missile you most likely intend to hit with your missile. Your missile is entered into the simulation and undergoes forces that cause it to home into it’s target until either it or it’s target is destroyed. Because of that strong homing force, I noticed that 9X% of the time the only collision that a missile would ever experience in it’s lifetime is one with it’s target. I figured that if I only performed the convex sweep query for a missile when it was getting close to it’s target that I could forego performing a ton of queries when the missiles are not close to their targets or much less when they aren’t even close to any block at all. It is substantially less expensive to simply calculate the straight-line distance between a missile and it’s target and then check if that distance is within a certain threshold. If it was within the threshold only then would I actually perform a convex sweep query to determine if I really did need to increase the simulation granularity. A side effect of this is that occasionally missiles will tunnel through blocks that they are not targeting. In practice though I have only actually noticed this happening two or three times. The computation savings was well worth the compromise in this specific application. I took this optimization one step further and also decided to sort the missiles per frame in order of how close they were to their targets. This meant that the missiles which were more likely to come up with a positive collision as a result of the convex sweep query were examined first. This then meant that a contender for the fastest moving collision during the frame was calculated earlier and thus increased the likelihood of being able to forego convex sweep queries on missiles that had less of a chance (due to their increased distance from their targets) of coming back with a positive collision because they were perhaps moving slower than the missile that we already had determined had collided. All of these optimizations led to a significant performance boost without sacrificing significant user perceived accuracy. Occasionally when there is a lot going on you can see individual frames take longer as the simulation granularity gets spiked but it is usually only for a few individual frames. For the most part, the player taps, sees a missile shoot out towards the structure and then watches it blow up, all with a fairly regular frame rate.

They Are So Small Anyway…

In that same strain of missile collision performance, I devised another slight performance win but it was instead born out of player experience rather than an egregious slowdown. In addition to targeting structural blocks, your missiles can also target the enemy’s defense missiles which are constantly homing in on you. Unfortunately, because of the missile’s speeds relative to their sizes, what would usually happen is that the player’s missile would speed towards the enemy missile very quickly and then violently hover around it for a while before finally colliding with it and exploding them both. This was not a very fun player experience and was also perceived as being a very erratic result. So I decided to pull missile to enemy-missile collision into my own code instead of using the physics engine. So I instead calculate the straight-line distance between the two and when the distance falls below a certain threshold the missile explodes and takes the enemy missile with it. This makes for a much more satisfying player experience. The player sees their missile heading toward it’s target and then both exploding when they appear to intersect. In addition to being a better game experience it saves on collision computation as well. In fact, we can probably say that we don’t need to even compute collision between a player missile and an enemy missile in the physics engine at all! Using collision flags (btBroadphaseProxy::m_collisionFilterMask, btBroadphaseProxy::m_collisionFilterGroup) I separated the missiles into specific groups that would only collide with the ground and blocks. The likelihood that two missiles would collide with each other when out in play area away from the ship was somewhat small (much less have a user even notice that it should have happened but didn’t) and so pulling those objects out of collision computation with each other was significant win. I also found that this was necessary because when the player got to higher levels of the game and started to fire missiles much more rapidly, their own missiles actually WOULD collide with each other (fairly often actually) and explode themselves very quickly after being fired because of the missile density around the area they were being fired out of the ship from. This was also a bad player experience that was fortunately also simply solved by taking the missiles out of collision tests with other missiles.

Modifying Bullet

At one point I experimented with extending Bullet itself to allow for more specialized control over it’s execution. I added some flags to collision bodies that allowed me to selectively control which bodies were fully put through collision detection and contact resolution system. My aim was to lower the hit of increasing the simulation granularity by keeping most of the simulation progressing forward at the rate of one full frame and instead only allowing the speedy missiles that were likely to tunnel undergo multiple sub-frame physics passes. That way the only increased performance hit would be involving the additional physics calculations associated with objects that I had determined were likely to tunnel. This would of course also sacrifice some accuracy because not all of the objects involved in a high granularity collision will be progressing forward like they normally would but the potential performance gain was too high to ignore. I took about a day to investigate this avenue and got things up and running but they didn’t quite work correctly. There were clearly many peripheral effects to my modifications that I did not yet fully understand and so I decided to instead keep Bullet an unmodified black box for Invader Zurp version 1.0 and planned on re-visiting that idea again when I had more time to dig into Bullet’s internals.

Other

Other small things I did to try to milk more performance was to keep my landscape geometry as simple as possible (in the case of Invader Zurp it is just one simple plane). I also read that adding the SOLVER_ENABLE_FRICTION_DIRECTION_CACHING flag to m_solverMode of btContactSolverInfo and lowering the m_numIterations in btContactSolverInfo (I lowered mine down to 2 down from the default 10) you could possibly gain some performance at the cost of simulation accuracy. Because of the quick gameplay pace and constant movement of Invader Zurp I felt that it was fairly tolerable of accuracy loss and so I added those settings in to my configuration. I didn’t do any formal or extensive testing to see if this had any effect but to the naked eye I couldn’t discern any significant performance gain (or loss, or accuracy loss for that matter) and so I decided to leave them in.

Conclusion

Well, that is about all the ones I could remember. I am sure that as time goes on I will be able to figure out more ways to squeeze the most out of Bullet. The grand takeaway is that in your application (especially games) it helps to look at your own specific situation and perform very narrow situation-specific optimizations. Are any of these optimizations applicable in your game/application? Any other ideas on getting more physics bang for your buck? Let me know in the comments.

So I have these thoughts…a lot. No not that kind! I get these fun ideas for neat projects or crazy inventions as I am driving in the car, singing in the shower, lying in bed or even in the midst of coding. Really creative ideas that may or may not be realistically viable but worthy of entertaining nonetheless. I get really excited and start working it over in my mind. It feels like inspiration. Then something happens… Life resumes and I need to deal with the actual task at hand. No worry, it was such a compelling idea that I will surely remember it later when I have cycles to devote to it. And then I never think about it again. Or even worse, the next day rolls around and I all of the sudden remember that I did think of a cool idea the day before, but I can’t quite bring the memory into focus. Does any part of this sound familiar to you?

Write It Down

“No more!” I declared. Nothing gets remembered unless it is written down. Hmm, idea notebook? TextEdit file on the computer? I did the text file thing for a while but I kept forgetting that it was there or didn’t have access to it at the time. No worries… Evolution of the computer to the rescue! Transferred things to Notes.app on the iPhone. Not good. Too jumbled, un-organized, too much information presented in-general. I knew of the million and a half “getting things done”, task management, mind-mapping and brain organization apps/ecosystems out there. But I found most were too general for what I wanted and too heavy. Well, if you want something done…

Version 1.0

I decided that I was going to create a web page where I could dump all my ideas instead. It would sort and organize the ideas so that I could easily review them when I was looking for an idea. I am not a web programmer and so it was kind of fun to learn about the latest in AJAX-ish web technology and poke around a bit with it. I got something up and running within an evening and it actually worked pretty well. No longer was I going to lose those awesome little nuggets of inspiration. I called it “Think Tank“.

Gone Native

Some time passed and I was thinking about how one of these days I should really re-write the Think Tank web page cleanly using modern web standards and methodologies. But then it dawned on me: “You’re a native iOS programmer. Why don’t you do what you do best and implement Think Tank as a native iOS app?” Yea! Then the interface would be nice and fast and I could take advantage of all sorts of iOS goodness as well. What’s more, I could then put it on App Store for others to take advantage of. I set to work and quickly got it up and running. I have been using it myself for about 3 weeks and it just what I wanted. Simple, straightforward and fast. Now when I get an idea I just whip out Think Tank, jot down what is in my head and get back to what I am doing. And now that Think Tank has just been released on the App Store for $0.99 anybody else can take advantage of it now as well.

Above is some footage from the current Invader Zurp alpha (apologies for the bad exposure). As you can see, things have come quite a ways since I released the first video! There are a lot more visual effects, 5 music tracks from Monte ‘Trance’ Emerson, a new gameplay mechanic, an in-game currency system and tons of other little advancements. I must say that I feel like things are progressing quite nicely! I still have a ways to go though and so I am going to continue cranking away with my head down until it is finished.

What are your thoughts on how development is going? Let me know in the comments. Also, a huge thank you to all of my testers who have given me invaluable feedback on all the builds thus far.

The Startup Grind is expanding! You may or may not have heard that we have recently opened up a Los Angeles chapter of the Startup Grind and to coincide with that we have added multiple chapter support to the Startup Grind.app! Now if you are a member of the Startup Grind LA your upcoming and past events should be visible when you log into the Startup Grind.app with your Meetup account. If you are a member of multiple chapters then you will be presented with a top level pane allowing you to switch between them at will.

As we open up more chapters of the Startup Grind you will be able to access all the information and interactive elements for the events for every new meetup that we get going. More awesome features are in the works so stay tuned!

In a previous blog post I outlined my need for an iOS server. I had found a sufficient but non-optimal solution for iOS devices running iOS 4.X. I mentioned at the end of that article that I had found an optimal solution utilizing some new features in iOS 5. Now that iOS 5 has gone gold master and the NDA has been lifted I can outline in detail how to get your own iOS server up and running. To review, the three requirements for setting up a server in my situation are that it must:

1. Be able to receive push notifications (so it can get it’s work)
2. Have it’s display turned off (to save energy and avoid things like screen burn-in/fatigue)
3. Require no human interaction (needs to be completely autonomous)

In the previous article I outlined why these were in conflict with each other on iOS 4 devices. However, there is some new functionality and behavior policies that allow all three requirements to be fulfilled.

The Solution? Newsstand

Yes that’s right, Newsstand. As in the virtual magazine functionality added to iOS 5. It seems that Newsstand apps are granted a nice little exception in iOS 5 in which they are able to receive push notifications even if the app is not the active application. Newsstand apps are the only apps that are currently granted this exception and so other apps will still not receive delivery of push notifications even if they are in the foreground but not the active application. So the first thing you need to do is add the UINewsstandApp key with a value of YES to your application’s Info.plist. In Xcode, my server app is a different build configuration of the client app and so I added this line to a build script:

/usr/libexec/PlistBuddy -c "Add :UINewsstandApp bool YES" 
"$CONFIGURATION_BUILD_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION/Info.plist"

Another Day, Another Mode

Now Springboard thinks your app is a Newsstand app and so it will from now on live in the magazine shelf inside of Springboard. The next thing you need to do is add newsstand-content to your UIBackgroundModes in the Info.plist. (in my project I don’t have any other background modes and so I just deleted the old one every time) Now this will allow your applications to receive push notifications without being active.

/usr/libexec/PlistBuddy -c "Delete :UIBackgroundModes" 
"$CONFIGURATION_BUILD_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION/Info.plist"
/usr/libexec/PlistBuddy -c "Add :UIBackgroundModes array" 
"$CONFIGURATION_BUILD_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION/Info.plist"
/usr/libexec/PlistBuddy -c "Add :UIBackgroundModes:0 string newsstand-content" 
"$CONFIGURATION_BUILD_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION/Info.plist"

The Special Bit

Your app will now receive the special push notifications but said push notifications need to be flagged as Newsstand content for the system to actually pass them along to your app. To do that you simply need to add one more field in your JSON push notification that you send to Apple’s servers:

$payload['aps']['content-available'] = 1;

Note that that is the number 1, NOT the string “1″.

Early Registration

Inside your app’s execution you of course need to register to receive push notifications so that they are actually delivered to your app. There is a new notification type that corresponds to the Newsstand content flagged notifications called UIRemoteNotificationTypeNewsstandContentAvailability. In my app I register for all types of notifications:

[[UIApplication sharedApplication] registerForRemoteNotificationTypes:(UIRemoteNotificationType)(
UIRemoteNotificationTypeBadge |
UIRemoteNotificationTypeSound |
UIRemoteNotificationTypeAlert |
UIRemoteNotificationTypeNewsstandContentAvailability)];

One Step Forward, Two Steps Back

We have gotten pretty far but there is still a slight roadblock in achieving our goal of an iOS server. You might notice now that on iOS 5 when you hit the lock button not only does your app become inactive but it also gets backgrounded (this is in contrast to the iOS 4 behavior of becoming inactive but remaining in the foreground executing). The solution to this is to actually take a step backwards and remove it’s iOS multitasking capabilities. You can do this by adding the UIApplicationExitsOnSuspend key to the Info.plist:

/usr/libexec/PlistBuddy -c "Set :UIApplicationExitsOnSuspend YES" 
"$CONFIGURATION_BUILD_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION/Info.plist"

What this does is it returns your app to pre-multitasking behavior so instead of continuing to execute or backgrounding when you hit the home button your server app now exits. This has an interesting side effect of keeping the app both in the foreground and executing even when the lock screen is up (to preserve behavior of applications written this way no doubt). So now your program not only keeps running while the lock screen is up and the display is off but it can also receive and immediately service push notifications.

The Unlimited Plan

But wait! We aren’t quite out of the woods yet. You might have noticed that you are only allowed one non-active delivered push notification per 24 hour period. Obviously you want your server to receive as many push notifications as you need. You can unlock this restriction actually by writing into your defaults the key NKDontThrottleNewsstandContentNotifications when you launch:

[[NSUserDefaults standardUserDefaults] setBool:YES forKey:@"NKDontThrottleNewsstandContentNotifications"];

Victory Dance

Now you have it! An iOS server that can completely power down it’s display, receive push notifications and require no human interaction. Now just plug in into a power source, sit it in the closet and let it do it’s thing.

Of course, you realize that an app authored in such a way would never pass the muster required to make it on to the App Store. But then again, this app is probably never going to leave your closet or server room anyway right?

Want to buy Caster for webOS? Click Here

Caster History

Much like Amit Singh and his Hanoimania, I regularly attempt to port my friend’s game Caster to as many platforms as I can. Because of some good language and API choices upfront (C++, OpenGL 1.X, SDL, etc…) the Caster codebase is very portable. That portability combined with its relatively humble (for these days) performance needs has lent the (currently still unified) codebase to easy porting. This multi-platform journey began many years ago before Caster was even released. Caster was still in early development when Mike approached me with the idea of maintaining a concurrent Mac build. At that time Macs had not yet moved over to x86 processors and so the most difficult part of the process was reverse engineering Valve’s Half-Life model format (which Caster uses for animated character models). The Half-Life model format was a binary, in-memory format utilizing offsets which meant that while loading it was quick and simple (just read it into memory, no processing whatsoever) it was unfortunately dependent on little-endian byte order to work. Other than that and a couple of other issues, the Mac port was really straightforward. So before release, Caster was solidly supported on both the Windows and Mac platforms. After release another guy helped out with the Linux port of Caster and so at that point we now had the major desktop operating systems covered. Over the years we added slightly different versions for the desktop platforms to adapt them to specific distribution avenues like Steam and the Mac App Store. The first port that required major reworking was iOS. That adventure was chronicled in my post Caster for iPhone: A Postmortem. Mike and I had the opportunity to give a presentation about what we learned at GDC Austin ’09. Later on as the Android and webOS mobile platforms took shape, I started to look into porting Caster to those platforms as well.

Adventures in webOS

About a year ago I was able to get a hold of a Palm Pre from a friend and so I started poking around the webOS PDK (Plug-In Development Kit). Unlike iOS, the primary development environment for webOS is (duh) web-based and not suitable for high performance games. However, through the webOS PDK you can write C-based programs that utilize OpenGL and other libraries much more suitable to game development. I was able to leverage a lot of the work from the iOS and Linux ports and within a day or two I was able to get a working version of Caster up and running using the PDK. Hardware-wise the Pre is somewhat similar to the iPhone 3GS in that it has an A8 CPU and PowerVR SGX530 (as opposed to the SGX535 in the 3GS). This fact made it easy to just reuse all of the carefully optimized assets we had worked on for the iOS port. There were some things like the PDK touch APIs that needed a bit of reworking but it was pretty straightforward. I considered the port to be 98% complete but there were 4 show stoppers and general performance issues that prevented me from releasing it to the webOS App Catalog at that time. I had suspicions that my issues might be resolved in a future version of the OS and so at that point I decided to wait for the next webOS update and then re-visit the port. Unfortunately I was already running the final version of webOS released for the Pre (1.4.5) and so I put the unfinished port on the shelf and went on to other things.

Throw Hardware At The Problem

Fast forward to a week ago. I was able to procure some newer hardware (a Pre2 and a TouchPad) that would allow me to test the Caster codebase on webOS 2.X and 3.X. I had hopes that the newer versions of webOS would resolve some of my show stoppers. Fortunately they did and I was finally able to finish up the port that I had started a year ago. Caster now runs really smoothly on the Pre2 thanks to the updated OS and a CPU clocked significantly higher than the original Pre. Caster also will run on the HP TouchPad in a manner similar to how it runs on the iPad. Performance on the TouchPad is somewhere in-between the iPad 1 and the iPad 2. Getting the year old port to work on the Pre2 was a piece of cake seeing as the hardware was very similar to the original Pre. The TouchPad had some interesting differences (outlined by Palm here) which actually made it more like developing for a desktop platform than a mobile one.

One More Down, Many More To Go

Another platform under the belt. About a day or so after submitting Caster to the HP App Catalog it was approved and went live. It was definitely a fun exercise to develop for webOS. One thing I loved about PDK development was that you have gdb directly on the device. That and the ability to ssh as root on stock hardware + OS made debugging a lot of things a snap. Over a year ago I also started a port of Caster to Android using the Android NDK but for various reasons it has not yet been released. One of these days it will get completed. Maybe not before the Dreamcast port though

(a year and a half ago I did actually get a Dreamcast build of Caster to compile, link, launch, execute and display a simple quad onscreen)