Speeding up RubyMotion compile times by 95%



UPDATE 5/29/2015: RubyMotion 3.12 came out with much faster compile times. Go check out my follow-up blog post.

Original post:

Crash.

$ rake clean:all.

Wait.

$ rake pod:install.

Wait.

$ rake

Wait. And wait some more.

Crash.

It's 10:30 PM. I'm sitting at my kitchen table. My four kids and Chy, my wife, are all already in bed. And I need to get this app ready for submittal.

desk flip

And I am pissed.

The error message is incomprehensible. I've been doing RubyMotion iOS app development for three years, and I've never seen this. I'm trying different things...downgrading RubyMotion. Trying a different iOS SDK version. Commenting out lines of code.

And each time, it's taking eight minutes to clean the cached files and recompile. I can't do this.

Inspiration

As I unenthusiastically watch the Compile /Users/jh/... lines fill my screen, I observe that they seem to be scrolling by at a somewhat constant rate. Large files, small files. All at about the same amount of time.

I pull up Slack. The only one online is Gant Laborde. When I message him, he's about ready to go to bed, but humors me.

I had an idea

Do you think concatenating all RubyMotion files into one file before compiling would make it compile way faster?

Gant thinks it's a bit crazy. "Have you tried it?"

Nope. He goes to bed, after I promise to tell him my results.

Concatenation

I build a small script that concatenates ProMotion files into a single large file, and then points the compiler to that concatenated file. At first, it crashes right away, because I don't have the load order correct. Once I rearrange things a bit, it builds and displays the app.

There's a noticable pause on the generated, concatenated ProMotion-concat.rb file. I run time rake on both the non-concatenated version and the concatenated version.

I can't believe my eyes. It builds in about 500 seconds with the regular version, and in about 25 seconds with the concatenated version.

I run the test suite. It passes with both, but 784 seconds reduces to 266 seconds. Most of the 266 seconds is actually in the test suite run (about 200 seconds), so we're talking a 90-95% speed improvement. Just by concatenating the project files into one large file.

Considerations

It's an awesome discovery, but there are a few things to work through in the coming weeks.

This only improves full, clean installs. Incremental compiles of one or two changed files will still be faster the normal way. So for normal apps where you're adjusting one or two files and recompiling without running a rake clean:all, it may not bring much benefit.

I also need to solve the dependency issue in an automatic way. Having to manually specify file load order isn't a long term solution.

It also obfuscates your backtraces. It'll say the crash happened at line 4952 of "ProMotion-concat.rb", which isn't as useful.

Gem Authors

For gem authors, this is an opportunity to help your users out in a big way. The app I was working on last night included Sugarcube and MotionKit as well as ProMotion. When you add in the app files, we are compiling around 320 files total. If ProMotion, MotionKit, and Sugarcube were one concatenated file apiece, we'd be down to 126 files, reducing the compile time significantly without compromising the debug backtraces for the app.

Next

In the coming weeks, I'll be experimenting with this technique and trying it in real apps. I'll see if there's opportunity for a "speed compile" gem. At the very least, I'll be providing an optional (or default?) concatentated version of ProMotion, referenced something like this:

gem "ProMotion", require: "ProMotion-concat"  

If I can save app developers several hundred seconds of compile time by doing this, it's a no brainer.

And maybe, just maybe, we won't groan when we have to rake clean:all.