IronRuby vs JRuby vs MRI. Performance Mini-Shootout.

27 Aug 2008

One of the things I went over in my eRubyCon talk was the relative performance of IronRuby compared to both JRuby (which I consider to be a cousin of sorts to IronRuby) and MRI. I did this by running the same tests that Antonio Cangiano ran in December.

There were a few test results that really stood out. The lists test was by far the worst for IR, where it was 56 times as slow as MRI and 3 times as slow as JRuby. But this wasn’t the only sore spot, there were other places that show room for improvement. Namely, vm1_blocks in which IronRuby was almost 9 times as slow as MRI. Also so_exception where IronRuby brought up the tail by being 10 times as slow as MRI.

The nitty gritty is that IronRuby is about twice as slow as MRI with JRuby being slightly faster, overall. As the below graph illustrates:

There are a few high points in this test though. Number one, is that IronRuby ran the majority of the tests. There were only four failures out of forty, for a 90% success rate. For comparison JRuby had 95% success rate. Number two, there were actually a few tests in which IronRuby beat JRuby and MRI. No small feat for such a young implementation.

I think this mini-shootout illustrates that there is a lot of room for improvement for IronRuby. However, a little perspective goes a long way. JRuby was announced in 2001, IronRuby was announced last April. The JRuby team has had 7 years to get to where they are today, whereas IronRuby has only had almost a year and a half! As the DLR improves IronRuby will receive those performance gains for free, and as compatibility becomes more stable the core team can focus on performance as well. The raw data can be downloaded here.