Ruby Findings

I was really impressed by ActiveRecord’s ability to abstract an underlying database schema into a full-featured set of class based access methods. Defining relationships, decorating them with speficic methods, etc and so on. There is one aspect of ActiveRecord, however, that I’m really learning to dislike: database concurrancy.

So, first of all, if you already know that ActiveRecord is not thread safe by default, then you can probably move on to reading something else. You’ve been through the frustration I’ve been going through and you’ve either coded your way around ActiveRecord’s blasie attitude towards threads or you gave up trying to use it in a threaded application. If, however, you haven’t see the pain and carnage that ActiveRecord can introduce into your nice, thread safe projects then read on. I may be able to save you some pain.

ActiveRecord is not, by default, thread safe. I haven’t hacked into the code to figure out why it isn’t but through trial and error and reading pages on the Intertubes, it’s clear that ActiveRecord is thread challenged. There is a very cheesy way to attempt to ensure that ActiveRecord won’t immediately barf in a threaded environment: ActiveRecord::Base.allow_concurrency.

Oh yes, you think when you first see this, this will solve my problems. Well, sort of. If you put

ActiveRecord::Base.allow_concurrency = true

in your code ActiveRecord will change the way it talks to its backend database. In the Mysql world, this means that ActiveRecord will open a new database connection thread each time it needs to query the database - I think. I have to be honest I’m not sure this is exactly accurate: it may only open a new thread when it needs to execute a query that it hasn’t already executed. Either way, even a trivial script that just imports data into models can easily open 100+ threads to the database. Which will make your database quite unhappy and result in anger on your part.

My understanding is that in addition to creating a mega amount of database connections, ActiveRecord will also not release its database resources until the process in which ActiveRecord is running dies. If your script will only create a few database calls, ActiveRecord’s concurrency option is probably satisfactory for the job (although, if your script is that light-weight, why would you need threads anyway) however, for long term processes like daemons, this really a show stopper.

Even better - the Mysql driver has a lock that engages when the database connection is waiting for a statement result set to return from the database. This results in a lock on the Ruby interpreter which results in your entire script grinding to a halt while waiting for Mysql to strut its stuff.

If you are part of the Ruby 1.9 crowd, you can take a look at the brand-spank’in-new git project espace/mysqlplus that offers “An enhanced mysql driver with an async interface and threaded access support”.

But I’m stuck on 1.8.7 so I’m going to spend some time over the next few days looking for a reusable and sustainable solution that will permit my scripts to use threads to divide work and be able to use ActiveRecord at the same time.