Ruby Findings

I was really impressed by ActiveRecord’s ability to abstract an underlying database schema into a full-featured set of class based access methods. Defining relationships, decorating them with speficic methods, etc and so on. There is one aspect of ActiveRecord, however, that I’m really learning to dislike: database concurrancy.

So, first of all, if you already know that ActiveRecord is not thread safe by default, then you can probably move on to reading something else. You’ve been through the frustration I’ve been going through and you’ve either coded your way around ActiveRecord’s blasie attitude towards threads or you gave up trying to use it in a threaded application. If, however, you haven’t see the pain and carnage that ActiveRecord can introduce into your nice, thread safe projects then read on. I may be able to save you some pain.

ActiveRecord is not, by default, thread safe. I haven’t hacked into the code to figure out why it isn’t but through trial and error and reading pages on the Intertubes, it’s clear that ActiveRecord is thread challenged. There is a very cheesy way to attempt to ensure that ActiveRecord won’t immediately barf in a threaded environment: ActiveRecord::Base.allow_concurrency.

Oh yes, you think when you first see this, this will solve my problems. Well, sort of. If you put

ActiveRecord::Base.allow_concurrency = true

in your code ActiveRecord will change the way it talks to its backend database. In the Mysql world, this means that ActiveRecord will open a new database connection thread each time it needs to query the database - I think. I have to be honest I’m not sure this is exactly accurate: it may only open a new thread when it needs to execute a query that it hasn’t already executed. Either way, even a trivial script that just imports data into models can easily open 100+ threads to the database. Which will make your database quite unhappy and result in anger on your part.

My understanding is that in addition to creating a mega amount of database connections, ActiveRecord will also not release its database resources until the process in which ActiveRecord is running dies. If your script will only create a few database calls, ActiveRecord’s concurrency option is probably satisfactory for the job (although, if your script is that light-weight, why would you need threads anyway) however, for long term processes like daemons, this really a show stopper.

Even better - the Mysql driver has a lock that engages when the database connection is waiting for a statement result set to return from the database. This results in a lock on the Ruby interpreter which results in your entire script grinding to a halt while waiting for Mysql to strut its stuff.

If you are part of the Ruby 1.9 crowd, you can take a look at the brand-spank’in-new git project espace/mysqlplus that offers “An enhanced mysql driver with an async interface and threaded access support”.

But I’m stuck on 1.8.7 so I’m going to spend some time over the next few days looking for a reusable and sustainable solution that will permit my scripts to use threads to divide work and be able to use ActiveRecord at the same time.

Time to upgrade your ruby installs. This advisory came out yesterday regarding 5 CVE’s impacting every current ruby release.

Officially impacted versions: 1.8 series

• 1.8.4 and all prior versions
• 1.8.5-p230 and all prior versions
• 1.8.6-p229 and all prior versions
• 1.8.7-p21 and all prior versions

1.9 series

• 1.9.0-1 and all prior versions

Read the advisory for remediation information. Matasano also has a great writeup on the advisory here. Wonder what it would take to blow out the rails params array with this (I’ll leave that to the professionals, I’d rather just patch it and move on right now). Scary stuff!

Update Per comments on the Ruby on Rails blog post you will break your rails install if you upgrade to 1.8.6-p230. And 1.8.7 is only compatible with Rails 2.1. You might want to test out the latest ruby with your app on a dev site before blindly upgrading your production ruby install.

If you want to put a little laugh back into the day, read Zed Shaw’s rant. And yes, there is actually some good info there, it’s worth a read.

I work on a number of projects, some in ruby, some in “other” languages. Time and time again I find myself working on a piece of code in those “other” projects and thinking “man I wish this was in ruby”. The most recent one was with arrays in PHP, so I figured I’d write a little piece on why it made me sad. Sometimes, the simple things in a language are what make it pleasing to work in.

I’m feeling kind of dumb tonight. I must have read the little (and I do mean little) section on the Regexp class in my Ruby in a Nutshell book a billion times now (if that doesn’t say stupid, I dunno what does) and somehow every time I’ve missed this little beauty.

I needed to write a reporting script the other day that dealt with time durations. The script basically processed a series of log files determining the ‘start’ and ‘end’ times of certain events. Part of the report output needed to be the duration of each event. Calculating this using DateTime objects should have been as easy as this:

def duration
  @end_datetime - @start_datetime
end

Ack, Instead of providing me with something intuitive, I got an object back of class Rational. A fraction of all things:

start = DateTime.parse('2/1/2008 10:35:00')
finish = DateTime.parse('2/1/2008 21:15:00')
duration = finish - start   # => Rational(4, 9)

Well, what the heck do I do with a fraction?

It turns out that Date has a class method called day_fraction_to_time() that takes a Rational and returns the number of hours, minutes, seconds and factions of a section. The beast in action:

start = DateTime.parse('2/1/2008 10:35:00')
finish = DateTime.parse('2/1/2008 21:15:00')
duration = finish - start
Date.day_fraction_to_time(duration)   => [ 10, 40, 0, Rational(0, 1) ]

Joy, this works, I now have my duration in hours minutes and seconds. The only conclusion I can draw regarding using Rational is that it can provide more accuracy than a Double or Float. In other words, it’s a more accurate representation of the percentage of a day since Rational( 1, 3 ) will not lose precision whereas ‘.3333333333’ is only as precise as the number of bytes a floating point number is stored as.

I was a little bored tonight and thought I’d respond to a post over Happy Python with some code. Thanks for the inspiration Matt!

ruby>> require 'ieeemac'
# => true  
ruby>> macs = Mac.find_macs `ifconfig -a`
# => [#<Mac:0x1357f40 @mac=["00", "16", "cb", "95", "13", "50"]>, #<Mac:0x1357f2c @mac=["00", "16", "cb", "ff", "fe", "5d"]>, #<Mac:0x1357d74 @mac=["00", "17", "f2", "41", "ec", "ec"]>, #<Mac:0x1357c20 @mac=["00", "50", "56", "c0", "00", "08"]>, #<Mac:0x1357ae0 @mac=["00", "50", "56", "c0", "00", "01"]>, #<Mac:0x135798c @mac=["00", "1c", "42", "00", "00", "00"]>, #<Mac:0x13577e8 @mac=["00", "1c", "42", "00", "00", "01"]>]  
ruby>> macs[0].to_format :cisco
# => "0016.cb95.1350"  
ruby>> macs[0].to_cisco
# => "0016.cb95.1350"  

Read the rest of this post for the full class.

"ActsAsSecure adds an ability to store ActiveRecord model's fields encrypted in a DB. When a model is marked with acts_as_secure, the :binary type fields are recognized as needed to be stored encrypted. The plugin does before_save/after_save/after_find encryption/decryption thus making it transparent for a code using secured models.

The plugin supports a master key approach as well as individual records encryption keys. It does not contain any crypto provider but allows to plug in any external one as long as it supports encrypt/decrypt methods."

http://revolutiononrails.blogspot.com/2007/04/plugin-release-actsassecure.html

The site has some other goodies too.

Sometimes you run across neat and interesting libraries when you are reading up on something completely different. Case in point, via another blog, I stumbled on Reliable Messaging with Ruby. This looks to solve a problem I haven’t yet run into, but I know I’ll find a use for it in the future (maybe for a game I’m slowly working on, or a work project). As of today, there haven’t been any commits to the project for two years, but it’s entirely possible there are no bugs (except for the one I found below) and the author considers it feature complete.

Current feature list:

• Simple API.
• Transction processing.
• Disk-based and MySQL message stores.
• Best effort, repeated and once-only delivery semantics.
• Priority queues, message expiration, dead-letter queue.
• Message selectors.
• Local and remote queue managers using DRb.

Version 2 of Net::SSH and Net::SCP are on their way.

http://weblog.jamisbuck.org/2007/7/29/net-ssh-revisited

I’ve read a bunch of conflicting advice on how to use active record models defined in a Rails application from outside Rails.

The following is the result of experimenting with others’ advice plus my own digging around:

require 'rubygems'
require 'active_record'
require '/path/to/rails/config/environment'

This should allow you to use your Rails models just like you would in a Rails controller:

#!/usr/local/bin/ruby

require 'rubygems'
require 'active_record'
require '/var/www/railsapp/config/environment'

m = MyModel.new
m.find_all.each { |row|
  puts row.some_column
}