Quick Hits: Setting the User Agent Header in Webrat

If you’ve read the new PragProg beta e-book on RSpec, you may have read that you can set HTTP headers for your Webrat request like so:

Given /^I am browsing the site using Safari$/ do
  header "User-Agent" , "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us)"
end

Like me, you may have found out the hard way that this doesn’t work. Webrat does not automagically apply these new HTTP headers to your request – they certainly don’t make it to my controller. What worked for me:

Given /^I am browsing the site using Safari$/ do
  headers["User-Agent"] = "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us)"
end
 
When /^I visit my precious site$/ do
  get '/my/precious/path', my_query_string, headers
end

In the code above, headers is a method call that returns all the HTTP headers for your request. Just tack headers on as the third argument of your request, and you’re good to go.

Playing with JRuby, Part 1A

First, thanks to Lopex, Charles/Headius, and Thomas for their well-coordinated effort to alleviate some of my JRuby ignorance in my earlier post. And for not going out of their way to make fun of my janky “benchmarks” – like I said, those weren’t meant to be real benchmarks, just something to give me an idea of relative performance with some operations I care about.

In that same spirit, I took Lopex’s advice and turned on the –server flag; performance gain went from almost 2-to-1 to almost 3-to-1, with no other changes or optimizations.

Before:

[~/project/jrubytest/primes]> ruby ./primes.rb
FINDING first 100000 PRIMES
1299709
Time: 57.716853s
[~/project/jrubytest/primes]> jruby ./primes.rb
FINDING first 100000 PRIMES
1299709
Time: 33.885273s

After:

[~/project/jrubytest/primes]> ruby primes.rb
FINDING first 100000 PRIMES
1299709
Time: 57.204178s
[~/project/jrubytest/primes]> jruby --server primes.rb
FINDING first 100000 PRIMES
1299709
Time: 21.067204s

Thanks to the JRuby crew for stopping by here, and for being so helpful generally.

Playing with JRuby, Part 1

As part of a new project, I’m experimenting with JRuby, to see (a) if the alleged performance gains are all that and (b) see how far the Java integration – especially interface implementation – can be pushed.

On the performance front, I did a quick-and-dirty test with a CPU-expensive script that finds prime numbers – not a real benchmark, but something to give me an idea of how JRuby held up under a steady load of basic math operations. Details follow.

Read the rest of this entry »

Hadoop Streaming for Rapid Prototyping of Distributed Algorithms

Note: This article assumes that you know a little about MapReduce, or that if you don’t, you might skim the enclosed links so you know what I’m talking about when I get to the examples, or check out the Hadoop Tutorial. It also assumes that you have Hadoop set up – either clustered or pseudo-clustered – if you’re going to run the examples. Or you can just read along.

Hadoop is a framework (written in Java) that supports distributed computing – specifically Google’s MapReduce algorithm. It also comprises HDFS (the Hadoop Distrubuted File System), which allows you to redundantly store large quantities of data across multiple disconnected disks as if they were a single storage unit. I’ve used Hadoop at two jobs and at home, and it rocks.

It comes with a problem, though, which you may spot in the first paragraph: It’s written in Java. Now, don’t get me wrong – Java’s a great language. But developing software in Java with the most commonly used tools (Eclipse, Ant/Maven, &c) is a monumental pain (and you’re hearing this from an old Visual C++ hand). I’m the first to admit that I should shore up my skills with the Java tools, but even if I were better at it, getting a non-trivial Java project from zero to first runnable build is still about as complex as the invasion of Normandy, and the proliferation of XML config files is just inhumane.

Still, the performance and solidity of Java make it the right choice for a production Hadoop project. But what if you just want to kick around an idea or test an algorithm? Wouldn’t it be nice if you could do that in 2 hours instead of a day and a half? Wouldn’t it be nicer still if you could do it in your language of choice?

Read the rest of this entry »

Quick and Dirty Messaging

Our Rails Rumble 2008 entry, Great Minds (you can have a look at the latest version or the original Rumble version), required a messaging system. Such systems are easy to do wrong, and we knew we’d need something that would stay solid under unknown load during Rumble judging.

The solution was quick & dirty (as most solutions are during Rails Rumble), but the results worked, and allowed us to qualify for judging and reach a respectable 28th place finish in the “Completeness” category.

Check out the details after the fold.

Read the rest of this entry »