Quick and Dirty Messaging

Posted: November 22nd, 2008 | Author: Brad | Filed under: Great Minds, Javascript, Programming, Rails | Tags: gm, Great Minds, Javascript, message queue, messaging, Programming, Rails, rails rumble, ruby, ruby on rails, rumble | 1 Comment »

Our Rails Rumble 2008 entry, Great Minds (you can have a look at the latest version or the original Rumble version), required a messaging system. Such systems are easy to do wrong, and we knew we’d need something that would stay solid under unknown load during Rumble judging.

The solution was quick & dirty (as most solutions are during Rails Rumble), but the results worked, and allowed us to qualify for judging and reach a respectable 28th place finish in the “Completeness” category.

Check out the details after the fold.

Design Constraints

Great Minds is a word game played by 2 or more players. The game is played in a virtual chat Room, containing one or more Players. Players can take various actions, such as submitting a word to the game, chatting, or changing their handle/username. The results of these actions must be transmitted to other players in the room.

Other constraints on the design included:

• No New Technologies: There are tools like Juggernaut that make messaging pretty easy, but we had no hands-on experience with them, and learning a new tool on a 48-hour schedule was deemed a risk to completing the project & qualifying for Rumble judging.
• Broadcast: We needed something more than a simple message queue like Beanstalk; the consumer of a message couldn’t remove it from the queue, as there might be other consumers needing the same message.
• Room State Reconstruction: Someone coming to the room for the first time (or leaving and returning) should be able to see the current room state, including all participants, recently played words, and recent chat.
• Replay: Though we didn’t get around to implementing it, we wanted to offer the ability to review a particularly funny game.

The first two constraints affected our design and technology choices; the second two demanded that the messages be persisted. (Alternatively, we could have saved the room state in some other way, but as long as we had the messages anyway, it seemed simpler to just keep them and reconstruct the state.)

Client Implementation

In the absence of a viable push solution like Juggernaut, we opted for a polling solution: Clients would periodically send a signal that:

• let the server know they’re still alive and connected,
• sent any new messages to the server, and
• requested any pending messages from the server.

On the client side, this meant that every message got packaged as JSON, stuck in a queue, and periodically collected and sent to the server as the optional payload of a keep-alive signal. With some help from Prototype, this was as simple as:

  function SyncMessages() {
          var data = Object.toJSON(MESSAGES_QUEUE);
          var url = '/room/sync_messages';
          var params = {
                  authenticity_token: AUTH_TOKEN,
                  messages: '[]',
                  //messages: Object.toJSON(MESSAGES_QUEUE),
                  last_message_id: LAST_MESSAGE_ID };

          new Ajax.Request(url, {
            method: 'post',
                  parameters: params,
            onSuccess: HandleMessages });
  }

  SyncMessages(); //start first on load
  var message_syncer = new PeriodicalExecuter(SyncMessages, 3);

HandleMessages() updates the DOM based on messages coming back.

The other feature to note is the LAST_MESSAGE_ID token above. Each client is responsible for keeping track of the last message ID it got. On the server side, message IDs are assigned sequentially, so all new messages are guaranteed to have a higher ID.

We could have kept track of each client’s last message on the server side, but this would have been much more complex – we would have needed to store data either in the session (which already held a fair amount of stuff) or in another table in order to keep track of who had received what.

Server Implementation

On the server side, messages were persisted in a table. This table would have relatively frequent inserts, somewhat more frequent reads, and no updates or deletions. For this reason, we went with the default MyISAM table architecture. (We had considered a design that would have updated the table with delivery status information and would have required InnoDB’s row-level locking, but in the end we opted for the simpler design.) The table looks like:

  class CreateMessages < ActiveRecord::Migration
    def self.up
      create_table :messages do |t|
        t.integer :message_type_id, :room_id, :game_id, :user_id
        t.string  :data, :user_handle
        t.timestamps
      end
    end

    def self.down
      drop_table :messages
    end
  end

It also has an index:

  class AddRoomIndexToMessages < ActiveRecord::Migration
    def self.up
      add_index :messages, [:room_id, :id]
    end

    def self.down
      remove_index :messages, [:room_id, :id]
    end
  end

Recall that the client keeps track of the last message it got, and periodically requests all messages since. This index speeds that request - which is just msgs = Message.find :all, :conditions => ['room_id = ? AND id > ?', session['room_id'], last_message_id], rder => 'id'. (Yes, I know that should be a named scope. I hadn't gotten around to learning those yet at the time of the Rumble.)

When the a keep-alive signal hits the server:

• any new messages attached to the signal are added to the messages table,
• any game mechanics related to those messages are executed (which may result in more messages being added to the table), and
• all messages for the caller's room with an ID greater than the caller's LAST_MESSAGE_ID are packaged as JSON and sent back down in the HTTP response.

A note: If you're in a room and you enter (for example) a chat message, there's actually a server round-trip involved (pushing the message to the server and getting the same message back as a response) before that chat message is shown in your browser. We worried about this, but not for long - the interactivity delay turned out to be minimal, and it was much simpler to keep it this way (every message is treated the same for all clients) than to create an exception (you update your DOM with your chat messages immediately, but your own chat messages are somehow filtered from your message stream).

What's Missing?

Obviously, this was a quick & dirty solution, tailored to the immediate needs of a Rails Rumble project. Here's what I'd do differently (and indeed, will do differently, as we have plans for the future of Great Minds - stay tuned):

• Security: We do use the Rails session authenticity_token, but there's still no protection from a server flood from someone who has a token, either by cranking down the polling interval or by fiddling with the LAST_MESSAGE_ID.
• Server Push: Polling is wasteful - requests are made whether or not there are any messages to send or receive. We plan to move to Juggernaut soon.
• Decoupling from Game Mechanics: In the first iteration, message receipt and delivery were too intimately tied to game mechanics, with Message objects being explictly constructed and followed by Message.save! calls. I've already started the process of teasing them apart in preparation for the move to Juggernaut, factoring this into a deliver_message() method.

Questions? Thoughts? Add a comment! I promise to get back to you.

---