Saturday, March 31, 2012

STOMP with Rails

I am looking for some message-oriented middleware to make the ETL between the operational Casebook database and the data warehouse DB (with star schema) real-time (in fact, Pentaho uses JMS for real-time ETL, but it comes with the enterprise edition only). This week, I experimented with the Apache ActiveMQ that speaks STOMP, among other protocols (ActiveMQ does not yet support AMQP, though). I checked this tutorial on the ActiveMessaging plugin, and also this tutorial from the Rails Recipe book. Finally, I decided to write a PoC on my own. ActiveMessaging brings in the idea of a Processor class, which is basically a subscriber that listens to a message queue. In my demo app, the user can create orders through the front-end, which gets saved to the database through the usual controller-model handshake, with an initial status 'NEW'. However, the controller also writes the data about the newly created order to the message broker. The OrderProcessor, which listens to the same message broker, parses the XML-formatted data using the REXML api, finds the order through the id, and updates the status to 'SHIPPED'. Note that, although the controller and the processor have a client-server (or publisher-subscriber) relationship between themselves, they are both clients to the ActiveMQ server, which, in this case, speaks STOMP.

I will look next how the "publish" part to the message broker can be triggered from the operational database, as inserts, updates and deletes take place there.

No comments:

Post a Comment