Parsing the SXSW Twitter Stream for Fun

Over the weekend I decided to toss together a simple twitter stream monitoring app that would capture SXSW tweets. I wanted to build on some of what I learned while hacking together the stuff for code2009 and it was also an excuse to play with node.js and a few other things. I figured I would put together a quick post with all the different parts and as I have time I’ll pull together some of the more important sections into larger posts. Be warned that the site may stop functioning since it was only a few hours work and mostly put together with bailing wire and gum.

Server side:

  • node.js - Used to process the long polling requests
  • ngnix - Front for multiple nodejs instances and serving some static files
  • underscore.js
  • rabbitmq - Used to monitor the realtime feed
  • Redis - Used as a datastore for everything
  • ruby - Used to glue a bunch of things together, tons of gems used
  • Resque - Used to handle the screen captures

Browser side:

Hosting: