node.js + mongodb + heroku

so.. I needed to make something reliable and scalable, and my favorite single-machine single-process in-memory-database solution used in talent court — which has been running for months without apparent issues — seemed dangerous for this application, for several reasons:

(1) the ec2 machine might crash, bringing down the site completely. I didn't want to stay awake fearing this. even though I've never seen it happen.

(2) the app could crash, losing data since it last serialized the in-memory database. possible data loss for this application seemed too dangerous, because its keeping track of how much to pay people.

(3) the app might not scale enough, and a single-process in-memory-database has a pretty hard scaling cap.

so I replaced single-machine single-process with heroku, and in-memory-database with mongodb.

I am quite pleased with both.

problems/solutions:

login — oDesk login, using passport + passport-oauth + node-odesk

keeping keys secret, even while project is public in github — environment variables, with a list of them in the readme

uploading data — I had an api in the app itself, but it is easier to connect to the database directly

nested callbacks — node fibers, with "promise" abstraction (not sure where I stole that abstraction from, but I didn't think of it)

rest api — I noticed json rpc, for which there is a node.js module, but it is too non-simple for me, so I boild it down to this, which takes input like this { add : function (arg) { return arg.a + arg.b } }, and is called like this: example.com/rpc?{"func":"add","arg":{"a":1,"b":2}}

exception handling — catch-all exception handler here at the top, and express turns exceptions into HTTP 500's, which invoke a catch-all exception handler (onError) on the client, which just tells the user essentially "sorry, I made a mistake, please try refreshing the page"

version updating — because the client is just a single static webpage and everything is done with ajax, I needed a way to tell the client to refresh, in case the ajax api changed. This is done by having every request ask for the server version, and if the server version doesn't match the client version, the client pops up a "version upgrade, please refresh the page". This is not super graceful since it waits until someone tries making a possibly outdated ajax request before telling them to upgrade, alas..