generating reports

people want to be able to get results from february-fire. Currently they ask me, and I run database queries and generate a csv file. I want to generate that file automatically from a web address.

My plan is this blah.heroku.com/csv?batch=batchName&begin=3-10-2013&end=3-17-2013

ok.. let's get a stand-in for this that returns "hello world".. done
next, make it echo the input arguments.. done

ok.. now let's run a query to get all the stuff I want..

hm.. the query would be easier if I had a single doneAt field, whereas there are currently two ways a record could be done: reviewAt or rejectedAt.. I think I'll add a common doneAt field, and convert everything else to have it two, which should just be a couple of queries.. oh wait.. I guess it won't quite be a couple queries, since I need to set a different value for each item.. drat..

hm.. I guess I could do it with an eval.. let's try that, and see how it goes..

but first I need to update the code so it adds this, so items from now-on will have it.. done

ok, so now let's update all the other items with eval.. did it locally, and it works.. here we go.. hm.. this query doesn't delete anything, but I think I'll take a backup anyway, because it's a pretty intense query..

seemed to work.. horray.. here's the query for reference:



db.runCommand({
    eval: function () {
        var count = 0
        var recs = db.records.find({reviewedAt:{$exists:true},
            doneAt:{$exists:false}})
        for (var i = 0; i < recs.length(); i++) {
            var rec = recs[i]
            db.records.update({_id:rec._id}, {$set:{doneAt:rec.reviewedAt}})
            count++
        }

        var recs = db.records.find({rejectedAt:{$exists:true},
            doneAt:{$exists:false}})
        for (var i = 0; i < recs.length(); i++) {
            var rec = recs[i]
            db.records.update({_id:rec._id}, {$set:{doneAt:rec.rejectedAt}})
            count++
        }
        return count
    },
    nolock: true
})


note: "length()" instead of "length"

ok.. now that we have the doneAt field, let's go back to working on the csv api call..

..ok, I got streaming working on the api call — apparently mongojs upgraded to support streaming between the time I started february-fire, and now.. what luck

..but I need to go to dance