work notes

ok, I was going to do 4 things,
1. eat
2. put on music
3. deal with people having reviewed their own work, so I don't have special cases for that in the payment code
4. I'd also like to shower and get ice cream.. ice cream for after payments are done I suppose, but I'll need to buy it before midnight..

and I had decided to shower, and then buy ice cream, and get Subway..

I showered, and bought ice cream, but Subway was closed (I'm guessing they close at 9pm), so I got Safeway frozen dinner.. now let's put on some music and eat, and then we'll be ready for number 3..

ok, good.. let's do this.

I feel like I'm performing surgery on data.. let's do it on our local copy, and make sure it's right.. we'll just write temporary code directly into pay_people.js, since this is just something I need to run once, and that code already has the infrastructure to access the database..

ok, what do I want to do.. I think I want to iterate through each record, and see if there are any cases of people reviewing their own word, and print those all out, so I can see how much there is to operate on..

..oops, lucky catch.. I was going to be overwriting my _output.txt file, so I looked at what was there, and there was data of stats I had calculated for different users, and there was a user with deservedCents = null, and it occurred to me that it could be null if the user didn't do any reviewing, since my calculation is: u.answerAcceptedCount * 28 + u.reviewAcceptCount * 4, which will be null if either u.answerAcceptedCount or u.reviewAcceptCount is null.. so I think I want: (u.answerAcceptedCount || 0) * 28 + (u.reviewAcceptCount || 0) * 4..

fortunately I don't think it would have screwed things up to much.. it would have not paid people what they deserved, but it wouldn't corrupt the database, meaning that once the error was fixed, everyone would get paid correctly..

..anyway, that's not what I was working on.. I'm listing instances of people reviewing their own work..

ok, there are 84 instances. I think they may all have only one thing in their history, since they happened pretty early on.. let's see.. yup. So that makes things a bit easier. The query we can use to get them all is.. well.. something like:

find records where record.answeredBy and (record.answeredBy == record.reviewedBy)..

but I'm not sure mongodb allows comparing record fields with other fields.. let's see..

hm.. apparently "yes", but, it involves sending javascript to the server, e.g., db.T.find( { $where: function() { return this.Grade1 > this.Grade2 } } );

I wonder if mongoHQ will let me do that.. I wonder if I can do that from the mongojs node.js driver..

answer to "can I do it from mongojs" is YES, but I need to send the function as a string, e.g., db.collection('records').find( { $where: '' + function() { return this.answeredBy && (this.answeredBy == this.reviewedBy) } }, p.set). note where I have: '' +. in front of function () {...}.

answer to "can I do it on mongoHQ" is also YES. sweet! :)

ok.. so my update will be something like:


db.records.update({
$where: function() { return this.answeredBy && (this.answeredBy == this.reviewedBy) }

...

hm.. the update part would also need javascript, but I don't see a thing for running javascript there..

so I guess we'll have to do it with code..

(put on another album: Sufjan Stevens)

..hm.. all but one of the instances of a reviewer reviewing their own work is one person..

hm.. this person actually modified their answer during some of the reviews.. I do want someone else to review their work, but I suppose I want to review their updated version, so I'll need to copy it out of the history.. wait, no I don't. The history has the old version, so that's good.

ok, it seems to be working.
let's try running it on the real database.. crossing fingers..
seems to work.. good.. now let's remove the little "temp" flag I put there so I could query for the items I changed to make sure they were good..
done

let's put the code that did that somewhere for reference.. how about here:


db.collection('records').find( { $where: '' + function() { return this.answeredBy && (this.answeredBy == this.reviewedBy) } }, p.set)
_.each(p.get(), function (doc) {
db.collection('records').update({ _id : doc._id }, {
$set : {
// work here
temp : true,

availableToReviewAt : 0,
'history.0.answeredBy' : doc.answeredBy,
'history.0.answeredAt' : doc.answeredAt,
'history.0.reviewedBy' : doc.reviewedBy,
'history.0.reviewedAt' : doc.reviewedAt,
'history.0.reviewAccept' : true
},
$unset : {
reviewedAt : null,
reviewedBy : null
}
}, p.set)
console.log("done: " + doc.reviewedBy)
p.get()
})

Ok, now it is on to the payment code again..
ok, it seems to be working.. let's strip out the scaffolding, and run it on heroku, but make it artificially throw an exception after making the first payment, saying who was paid.. and let's check that that payment went through..

ok.. ready to press go.. hm.. let's throw the error before the payment is made, and make sure it works, and see who is going to be paid, so I can find them on oDesk and see how much they've already been paid for comparison purposes..

hm.. it's not as easy as I thought to guarantee that I'll be looking at the same person as the "first to get paid" on the second run.. let's look through the teams and see if anyone has been paid.. if they're all zero, that will make things easier..

ahh, how about I'll filter for a particular person to pay..

I ran: heroku run node pay_people.js

it's running...
hm.. it threw an exception, as expected, sortof, except that it's complaining: Cannot find module '/app/pay_people.js'... but it was running for a while before showing that.. so I suspect it may actually be an error trying to display the error.. let's see if anything was logged or changed in the database..

oh.

I forgot to upload my code.

let's do that..

ok, it ran.. but it ran entirely too fast. Something did not go right. Let's see what happened..

well, the console.log did show up in my local console using heroku run, so that's convenient. We'll just add back some console.logs..

(need more music.. Tilly and the Wall)

..oh, I need to remember to create a fake version of february-fire and do a full-on 5000 person blitz attack against it, to see how many dynos I need..

ahh, I see my mistake. I was returning from my _.each loop for all non-matching users, but I was returning "false", which is a special return value to break out of the _.each loop, so it was entering the loop, not finding the user it wanted, returning false, and not checking any other users..

oh yes, I need to first login to the running heroku app, so it can get my accessToken..

ok, it did it.. things to check..
did the person get paid? hm.. it doesn't look like it.. at least, I don't see it in the oDesk interface..
did I get a "payment receipt"? yes.. hm..

ugg.. this person has two contracts in two different team rooms.. I must have paid the other one, but that one already has been paid a fair amount, so I'm not sure if my payment was included.. is there a way to view payment history?.. hm.. I can't find one in the UI.. maybe there's an API call for it..

hm.. I ran this API call: https://www.odesk.com/gds/finreports/v2/buyer_teams/TEAM_REF/billings?tq=SELECT%20amount%20WHERE%20provider__reference%20=%20'PROVIDER_REF'&tqx=out:json

and it showed a bunch of data, and those bits of data added up to the amount of money that the UI says this person has been paid..

but none of those bits corresponds to my payment (they are each for an amount, and my amount isn't among them)..

hm.. I'm guessing the payment failed.. how how? the API call succeeded.. where did the money go?

AHA! It did NOT fail :) -- what I was looking at before was "billings", which shows how much money  we lost, but that's 10% more than the amount of money they gained, which there's another API call for with the word "earnings", and when I do that, I do see my payment.

so phew..

hm.. it occurrs to me that before I can run this, I need to confirm whether the 28 cents and 4 cents rewards are before or after oDesk's 10% cut. I assumed they were after when I paid this person, so I may have overpaid them..

I did send an e-mail asking this earlier.. let's see if they got back to me..

I haven't heard back. I don't think I can go further with payment before know, so I'll wait till tomorrow. It's just about done though, so I suppose ice cream is still in order..