fixing pima

pima is taking a while to save, like 4 seconds... I patched this earlier by making it save less often, but now I have time to track down the problem. I suspect there is just a super large card in there, probably from a super long e-mail..

how to find it.. I used to have an "eval" feature in pima. I may need to re-add that. Hm.. I should be humanscripting this.. I've lived with the long saving till now..

hm.. before that, I have some merging to do.. my current version is not in sync with github.. done

ok, let's humanscript this:

hm.. I need to see how it will run for someone starting from scratch, with no data..
let's try cloning the repo in a new place..
ok, I added some instructions to the readme..

let's try the humanscript again:

- clone https://github.com/dglittle/PIMA
- put these instructions in humanscript/hs_748569937/hs.txt
- see README to run it
- add "eval" button next to "save" button
- pressing "eval" button should:
    var div = new jquery div
    var top = new jquery div
    top.append("run" button)
    top.append("close" button)
    var input = textarea with 100% width and height
    var output = textarea with 100% width and height
    div.append(_.splitVert(1, top, _.splitHorz(50, input, output)))
    "close" button should call _.closeDialog()
    "run" should eval the code in input,
        where any calls to console.log will print out to the output textarea
    _.dialog(div)
- test it
    run it,
    press "eval"
    type: console.log("hi")
    press "run"
    should see "hi" in output
    take screenshot
    put in humanscript/hs_748569937
- check in
- make pull request to original repo

Ok, I want to post this, but my current system is too cumbersome. I'm supposed to paste this into a json file, and run some command line on it.. I want an interface running at some url where I can just paste my humanscript into a textbox, press "post", and later press "hire", and later press "pay/close".. how is this different from the oDesk interface itself? Well, eventually I want the system to automatically hire people.. before it does that, the main benefit will just be a convenient way to track my jobs separately from the other people who post jobs into the same teamroom as me.. and allow me to add little efficiency gains, like setting the defaults the way I want..

it sounds like I want another heroku/node.js/mongodb app.. it will take a day to build this.. I probably can't do it until my non-meeting days, which will start Friday or Saturday..

so in the meantime, it sounds like I need to either solve my slow-saving pima issue myself, or wait..

or.. I could post it on oDesk manually.. though then I would need to track it manually..

or.. I could have my assistant post it on oDesk for me.. let's do that.. e-mail sent.

work notes

HumanScript

I've been manually keeping track of my sleep patterns in this google doc, but as I mentioned before, I want an icon on my phone that will automatically update this spreadsheet.

And I've decided to HumanScript this... here is the script (which has been posted on oDesk):

- create public repo on github (with a default README file)
- put these instructions into the repo in humanscript/AA577/hs.txt
- create two simple icons (they can look crappy)
    - one icon represents "awake" (like a face with eyes open)
    - one icon represents "asleep" (like a face with eyes closed)
- create an android widget that behaves as follows
    - it allows the user to add a widget to their main app area
    - at first it shows the awake icon
    - when the user clicks the icon, it changes to the asleep icon
    - when the user clicks the icon again, it changes back
- test the widget
    - install the widget
    - take a screenshot showing the awake icon
    - click it
    - take a screenshot showing the asleep icon
    - add these screenshots to the repo in humanscript/AA577/
- create simple instructions for installing the app
- put these instructions in the README file
- send me a link to the github repo

- in your cover-letter, please summarize what I want.

Thanks,
Greg

Note that this is just a starting point. It creates a widget that I can click. It doesn't actually send any notification to a server when I click it. I plan to add that in a second HumanScript.

Machine Learning

I was getting stuff settled on an ec2 machine that I thought would be the final destination for my classifier, but things have changed a bit, and I don't think that ec2 machine will be the final resting place for my code. So I've brought my code back to my own machine, which will make hacking on it a bit easier anyway.

My goal at this point is to create an ROC curve, because I want to see how much we can reasonably expect to gain from this classifier — that is, we only want to trust things that the classifier is darn sure about, and give everything else to someone to look at. So I've trained on some data, and I ran liblinear's predict on some test data.. and I have a file with linear regression probabilities for each test item.. there is probably some utility that will convert this file into an ROC curve for me. Let's see.. well, my brief search didn't turn up anything.

Hm.. question: the file looks like this:

labels 1 0
0 0.0110771 0.988923
1 0.985343 0.0146575
0 0.0264095 0.973591

The 0 or 1 in the first column is the label, but is it the predicted label, or the real label? How could it possibly be the real label you ask? Well, because my test data has real labels in it, and liblinear didn't balk at that, which suggests it may have been expecting them, i.e., expecting labeled test data rather than new/unlabeled data.

I guess I can see if these labels ever disagree with the second and third columns (which I assume are probabilities of labels 1 and 0 respectively, and add up to 1).

..Answer: they are the predicted labels, i.e., they correspond perfectly to whether the first column is greater than 0.5.

So, this file isn't enough in itself to create an ROC curve anyway.. I need the real labels as well.

And here it is:

Hm.. not quite as good as I was hoping. But I think there's room for improvement (I've seen a better ROC curve on this same data — well, similar data — so my choice of features is probably not ideal).

But, if we went with this, then using a low probability threshold of 0.01, we get 97% recall, with a 67% false-positive-rate, which I think means that 33% of the good items would be deemed good by the system, and 3% of those would be bad. Is 3% too high? I suspect yes.. but I'm not sure. I'll need to discuss this with people to decide how much to try and improve the system.

getting closer

I ran a human script to create an android widget. I offered $20 on oDesk. Here is the result. I tried it on my phone, and it works. Here's a couple screenshots that I asked them to put in the repository:

 

notes:

• they asked if they should use their github account or mine. I wonder if people sometimes give their github credentials to oDesk workers. If that is common, I should make it clear in my humanscript that I want them to use their own github account.
• they asked what name they should give the project. I asked them to make up a name. I should make this clear in the humanscript too.
• what they put in the hs.txt file is not what I wanted. I wanted the entire set of instructions, whereas it looks like they put a summary. I should make this clear too.

next step:

I posted a new humanscript on oDesk...

- fork https://github.com/dglittle/ImageChangeWidget

- copy-paste this job description to new file: humanscript/AA578/hs.txt

- widget is currently 2x2

    - make it 1x1

    - may require resizing the images

- create a new icon to represent a "working" state

    - I don't care what it looks like

    - maybe it can be an hourglass

- if icon has eyes open, and user clicks icon:

    - set icon to "working" icon

    - let x = a random number from 0 to 1

    - if x < 0.5

        - make http request to www.google.com

    - else

        - make http request to http://blahblahblah5678123.com

        - this domain should not exist

    - if the http request succeeds

        - change icon to eyes closed

    - else

        - change icon to eyes open

- if icon has eyes closed, and user clicks icon:

    - do the same thing as before,

        except if the request succeeds,

            change the icon to eyes open,

        else,

            change the icon to eyes closed

- take screenshot of widget running on phone in "eyes closed" state

- put screenshot in humanscript/AA578 directory

- make pull request to original repo

- send me message saying it is done

- in your cover-letter, please summarize what I want.

Thanks,

Greg

february-fire first humanscript

finally! I'm posting a humanscript for february-fire — this was the secret goal all along with taking on that project. Here is the humanscript in github issue form: https://github.com/dglittle/february-fire/issues/1

ok, posted it.. I put it as a $50 job, since it involves comprehending my code a little bit. we'll see if we get any takers..

a couple humanscript side projects

I had a couple of ideas that I want to humanscript. One is an advanced-chess server. The other is an add-on to the yabla language learning video player — I want it to show translations for each word directly beneath each word.

For the advanced-chess project, I created this issue: https://github.com/dglittle/advanced-chess/issues/1 , which has someone create a chess board using unicode characters. The next step will be to draw a chess board given a FEN string.

For the yabla project, I created this issue: https://github.com/dglittle/yabla-extension/issues/1 , which has someone create a starting point for a chrome extension that can interface with yabla. The next step will be to have it augment the way the transcription is shown.

I posted both of these issues on oDesk using nodesk.

human scripting

When writing the previous post on simulating probability, I wanted to provide interactive widgets so people could actually run the code. The widget would be a simple webpage with the code embedded in the url itself, so I could reuse it in an iframe. I'm sure this exists, but I couldn't find it. The closest I know of is jsFiddle, and my own simple JavaScript eval utility.

A simple idea, but I often forgo implementing such ideas since they take time to code and test, and more than anything they take time for me to context-switch my brain into programming mode and load up all the proper files and command prompts into the proper places.

But recently I've been playing with human scripting, which doesn't require me to context switch (as much). I opened my JavaScript eval utility in a browser, and looking at that wrote down some changes that needed to be made to realize my vision. Here's the script:

[header stuff from template... says to fork https://github.com/dglittle/glittle.org]

- go to glittle.org directory
- make copy of utils/JavaScriptEval
- call it jsEval
- make these changes
    - remove input box
    - make script textbox consume 100% height (like the output)
    - remove the "script" and "output" labels
    - remove the "history" button
    - move the "run >" button between the textareas (where the "<" button is, but at the top instead of the bottom)
    - remove the "<" button
- watch for any changes to the text in the input box,
    if there is a change,
    add a link below the input box that says "get url to this code"
- if the link "get url to this code" is clicked,
    navigate to a new url which is the current url + "#" + url-encoded version of the code in the input box
- when the page loads, check to see if there is a "#" in the url,
    if so,
    then put the stuff after it into the input box (and deal with the url-encoding.. not sure if you'll need to decode it)

- test this
    - run: node _server.js
    - load http://localhost:8080/utils/jsEval
    - enter: print("it works!")
    - click "get url to this code"
    - press run button
    - take screenshot (include address bar)
    - put screenshot in "humanscript/output"

[footer stuff from template... saying to commit, push and make pull request]


I wrote about 26 new lines of HumanScript. Not sure how long that took, maybe 10 minutes. I submitted the job to a mailing list I keep of programmers. Someone grabbed the task an hour later, logged their time on oDesk, and about an hour after that delivered this github pull request.

I looked at this image in the pull request:

It seemed to be working — had the code in the anchor part of the url like I wanted — so I clicked merge in github's interface. Then github automatically rebuilt glittle.org, and when I navigated over to glittle.org/utils/jsEval, I saw this:


I then went to work using this new utility to update my previous post.

So in summary, I wrote an e-mail, continued writing my blog post, later got an e-mail back saying "done", clicked a merge button in some web interface, and then had a working widget.

I want to run this humanscript: https://github.com/dglittle/heroku-mongo/issues/1

wow: http://thejh.net/misc/website-terminal-copy-paste

ideas: programming ideas to humanscript eventually, maybe:
- recreate infinite gearbox using WebGL
- create a side-scroller shooter (maybe with a paper airplane as the avatar)

want to do: go to Ireland and drink guinness from a guinness cup that was gifted to me

want to do: write something in Haskell

want to do: understand the problem with a big-number-solution presented here

hopefully last urgent task before humanscripting..

my goal has been to humanscript a lot of these tasks, but they've been to urgent.. this is hopefully the last one, and it's pretty simple. I just need to increase the delay for warned accounts from 48 hours to 5 days, and add 3 days of time to all the currently warned accounts..

step 1: make warning add 5 days instead of 48 hours.. done
step 2: update existing items in warned queue by adding 3 days (might make warned items that happened very recently have 8 days, but that's probably not worth worrying about).. this seems like a job for mongodb-side javascript..




db.eval(function () {
    var a = db.records.find({'status.warnUntil':{$exists:true}})
    for (var i = 0; i < a.length(); i++) {
        var r = a[i]
        db.records.update({_id:r._id}, {$set:{'status.warnUntil':r.status.warnUntil + (1000 * 60 * 60 * 24 * 3)}})
    }
}, {nolock:true})

...let's test it locally first.. ok, seems to work.. ran on server.. seems to have worked.. super. let's let people know.

oh.. I missed something.. I changed the "warnUntil" value, but that feeds into the availableToGrabAt value, which I need to also change (but only if it hasn't been grabbed)..

db.eval(function () {

    var a = db.records.find({'status.warnUntil':{$exists:true}})

    for (var i = 0; i < a.length(); i++) {

        var r = a[i]

        db.records.update({_id:r._id,availableToGrabAt:(r.status.warnUntil - (1000 * 60 * 60 * 24 * 3))}, {$set:{availableToGrabAt:r.status.warnUntil}})

    }

}, {nolock:true})

..that seems to have repaired it.

ok.. oh, and there's one other thing I need to do.. I added a feature to february-fire that people now want in the clone of february-fire (I'll need to humanscript some merger of these two projects.. though one of them will probably end before that happens anyway..)

phew, ok, done..

threads

hm.. it appears I have several threads going.. not sure how to proceed.. let's see..

I'm working on making my homepage more pinterestish. I'm thinking of just having my "best" projects, and having a picture for each one — so far I've got:

I'm working on several humanscript projects:

• the yabla extension: MVP is done, now where to put it..
• the advanced chess game: board is done, extracting suggestions from javascript ai is sortof done — someone did it, but they didn't wrap it quite as cleanly as I was hoping, so I still need to meddle with it..
• random.org/twister thing: all the pieces in place, but I lost faith that people would use it, but a friend restored some hope, and it's so close, I just need to push it a little more

There's also a number of hiring experiment things, including nodesk, some other oDesk hiring things, and some MTurk hiring things.. all of them are just easier ways of hiring for specific sorts of jobs.

I'm also trying to understand Integrated Information Theory, which is some person's theory of consciousness, and I've very interested in consciousness, so despite being skeptical of their theory, I do want to at least understand it, and it has some maths that I don't understand yet.

I've also been trying this new meditation technique, where I'm trying to stay.. er.. awake.. all the time. More awake. I feel like it is going in a good direction.. not sure yet.

Also, I fight depression, and although I'm currently not depressed, I worry that I'll slip down a few (many) more times before I've figured it out, and I don't want to get too into projects during these moments of coming up for air, since I don't want to drop too much when I go down again..

Also, I plan to go to Japan at the end of May,

and I want to actually learn a foreign language with the yabla extension thing,

and there are some other pet projects, and also some responsibilities with work, and also some other projects that other people would like me to help with..

I'm not quite sure how to prioritize it all..

humanscripting with nodesk (part 1: getting off ground)..

ok, the time has come, finally.. let's get nodesk up and running and start humanscripting again..

step 1: hm.. I want to create an app on heroku and link it to the current nodesk repo.. I'm not sure exactly what "heroku create" does.. hm.. let's backup everything in nodesk, clear it out, and try running "heroku create"..

hm.. seems to have create an app ok.. now I want to name this one "nodesk", let's try renaming it on heroku.. actually, let's see if there's a command-line for that.. yup: heroku apps:rename newname..

ok, I got http://nodesk.herokuapp.com, sweet..

now let's put a skeleton app up there..

ok, we'll copy from nar-nar, since that's the most recent project..

ok, "hello world" version is up..

now I'm thinking I want to merge nodeutil.js and u.js.. hm.. that actually seems like something I should humanscript later..

ugg.. I tried upgrading node with brew, but apparently I didn't install it with brew, so I tried installing it with brew, but it is caught in some intermediate state where I can't run the old node or the new node.. I'm nodeless! nodeless... nodeless...

I'm back! noded. just needed to delete some files it was complaining about.. hopefully unnecessary files..

ok.. let's get login working..

hm.. I just noticed that the following code in login.js...



db.collection('users').update({ _id : user._id }, { $set : _.omit(user, '_id') }, { upsert: true }, function (err) {
    if (err) return done(err)
    done(null, user)
})



...was missing the err, so err was referring to the error in the enclosing function. I think I'll fix this in nodesk's version of login.js, but I don't want to endanger the running apps like nar-nar and february-fire by "fixing" this in them, since it's more likely to introduce problems than prevent them..

ugg.. login isn't working: 500 failed to obtain request token (status: 401 data: � sK��IMQ(�W(K��LI,IU�O,-�P(�L�K,)-JUH� �g�� ���,)

things that are different:

- I'm using a different odesk api key and secret

- I'm potentially using a different version of passport

I guess I could try with the old keys..

..the old keys work. So I guess the keys are the issue..

..ahh, right, I set my "nodesk" odesk app to use the non-oauth authentication.. I guess I should re-create it..

ok, login working :)

now the first thing I want is the ability to post a job.. hm.. I'm tired.. let's take a break for a bit..

more humanscript work notes

I just created an issue here: https://github.com/dglittle/misc/issues/1
I created a super simple job — maybe too simple. It just says "github issue: create a throbber", and the task description just has a link to the github issue. We'll see how that goes.

Now to post the other one.. done. Ok, let's see how those turn out.

update: hm.. I don't give much indication up front about how big the task is — maybe I can just say in the title "about 1 hour". Ok, I edited the titles to include "(about 1 hour of work)".

idea dump

idea: commission someone to take a foreign language youtube song and create a version with subtitles in both that language and english, as a language learning tool.

idea: mturk app for sorting photos — you point it at picassa or dropbox, and it tells you which photos are worth showing your friends.

idea: create heroku app that serves random numbers with json-p. it can initialize a mersenne twister from random.org as often as their api allows, and serve pretty-darn-random numbers without rate limiting issues.

cool thing a friend made where people improve on each other's sketches: http://sketchabit.adamsmith.as/ and http://sketchabit.adamsmith.as/master.php

a couple notes about humanscript:

• I original thought the "big idea" was writing instructions in an imperative way, but I think the real insight is writing instructions in such a way as to never need to communicate with the person doing the work — the real power comes from just getting a result, with no management.
• humanscripting requires architecting a project differently — more UNIXy — since it's easier to specify a self-contained module than to specify how to mess with the "glue" code at the core of a project. this seems like a skill that people will need to learn, like learning to program.

sleep

Following up on this post, here is a link to my sleep patterns.

It is currently a google spreadsheet. Here is my improvement plan:

• Create visualizer hosted on github.
• Have visualizer load data from google spreadsheet with something like this.
• Create android app that let's me say when I'm sleeping or waking up, that posts to google spreadsheet.

I want to write HumanScripts to do these things. My HumanScripting has been dormant for a while as I've been trying to transition to a different way of hiring people.

I had hired a small pool of JavaScript programmers, and would post jobs to a mailing list, but I got stuck when I wanted to post scripts that required other knowledge (e.g. android development). Also, my JavaScript programmers seem to have stopped checking my mailing list, perhaps because I don't post jobs often enough.

So my new plan is to post each HumanScript as a new job on oDesk, but using the API to make it easier. I've successfully posted a job, retrieved the list of offers, hired someone, sent them a message saying "begin!", and paid them, but I'm currently stuck ending the contract. I can end the contract without feedback, but I don't know how to end the contract with feedback.

humanscript notes

I posted four humanscripts last night.

Notes:

For one job, I asked someone to write a simple function for creating lorem-ipsum style random text. The person I hired had submitted completed code as part of their cover letter. I should have hired them to add the functionality I needed to an existing open-source lorem-ipsum generator.

For another job, a hired the first applicant who answered my three questions correctly, and they sent me a screenshot for verification this morning — it looked correct.

Two of the humanscripts failed to get any applicants. I need to figure out why. One asks someone to create a simple throbber, and insert it into a larger project. I think I should split that one into two tasks: creating the throbber, and inserting it.

After talking to my friend, I think it will also help to make the jobs into github issues, for better accountability (something for me to put links to from here), and also to provide more context to the workers about the job.

So next steps:
- split the throbber job into two sequential jobs, and post the first one as an issue
- repost the alignment job as an issue

yabla

More progress on the language learning idea from this previous post.

Namely, it works, and is here: https://github.com/dglittle/yabla-extension , and was mainly created by this humanscript: https://github.com/dglittle/yabla-extension/issues/3

Here's what it does — it adds the red English words beneath the Spanish words:

posting a human script

I want to post a humanscript to make a change to february-fire. The task I'm interested in is allowing admins to change who the editors are.

I want to clone my old script. It would be great to have a clone feature in nodesk.

After thinking for a long while, I think the right next step is to make february-fire easily runnable after forking it, including having it generate just enough test data for the person who runs it to know it's running correctly.

I think I'll need to do this bit myself.

notes from CrowdCamp part 2

idea nuggets from CrowdCamp and CSCW 2013:

• a lot of creativity is "riffing". see this and this
• here's an interesting way to create a story: tell someone that you have a story in mind, and have them ask you yes-or-no questions about it. You can answer randomly, and they'll effectively create a story from their questions. see this
• from a key-note, and from someone's interpretation of the key-note:
• longer one-on-one interactions with people are more effective for one's reputation than lots of small interactions, even if the longer one-on-one interactions exclude small peripheral interactions. The reason is that reputation consists of stories people tell about you, and not the interactions themselves, and longer in-depth interactions inspire better stories.
• it is valuable to have people be creative in separate groups, and exchange ideas more rarely, like every year. I don't know if the following anecdote is true, but I got the impression that Steve Jobs had many groups at Apple design an iPhone separately, without talking to each other.
• this is the first speaker I've heard make fun of the people asking questions. To one person he said: "people who start questions that way haven't been listening, but have just been focussed on the question they want to ask". To another person he said: "have less of a preamble to your question".
• the Microsoft Kinect can quickly identify all the pixels in an image that are part of a human. see this
• the river walk is nice. possible reasons: the road is replaced with a river; there are escher-esque stairs; people are laid back; it feels like a cozy canyon.
• Winter Mason suggested that oDesk include offline non-remote work as well, e.g., managing the payroll of small companies that have on-site employees.
• Anand Kalmari informed me that oDesk has healthcare options for contractors, which I didn't know about, see this.
• Anand also showed me a demo of a system for hiring writers quickly on Mobileworks, and I told him about the humanscripting idea.
• I talked with a few other people about humanscripting, including: Nikki Kittur, Walter Lasecki, and Nicolas Kokkalis. The idea that seemed to resonate most with people was thinking about specifications imperatively rather than declaratively
• I saw Nicolas present the GmailValet paper; I hope this generates interest in valet-keys for more services
• I told someone that it was exhausting to talk to lots of people at parties at the conference. They said that's what alcohol was for. This finally made sense. Unfortunately, I had to drive, so I couldn't employ that strategy.
• I saw the future or crowdwork paper. I think everyone has similar visions of where things are going, and I'm sure we'll get there. Though, I was reminded that it may be worth coming up with a compelling positive vision for the future of work as a defense against potential negative reactive regulations.
• I heard Rob Miller talk about plans for expanding a nice paired-research system he's been experimenting with. The idea is to pair people up every week so they can help each other, and presumably cross-pollinate ideas. Some people external to his research group are on the list, including me, and he pitched the idea of having groups at multiple schools on the list, where the system would usually pair people with someone at their own school, but sometimes pair them with external people. This idea seemed linked to the key-note mentioned above.
• I had multiple discussions with Walter Lasecki, who is an interesting person.
• He experiments with extreme sleep schedules, which interest me. I expressed my concern that waking up is hard, and asked him if waking up was hard for him. He said it was like sticking forks in his eyes. But he manages.
• He has 12 undergrads working for him, and he's starting to hire people on oDesk as well. I want to pick his brain about his process. It seems like he is doing something right with regard to management.
• I'd like to spend some time with his group (Jeff Bigham's group) at Rochester. I've been there before, while interning at Xerox. In fact, I was wearing a sweater at CSCW that their group gave to me :) — in fact, the logo on that sweater was drawn by Kyle Murray, who is now in Rob's group at MIT, and will be interning at oDesk this summer, extending the  humanscript idea, which he created. So, lot's of social ties.
• write more nuggets here when I remember...
• oh, I re-had an idea of extracting idea-nuggets from papers, and posting them, and trying to get the community to think in terms of idea-nuggets, rather than papers — assuming this is a good idea