Summary: Answer the question to the right, and see how other turkers and blog readers answered similar questions. After you answer the question, the red dots will show where people answered "yes".

This is just for fun. Stuff like this has been done before. For instance, this CrowdFlower post shows what names people give to different colors.

I had originally wanted to ask if the dot was "above" the bar, but I feared the result might look to boring, like a rectangle.

Technical tid-bits:

The url in the iframe is exactly the same url I posted on MTurk. The JavaScript detects if it is being viewed on MTurk and does two things. First, if it is being viewed in preview mode, it overlays a grey rectangle with the word "preview" to prevent people from accidentally answering the question before they accept the HIT. Second, it makes it so that answering the question submits the HIT, rather than showing the results. The utility function for submitting the HIT creates a form with the necessary elements, including a dummy result so that MTurk doesn't feel left-out -- this is necessary, since MTurk considers it an error to submit no results. Of course, the actual result is submitted to my server via ajax before submitting the HIT.

I only use one API call to MTurk, and that is to create the HIT. I don't get results, and I don't approve people. I don't get results because they have already been submitted to my server, and I don't approve people because they will automatically be approved after one hour -- this is a parameter when creating the HIT. But Greg, doesn't that mean people can cheat? Well, yes, but I find that people typically don't cheat on questions like this, because they're simple and kindof fun. Also, I wouldn't know if they did cheat. Each person is only allowed to answer the question once, so I can't even see if their results are statistically improbable.

Another tidbit is that I sometimes have unrealistic scalability fears, like "what if lots of people read this post and answer the question, endlessly increasing the size of the data structure until it takes forever for people to download?" So, I made this data structure have a maximum size: there are a finite number of points it will ask about (you can see them arranged on a grid after you answer the question), and it just records the number of yesses and noes for each location. But Greg, that doesn't really limit the size of the data structure, since the number of yesses or noes can grow infinitely. True, but first, that is a scalability issue I'm willing to sleep at night with, and second, JavaScript numbers cannot actually be infinitely large, like Python numbers can. I think JavaScript uses 64-bit floating point numbers under the hood. So at some point, the data structure would just be wrong, rather than too big...

No comments:

Post a Comment