list experiment

I keep remembering "the list experiment idea is so cool", but I keep forgetting what it is.

So here is what it is. We want to ask people if they agree with X, but they don't want to tell us, because they don't want to be thought of as someone who agrees with X.

So we get two groups of people, and we ask everyone in the first group this question: of A, B and C, how many do you agree with?

Then we ask everyone in the second group this question: of A, B, C and X, how many do you agree with?

If we then subtract the average response of the first group from the average response of the second group, we'll get the percent of people who think X.

Of course, one hole in this scheme — it seems to me — is that if I'm presented with A, B, C and X, and I agree with all of them, I'm unlikely to provide 4 as my answer, since then people will know that I agree with X. It would be cool to think of some way to guarantee plausible deniability for each person who takes the survey.

update: Here's an idea for guaranteeing plausible deniability: the user picks a number between 1 and 6, then they click a button to roll a dice, and if the number on the dice is their chosen number, then they are instructed to lie. Otherwise they tell the truth. That way, if people don't like their answer, they can say "well, I lied, because my chosen number came up."

It would be fun to test this on Mechanical Turk...