I've taken a break since Tuesday from work and MTurk for the most part. I wanted to do a lot of hits last night but was only able to do about 200 since the Image Adjustment hits available were running low and I don't like doing the Confirm Artist Name hits.
There are rumors on the Turker Nation boards that people are running scripts just to accept hits and randomly click an image and submit them as rapidly as possible. There has been no word from the Monolith at Amazon of course, so there's no way to confirm or deny it. Greedy people running scripts with 25 tabs open at once on their browser could also account for some of it.
Hopefully Amazon will seriously consider putting a hard limit of less than 10 hits that can be checked out at once. I'm sure this will slow some people down, but doing 1000 hits an hour is not really providing valuable work anyway. It's just taking advantage of the fact that the A9 images are skewed so heavily towards the "None of the Above." If "None of the Above" were only the correct option about 20% of the time, you'd have to spend more than 4 seconds on a hit or your accuracy would suffer greatly.
I suspect that Amazon might not care about getting "real" work done with IA hits so much right now. I see the same IA hits over and over again so I think they're recycling a lot in order to maintain load on the system and get more data they can use to optimize things.
2 comments:
I think that if they were to impose a limit like that, the true accuracy would improve -- and I'm not talking about the rejection rating they're showing today.
Or maybe I'm just bitter that I'm not able to get my rejection rating below 17 percent. Without feedback from them, how am I supposed to know which angle of the front door of 8383 Wilshire to pick the next time it's fed to me? Or whether seeing ABC investments on the second floor of the strip mall is better than picking a NOTO?
That's a good sign -- I've been picking the one with the street lamp on the left, too, probably 50 times by now. It's usually the sixth image, unless NOTO is the sixth or seventh.
Post a Comment