Human Algorithms

I thought I'd spend a little bit of time discussing the techniques I use when doing work on Amazon's Mechanical Turk. I'm a QA Analyst by profession so it's in my nature to consider process improvement and usability. I'm only going to consider Image Adjustment Hits at this time simply because these are the only hits I've worked on and they're also the only hits available now on MTurk, at least recently.

First of all, I've found I prefer to work on cities like New York or San Francisco that have a higher population density. This is because they tend to have smaller store fronts that have lots of signs and seem to have their street address displayed more often. I could be completely wrong about this and have no data to back it up, but my internal neural net is wired this way so that's how I approach it currently. Working on images from a city like Phoenix also seems to have a lot of "None of the Above" answers, which myself and some other users feel are being held in "Pending Status" too long and doesn't lead to quick payment.

So once I choose a city to work in, I just start working. I take the first assignment that comes up and don't worry too much about pre-scanning it before I accept. The first reason for this is because if you wait more than a few seconds to accept it, there's a good chance it will be assigned to someone else. If you were quick enough you could probably get a quick idea if it's a potential "None of the Above" and skip it, but I think that would take only slightly less time than actually just completing the hit anyway.

As soon as I accept the hit, I look at the business name and just the numbers of the address. So if it was "Joe's Tacos" at "113 West Smith St" I'd just mouth "Joe's Tacos 113" to myself then start scrolling down to look at the images. If I see a good image I click on the radio button and keep scrolling. If I don't see a good image by the time I hit the bottom I click on "None of the Above" and submit and immediately start on the next hit. A lot of the times, at least recently, it seems that it's very obvious very quickly that it's "None of the Above." When you see five images of an empty lot you can finish it in less than five seconds probably.

Things slow down often when there are multiple images of the correct store front. This is also where I suspect I get most of my rejections. If there's a choice between two images that both clearly show the store front, but from slightly different angles then it becomes difficult to decide which to choose. I've generally just gone with the first one in that case since I assume that's what others will do. Since no guidance is given from the requestor regarding their preferences I don't know how else to approach it. Maybe they prefer shots that are angled from one side or the other. Who knows? It's likely just a popularity contest at that point.

I haven't really timed myself on how fast I can do Image Adjustments but I know I could go faster if I was sitting down at a table and really concentrating on it. Currently I watch things on my Tivo while I'm doing it and I often stop to click back for some dialogue I missed. I also tend to stop about every 10 hits and look at the total I've done and calculate how many more I need to do to get the Petzl helmet that is my goal from all of this.

Some people have asked if you could make a living doing this. I think you could probably do somewhere around 800 to 1000 a day if you did it at least 8 hours a day and were really dedicated. I'm sure I couldn't. Regardless though, if you submitted 800 a day and did this for 22 days a month, that would be 17,600 hits a month submitted. Assuming they were all processed and you had a rejection rate of about 15% then you'd be approved for 14,960 which at three cents a hit would leave you with $448.80 which means you would make about $2.55 an hour for the 176 hours you put in. These are very rough numbers of course, so your mileage may vary.

So I would say no, you can't really make much of a living doing this. I treat it as a way to get a discount on merchandise from Amazon by doing some work for them. It would be similar to washing a few dishes at your favorite restaurant before you sit down to eat. I have a specific goal that I'm working for, plus when I'm working on hits is the time of day that's usually almost completely unproductive for me anyway so why not make a buck while I watch an episode of Law and Order from 10 years ago?