The Concerns of AI Infiltrating Crowdsourced Work

A recent study by Swiss university EPFL has revealed that a significant proportion of crowd workers on Amazon’s Mechanical Turk service may have cheated when performing certain tasks that required humans instead of robots. The research suggests that anywhere between 33% and 46% of these workers used AI tools such as ChatGPT to complete assigned tasks. Amazon’s Mechanical Turk has traditionally been a go-to service for developers seeking human assistance with tasks that require human analysis or expertise. However, the prevalence of AI tools being used to complete tasks could undermine the confidence in the quality of data that is generated. For machine learning models, it is challenging to distinguish the origin of data between generated text and human-generated text. Crowdsourced workers were given a task where they had to summarize research abstracts from the New England Journal of Medicine into 100-word summaries. The researchers were able to distinguish between machine-generated and human-generated text using this methodology. This type of task is something that generative AI tools like ChatGPT are good at, and this could mean that the use of Mechanical Turk may be under threat.