AI Boosts Productivity and Quality: New Studies Show Consultants Outperform with ChatGPT-4
Recent studies demonstrate the significant potential of incorporating AI chatbots, specifically ChatGPT-4, in knowledge work. Researchers at the Boston Consulting Group (BCG) and Wharton Business School found that consultants using ChatGPT-4 outperformed those who did not by a significant margin. They completed 12.2% more tasks on average, finished tasks 25.1% more quickly, and produced 40% higher quality results than their counterparts.
To evaluate the impact of AI on knowledge work, the researchers assigned hundreds of consultants to either use AI or not. Both groups were given the same writing, marketing, analytical, persuasive, and creative tasks, such as writing press releases and performing market segmentation. Consultants with access to AI consistently outperformed the control group in all measures, including task completion time, number of tasks completed, and quality of outputs.
A noteworthy finding is that AI acts as a skill-leveler, benefiting lower-performing consultants the most. Those who performed poorly at the beginning of the experiment experienced a significant 43% improvement when using AI, while the top performers saw a lesser boost. This suggests that AI can elevate the performance of lower performers, bringing them closer to the level of top performers.
These results align with a similar study conducted by Stanford and MIT, which examined the performance of 5,000 customer service agents in a Fortune 500 enterprise software firm over a year. The study found that agents augmented with AI were 14% more productive on average, with the least-skilled workers benefiting the most by completing their work 35% faster.
While AI augmentation has the potential to increase productivity and efficiency, it also raises workforce implications. Companies may achieve more with the same number of employees, leading to higher revenues. Highly skilled workers can focus on specialized tasks that AI cannot perform, resulting in a workforce with a broader range of skills. However, the increased efficiency and productivity brought about by AI can also create higher performance expectations, potentially causing stress or job dissatisfaction. Additionally, it may lead to downsizing in certain areas.
The tangible impact of AI on the workforce is evident in a recent report by job placement firm Indeed. The report analyzed over 55 million job postings and 2,600 job skills, revealing that nearly 20% of jobs are highly exposed to the impact of AI, while 45% are moderately exposed. As AI continues to perform a significant portion of required job skills, it further reinforces the potential influence of AI in various roles.
To explore the capabilities of AI in more complex tasks beyond writing press releases, Section School conducted a thought experiment on the analytical abilities of current chatbots. They asked four AI chatbots to provide feedback on a board slide deck before a board meeting. The quality of output varied among the chatbots, but one chatbot named Claude from Anthropic was almost as good as a human board. It showcased an understanding of the macroeconomic environment, demonstrated appropriate ambition, and quickly delved into third-level implications and big picture opportunities.
This implies that AI advisors could potentially augment, or even partially replace, human experts and advisors in evaluating complex decisions, strategies, and plans. However, the question remains whether AI can truly replace human strategic thinking and creativity.
While AI evidently improves productivity, a separate paper examining the performance of job recruiters found that those who used high-quality AI became lazy, careless, and less skilled in their own judgment. The increasing quality of AI may reduce human effort and attentiveness, allowing AI to substitute rather than augment human performance. This highlights the danger of becoming overly dependent on AI and failing to exercise human judgment.
Widespread adoption of AI could potentially lead to a future where people rely on autopilot, similar to our dependence on smartphones. Previous research on smartphone usage has shown a weakening of intellect as the brain becomes reliant on technology. The same may hold true for any information technology that delivers content without requiring active learning or discovery.
Concerns about potential intellectual atrophy due to AI dependency have been raised by Daniel Weld, a professor at the University of Washington. He worries that human abilities may weaken if individuals feel they cannot surpass AI in various areas, leading them to stop trying.
As AI’s influence in the workforce becomes increasingly apparent, it is crucial for companies to leverage AI’s strengths while preserving human skills and judgment. Finding the optimal division of labor between humans and AI is vital to maximize human engagement and capitalize on the unique capabilities of each.
