Artificial intelligence technology ChatGPT – created by Open AI – has caused a stir with its recent bar exam performance. The AI scored better than 90% of human test-takers, leading to worries about its ability to carry out complex legal work. However, a Ph.D. candidate at the Massachusetts Institute of Technology has cast doubts on these claims, alleging that the AI’s score was actually closer to the 68th percentile. The results depend on which test is used as a benchmark and how percentiles are calculated. ChatGPT’s creators, legal AI company Casetext, stand by the original conclusions and plan to clarify misunderstandings when their research is published in final form.
Open AI is a company that was founded in 2015. Its mission is to create ethical artificial intelligence products.
Eric Martinez is a Ph.D. candidate at the Massachusetts Institute of Technology who has analysed the recent bar exam performance of ChatGPT, an AI developed by Open AI. He has claimed that the widely publicised claims that the AI outperformed nine out of ten human test-takers may have been overstated.
Title: Stellar or So-So? ChatGPT’s Bar Exam Performance Brings Controversy
Word Count: 173
