After Clearing MBA Exam, ChatGPT Now Passes US Law School Test

After clearing MBA exam, ChatGPT, a chatbot powered by reams of data from the internet, has passed exams at a US law school after writing essays on topics ranging from constitutional law to taxation and torts. Jonathan Choi, a professor at Minnesota University Law School, gave ChatGPT the same test faced by students, consisting of 95 multiple-choice questions and 12 essay questions, reported AFP.

In a white paper titled “ChatGPT goes to law school” published on Monday, he and his coauthors reported that the bot scored a C overall. While this was enough for a pass, the bot was near the bottom of the class in most subjects and “bombed” at multiple-choice questions involving mathematics.

“In writing essays, ChatGPT displayed a strong grasp of basic legal rules and had consistently solid organization and composition,” the authors wrote. However, the bot “often struggled to spot issues when given an open-ended prompt, a core skill on law school exams”.

Read | Five Indian-American Teenagers Selected as Finalists of Prestigious Science and Maths Competition in US

Officials in New York and other jurisdictions have banned the use of ChatGPT in schools, but Choi suggested it could be a valuable teaching aide. “Overall, ChatGPT wasn’t a great law student acting alone,” he wrote on Twitter.

“But we expect that collaborating with humans, language models like ChatGPT would be very useful to law students taking exams and to practicing lawyers.” Playing down the possibility of cheating, he wrote in reply to another Twitter user that two out of three markers had spotted the bot-written paper. “(They) had a hunch and their hunch was right, because ChatGPT had perfect grammar and was somewhat repetitive,” Choi wrote.

Earlier, a professor from the University of Pennsylvania’s Wharton School of Business put ChatGPT to an MBA test. Professor Christian Terwiesch published a research paper in which he examined ChatGPT’s performance on the operations management final test (that is a typical MBA core course). He found out that ChatGPT passed one of the toughest exams the MBA programme had to offer that too with some outstanding grades. In his study, Professor Terwiesch explained that the AI chatbot “does an amazing job at basic operations management and process analysis questions including those that are based on case studies.” However, when it came to math exams, ChatGPT “is so horrible,” he said.