ChatGPT succeeds in passing medical license and bar exams after passing Wharton MBA final

OpenAI's ChatGPT, a mass-market artificial intelligence chatbot, has recently gained recognition as knowledge workers use it to complete tasks such as writing emails and debugging code quickly.

Research has found that the system surpassed expectations in two notoriously difficult tests: the United States Medical Licensing Exam and the multistate multiple choice section of the Bar Exam.

The USMLE test typically requires years of postsecondary education and intensive study to complete; yet, ChatGPT achieved a score “at or near the passing threshold” with no specialized training or reinforcement.

The system also performed better than other AI models, showing a high level of concordance and insight in its explanations. The performance on the Bar Exam was similarly impressive, with the system landing “comfortably within the passing range” and its top two and top three choices being correct 71% and 88% of the time.

The success of ChatGPT has raised questions about the potential of automation to eliminate white-collar jobs. However, many knowledge workers actually find that the system increases their efficiency; a survey from Fishbowl revealed that 27% of professionals at prominent consulting, technology, and financial services companies have already used ChatGPT.

The Daily Wire reported:

The researchers concluded that ChatGPT “significantly exceeds our expectations for performance on this task” and noted that the rank-ordering of possible choices confirms the “general understanding of the legal domain” reflected by the system.

The system also achieved a grade between B and B- on a graduate-level operations management test at the University of Pennsylvania’s Wharton School, as reported by Rebel News.

“It does an amazing job at basic operations management and process analysis questions including those that are based on case studies,” wrote Professor Christian Terwiesch. “Not only are the answers correct, but the explanations are excellent.”

While ChatGPT answered most questions correctly and provided excellent explanations, the researchers noted that the system made “surprising mistakes in relatively simple calculations” of sixth-grade math that were often “massive in magnitude,” the Daily Wire reported.

Furthermore, the current version of the system is not yet able to handle more advanced process analysis questions, even when they are based on standard templates.