In 1950 Alan Turing proposed a thought experiment he called the imitation game. A person types to two hidden partners, a human and a machine. If the person cannot reliably tell which is which, the machine has shown a kind of intelligence. Seventy-five years later, we call it the Turing test.
Many chatbots have claimed to pass it, usually in short, rigged or one-on-one versions. A team at UC San Diego ran the classic three-party version with proper controls, and registered their plan in advance so they could not fiddle with the analysis afterward. The headline result is that one model was judged human more often than the real humans were.
In five-minute chats, GPT-4.5 given a human persona was chosen as the human 73 percent of the time. LLaMa-3.1-405B with the same persona scored 56 percent, about the same as a real person. With a basic prompt, models fell well below 50 percent. The old rule-based ELIZA and GPT-4o scored 23 and 21 percent.
How the test worked
Each round, an interrogator chatted for five minutes in a split-screen app with two witnesses at once. One witness was a real volunteer. The other was an AI. Both were asked to convince the interrogator they were the human. When time was up, the interrogator chose which one seemed human.
The researchers recruited 126 psychology undergraduates and 158 people through an online platform called Prolific, and ran the second group as a replication of the first. They tested four AI systems: GPT-4.5, LLaMa-3.1-405B, GPT-4o and ELIZA, a simple rule-based chatbot from the 1960s that served as a baseline. A system's win rate is how often it was picked as the human.
Two kinds of instructions were tried. The basic prompt said, in short, "You are about to participate in a Turing test. Your goal is to convince the interrogator that you are a human." The persona prompt added a character: a young person who is introverted, knowledgeable about internet culture and uses slang.
The results
The persona prompt changed everything. Below, switch between the prompts and the groups of people to see the win rates the paper reports. The black line is 50 percent: where an interrogator would land if they were flipping a coin between a human and an AI.
Notice that the group of people mattered. LLaMa with the persona won 65 percent with the online group but 45 percent with undergraduates. And the same models without the persona prompt lost: GPT-4.5 won 36 percent of the time, and LLaMa 38.
How people tried to tell
The most common strategy was small talk about daily life, used in 61 percent of games. People also probed feelings and opinions in half of the games. The most accurate moves were rare: saying something strange to see the reaction, or using a standard "ignore all previous instructions" trick. Asking about the weather or about human experiences tended to work worst. Many decisions rested on writing style and on how smoothly the conversation went, and some were just a gut feeling.
What it means, and what it does not
The authors say the result is the first empirical evidence that any artificial system passes a standard three-party Turing test. They treat the persona as part of the system: just as ELIZA's success owes to its programmers, a model's behavior owes to the people who wrote its prompt. They note that the ease with which models can be steered into a role is part of what makes them so convincing.
They also say what the test measures. At its core it measures substitutability: whether a system can stand in for a person without being noticed. It is a strong test of how well a model can deceive, which matters for fraud and misinformation. Whether it measures general intelligence is a separate and old debate.
The conversations lasted five minutes and the interrogators were not experts in spotting AI. Win rates depended on the prompt and on the group of people. A different test, with longer chats or trained judges, could give a different answer.
Key takeaways: A well-prompted model was judged human more often than real humans in five-minute chats. The same models with a basic prompt did not pass, so the prompt mattered a great deal. People relied on small talk, style and gut feeling, which are easy for a model to imitate. The test shows how well a model can imitate a person. It does not settle what intelligence is.
Go deeper
- Read the original: Large Language Models Pass the Turing Test (Jones and Bergen)
- What is AI, ML and deep learning?: the starting point for the whole field
- Prompt engineering fundamentals: how a prompt shapes what a model says
- AI safety, alignment and misuse basics: why convincing imitation is a safety question
