Skip to content
Large Language Models Pass the Turing Test 4 min

Cameron R. Jones, Benjamin K. Bergen · arXiv (UC San Diego) 2025

Large Language Models Pass the Turing Test

A pre-registered three-party Turing test found GPT-4.5 with a persona was picked as the human 73 percent of the time. Explore the results by prompt and by crowd.

Read the original paper

AI & ML · White Belt · Free · 4 min read

Large Language Models Pass the Turing Test

In 1950 Alan Turing proposed a thought experiment he called the imitation game. A person types to two hidden partners, a human and a machine. If the person cannot reliably tell which is which, the machine has shown a kind of intelligence. Seventy-five years later, we call it the Turing test.

Many chatbots have claimed to pass it, usually in short, rigged or one-on-one versions. A team at UC San Diego ran the classic three-party version with proper controls, and registered their plan in advance so they could not fiddle with the analysis afterward. The headline result is that one model was judged human more often than the real humans were.

In five-minute chats, GPT-4.5 given a human persona was chosen as the human 73 percent of the time. LLaMa-3.1-405B with the same persona scored 56 percent, about the same as a real person. With a basic prompt, models fell well below 50 percent. The old rule-based ELIZA and GPT-4o scored 23 and 21 percent.

How the test worked

Each round, an interrogator chatted for five minutes in a split-screen app with two witnesses at once. One witness was a real volunteer. The other was an AI. Both were asked to convince the interrogator they were the human. When time was up, the interrogator chose which one seemed human.

The setup Turing proposed, run as a controlled experiment. If people cannot reliably pick out the human, the machine is said to have passed. Here, interrogators had five minutes with each witness.

The researchers recruited 126 psychology undergraduates and 158 people through an online platform called Prolific, and ran the second group as a replication of the first. They tested four AI systems: GPT-4.5, LLaMa-3.1-405B, GPT-4o and ELIZA, a simple rule-based chatbot from the 1960s that served as a baseline. A system's win rate is how often it was picked as the human.

Two kinds of instructions were tried. The basic prompt said, in short, "You are about to participate in a Turing test. Your goal is to convince the interrogator that you are a human." The persona prompt added a character: a young person who is introverted, knowledgeable about internet culture and uses slang.

The results

The persona prompt changed everything. Below, switch between the prompts and the groups of people to see the win rates the paper reports. The black line is 50 percent: where an interrogator would land if they were flipping a coin between a human and an AI.

Notice that the group of people mattered. LLaMa with the persona won 65 percent with the online group but 45 percent with undergraduates. And the same models without the persona prompt lost: GPT-4.5 won 36 percent of the time, and LLaMa 38.

How people tried to tell

Numbers from the paper. The most common move was ordinary small talk. The most common reasons were about writing style and how smoothly the chat went, not about knowledge.

The most common strategy was small talk about daily life, used in 61 percent of games. People also probed feelings and opinions in half of the games. The most accurate moves were rare: saying something strange to see the reaction, or using a standard "ignore all previous instructions" trick. Asking about the weather or about human experiences tended to work worst. Many decisions rested on writing style and on how smoothly the conversation went, and some were just a gut feeling.

What it means, and what it does not

The authors say the result is the first empirical evidence that any artificial system passes a standard three-party Turing test. They treat the persona as part of the system: just as ELIZA's success owes to its programmers, a model's behavior owes to the people who wrote its prompt. They note that the ease with which models can be steered into a role is part of what makes them so convincing.

They also say what the test measures. At its core it measures substitutability: whether a system can stand in for a person without being noticed. It is a strong test of how well a model can deceive, which matters for fraud and misinformation. Whether it measures general intelligence is a separate and old debate.

The conversations lasted five minutes and the interrogators were not experts in spotting AI. Win rates depended on the prompt and on the group of people. A different test, with longer chats or trained judges, could give a different answer.

Key takeaways: A well-prompted model was judged human more often than real humans in five-minute chats. The same models with a basic prompt did not pass, so the prompt mattered a great deal. People relied on small talk, style and gut feeling, which are easy for a model to imitate. The test shows how well a model can imitate a person. It does not settle what intelligence is.

Go deeper

Found this breakdown useful?

Share it with someone else wrestling with this paper.

Discussion0

Join the Discussion

Sign in to leave comments, reply to others, or like insights.

Sign In to ScaleDojo

No comments yet. Be the first to start the thread!

Enjoyed this? Get more like it.

New paper breakdowns, levels, and one concept worth knowing, straight to your inbox.

No spam, ever. Unsubscribe in one click.

Enjoyed this content?

Your support keeps us creating free resources

We put a lot of hours into researching and writing these guides. If it helped you, consider buying us a coffee. Every bit goes toward keeping ScaleDojo's content free and growing.

$

One-time payment via Stripe. ScaleDojo account required.