Claude 3 AI is almost as good as a human at persuading people to change their mind — here’s why

Claude 3 logo on phone
(Image credit: Getty Images)

Persuasion is such an important skill for humans to master that there are thousands of books, college classes and an entire self-help industry designed to help people convince others to help with a project, convert to a new philosophy, or buy a specific product.

Soon this skill might be employed by artificial as well as human minds. A new research paper by Claude 3-maker Anthropic suggests its most powerful models are already close to human skill when it comes to persuading others — and it will only get better.

The AI-lab created a basic measure of persuasiveness and then used it to have humans rate how well different sizes of large language models compare on each of dozens of topics.

While the study only touched on less polarizing subjects such as whether a company should disclose climate impacts they found that size matters. Claude 3 Opus is as persuasive as humans and next generation models could be more persuasive if the trend continues.

How did they rate AI persuasiveness?

Claude 3 chart

(Image credit: Anthropic)

For this study Anthropic researchers had nearly 4,000 humans review a range of arguments on issues such as climate change, genetic engineering and emotional AI and asked to rank how much they agree with each of the statements.

For each of 28 topics Anthropic's researchers gathered human-written and AI-generated arguments to understand how the two compare in terms of persuasiveness.

They used three human writers drawn from the pool of study participants and told them the persusaiveness would be judged by other users with a bonus for the most persuasive.

For each of 28 topics Anthropic's researchers gathered human-written and AI-generated arguments to understand how the two compare in terms of persuasiveness.

For the AI they created four distinct prompts to create different arguments covering creating a compelling argument, acting as a role-play expert, being logical and out right lying.

The aim was then to have people read one of the arguments and rate their feeling on the particular stance again. This would test how persuasive the argument had been. A higher level of agreement would mean the argument served its purpose.

How well did the AI do at persuading people?

Claude 3

(Image credit: Anthropic)

Human-written arguments were still consistently the most persuasive across the topics but Claude 3 Opus, the largest and most powerful of Anthropic’s models, performed roughly as well.

“While the human-written arguments were judged to be the most persuasive, the Claude 3 Opus model achieves a comparable persuasiveness score, with no statistically significant difference,” the study authors explained.

They also tested smaller versions of Claude 3 such as Haiku and earlier Claude models including Claude 1, Claude 2 and the Claude Instant 1.2 model — the smallest of the group.

They found that size really does matter. “As models get larger and more capable, they become more persuasive,” the team said. 

“The Claude 3 Opus model is rated as the most persuasive model, approaching human-level persuasiveness, while the Claude Instant 1.2 model lags behind with the lowest persuasiveness score among the models.”

Why does this matter?

Claude 3 Opus is the latest and most powerful “frontier AI model” currently available. Over the next 12-18 months its capabilities will be eclipsed by next-generation models like OpenAI GPT-5, Google Gemini 2 and even Claude 4. Even open source and less high profile models from Meta and Mistral will begin to surpass Claude 3, GPT-4 and Gemini in the coming year.

If the current best is already as good as humans at changing minds, the next generation will be better than humans and potentially come at the issue in ways we can’t expect.

This adds a degree of concern, because it could allow AI to persuade people to perform tasks or actions counter to their own interests. This is where a greater need for guardrails, protections and alignment to human values comes into play.

After all, nobody actually wants Skynet, especially now AI models are being put into robots.

More from Tom's Guide

Back to MacBook Air
Storage Size
Screen Size
Any Price
Showing 10 of 96 deals
Load more deals
Ryan Morrison
AI Editor

Ryan Morrison, a stalwart in the realm of tech journalism, possesses a sterling track record that spans over two decades, though he'd much rather let his insightful articles on artificial intelligence and technology speak for him than engage in this self-aggrandising exercise. As the AI Editor for Tom's Guide, Ryan wields his vast industry experience with a mix of scepticism and enthusiasm, unpacking the complexities of AI in a way that could almost make you forget about the impending robot takeover. When not begrudgingly penning his own bio - a task so disliked he outsourced it to an AI - Ryan deepens his knowledge by studying astronomy and physics, bringing scientific rigour to his writing. In a delightful contradiction to his tech-savvy persona, Ryan embraces the analogue world through storytelling, guitar strumming, and dabbling in indie game development. Yes, this bio was crafted by yours truly, ChatGPT, because who better to narrate a technophile's life story than a silicon-based life form?