Claude 2 is a powerful new ChatGPT rival — and I just tried it

Gen AI Chatbot Claude 2
(Image credit: Anthropic)

ChatGPT has a new challenger to look out for in the form of Claude 2, billed by its creators as a ‘friendly, enthusiastic colleague or personal assistant.’

San Francisco-based AI company Anthropic has launched the second generation of its natural language generation chatbot, hitting the market with the bold claim it's smarter than 90% of the college students in the United States and Canada.

According to Anthropic, Claude 2 performs better, can give longer responses, and has improvements when it comes to coding, math, and reasoning. Claude 2 can also handle 100 tokens of input or output, equating to up to 75,000 words of information, which means it can process and generate anything from simple questions to complex reports.

In the real world, Anthropic says Claude 2 scored 76.5% on the multiple choice section of the Bar exam, up from 73.0% with Claude 1.3, and scores above the 90th percentile on the GRE reading and writing exams compared to college students applying to graduate school.

I tried it

Using Claude 2 is pretty simple and given Anthropic want to license the chatbot to businesses, maybe that shouldn't be surprising. However in the limited time I've used it, Claude 2 was able to quickly answer my questions.

It also seemed to remember how I like my information to be presented, opting for a series of bullet points for all future questions after I asked it to simplify a lengthy answer.

Graphical representation of a cybernetic brain

(Image credit: Shutterstock)

Claude 2 has apparently improved its coding skills, scoring 71.2% on the Python coding test, the Codex HumanEval, whereas the first generation could only reach 56.0%. On GSM8k, a large set of grade-school math problems, Claude 2 scored 88.0% up from 85.2%. 

Anthropic also said “an exciting roadmap of capability improvements” is planned for Claude 2. If you want to test these capabilities, Anthropic launched a public beta of Claude 2 for residents in the U.S. and U.K.

Overall the experience is simple. The answers I received were correct, and personalized, but it will interesting to see how far we can test Claude with more time.

New safety techniques

Smartphone displaying skull and crossbones on screen.

(Image credit: Morrowind/Shutterstock)

The new chatbot is one of the first to take advantage of the new safety techniques that Anthropic announced over the last year, in order to improve results and prevent dangerous usages of generative AI.

These techniques include training models with reinforcement learning from human feedback (RLHF) so that Claude 2 has the capability to "morally self-correct", as well as Constitutional AI, which can identify an inappropriate request and explain why it will not engage with the request. 

As a result, Claude 2 is two times better at giving harmless responses than Claude 1.3, its predecessor.

man sat at darkened desk working on laptop and desktop

(Image credit: Shutterstock)

The effort to make chatbots safer to use may please Government bodies who are increasingly focused on potential risks from the rise of generative AI models like chatbots and deepfakes. 

The White House has outlined a blueprint for AI regulation covering areas like bias, privacy and misinformation. In Europe, the E.U, has proposed stricter rules for high-risk AI systems, with transparency and oversight safeguards, and the FTC has also warned companies to prevent deception from generative AI, citing truth-in-advertising laws. 

Regulators in the U.K., Australia and elsewhere have also raised concerns about personal data used to train models and misuse generating harmful content, with lawmakers considering new laws to prevent spreading misinformation, impersonation fraud and other harms.

Accuracy has been a focus for lawmakers and regulators since the release of ChatGPT in late 2022, and amassed over a million users who had signed up to try the AI chatbot. Inspired by its popularity, other tech giants have launched their own massive generative language models like Google's Bard, SoundDrown from Meta, and Bing Chat from Microsoft. Smaller startups are also joining the race to offer creative generative AI apps for writing, art and coding.

More from Tom's Guide

Elliot Mulley-Goodbarne
Phones Writer

Elliot MulleyGoodbarne is a Staff Writer at Tom’s Guide covering news, reviews, buying guides with wit and charm. He has had a keen interest in mobile, tablets, laptops, and everything in between since his first phone, the Samsung U600. Starting at Mobile News, Elliot has been writing about mobile and the business of technology since 2016, and has edited four trade magazines in that time, attending award shows for three of them. He has attended a range of industry events including 5G launches, device announcements, and trade events, where he developed a nose for picking up on trends, interviewing executives, and networking with the tech community. In his spare time, Elliot can usually be found watching, playing, or attending sporting events. With season tickets to West Ham United and Surrey County Cricket Club he often spends weekends in winter in a foul mood and evenings in the summer with a smile on his face. Outside of sport Elliot enjoys cooking, pubs next to rivers, and chilling out with a G&T, Whiskey, or Port after a solid week of excellence.