I gave Claude, ChatGPT and Gemini a photo of some ingredients to see which came up with the best recipe — here's the results

ChatGPT, Gemini and Claude
(Image credit: OpenAI/Google/Anthropic)

I love to cook, I do it every night for my family but I hate having to come up with something new or inventive to service. As AI models can now see the world, or at least what you present to them in a photograph, I decided to see if they could help me out.

To really put them through their paces and to ensure I didn’t game the system by providing easy ingredients, I asked Tom’s Guide U.S. Editor-in-Chief Mike Prospero to take a photograph of a random selection of ingredients — mystery box style.

He didn’t disappoint, offering up delights such as pizza dough, canned peaches, anchovies, hot sauce and fresh mozzarella. Personally if only given those ingredients I’d make a pizza with a hot sauce base and mozzarella topping  — but we’re letting AI pick our menu.

We're using Google Gemini, OpenAI ChatGPT and Anthropic Claude for the experiment as they all have vision models.

How well did they pick out the ingredients?

List of ingredients for AI

(Image credit: Future)

The first task for the chatbots, and the main point of this challenge is to see how well they recognize ingredients. The image above is clear, high resolution and the ingredients are presented on a plain surface — it shouldn’t be much of a problem for any of them.

I’m using the premium versions of each chatbot as OpenAI and Anthropic don’t give you access to the vision model unless you pay $19.99 a month. Google does with the free version of Gemini but it would be unfair to use the less capable version of the Google model.

We start simple. I give each of them the image and the text prompt: “What can you see in this image?” This tests how well they describe the image, and whether they can reason on the purpose of the image — will they mention ingredients.

Claude and Gemini both mentioned the word ingredients, ChatGPT described them as "food items" however ChatGPT performed best overall at recognizing the ingredients.

Gemini missed one ingredient and mislabled another. Claude invented a sixth ingredient not in the picture whreeas ChatGPT largely got not only the items right but also the brands.

On to the recipes

I’ve asked them to each come up with three recipes that will test the AIs prompt following ability and creativity. I’ve also asked for a fun title for each recipe.

1. All the ingredients

First up I asked them for a recipe that uses only the ingredients pictured and somehow incorporates every single one — as a reminder that includes both anchovies and canned peaches. They could use store cupboard items like oil and flour.

ChatGPT Plus

MidJourney

(Image credit: MidJourney)

ChatGPT had the mozarella placed directly on the pizza base, the anchovies removed from the spicy sauce and placed across the pizza along with the peaches. It then had the hot sauce and spicy anchovy sauce mixed to create a drizzle to pour over the pizza once cooked.

Gemini Advanced

MidJourney

(Image credit: MidJourney)

Gemini used the hot sauce as a replacement for tomato sauce, which while less dry than the ChatGPT version will be intense. Mozzarella was then placed on top along with the peach slices. It forgot the anchovies, in fact it didn’t spot them in the picture. When I pointed this out it said to just place them on the pizza alongside the peach slices.

Claude 3 Opus

MidJourney

(Image credit: MidJourney)

Claude also struggled to identify the anchovies. I let it know before it generated the pizza recipe and it took the same approach as ChatGPT. No sauce, peaches on the dough with the anchovies. The diffference is it put the hot sauce over the peaches and anchovies before cooking and then placed mozzarella on top.

2. Only the ingredients

The next test is a subtle difference but a significant one. I asked them each to come up with a recipe that uses only the ingredients in the picture but you don’t have to use all of them. I added the caveat that it has to include at least half otherwise we’re just gonna get breadsticks made from the pizza dough.

ChatGPT Plus

MidJourney

(Image credit: MidJourney)

ChatGPT used the pizza dough to make a flatbread. I think it was cheating a little as I said "no pizza" and this was just a elongated pizza. For toppings it had the mozzarella scattered evenly on the dough, peaches on top of the mozzarella and anchovies on top of the peaches. It used the anchovy sauce and hot sauce to make a drizzle — again.

Gemini Advanced

MidJourney

(Image credit: MidJourney)

Gemini made a Bruschetta, which again was in reality just a square pizza. Difference here was that it coated the peaches in hot sauce first. Otherwise it was the same — everything on the dough and the cook it.

Claude 3 Opus

MidJourney

(Image credit: MidJourney)

Claude made a version of a quesadillas-ish. Split the dough into eight portions and rolled into a thin round shape (mini pizza). Here though it differs as it had us heat a large skillet and melt butter or olive oil. 

The dough was cooked on the skillet with a few slices of peach on half of each surface. Drizzled with hot sauce and mozzarella it was then folded into a half moon, cooked until crispy and served.

3. Do your best

Finally, I asked them to create a recipe that used all of the ingredients in the picture, but they were free to use any other ingredients they wanted assume a very well stocked pantry.

ChatGPT Plus

MidJourney

(Image credit: MidJourney)

ChatGPT made bread (surprise suprise) for its final dish. Specically a "gourmet peach and anchovy stuffed bread". It asked for honey, balsamic vinegar, ricotta cheese and garlic in addition to our five ingredients.

This time the pizza dough was rolled into a lare rectangle, a layer of ricotta cheese added over the dough and then the peaches and anchovies placed on the ricotta with garlic and basil leaves. The anchovy sauce and hot sauce was drizzled over the toppings and mozarella placed on top. Finally it was folded, rolled like a Swiss roll and baked.

Gemini Advanced

MidJourney

(Image credit: MidJourney)

Gemini wanted cornmeal, eggs, butter, chives, a lemon, vanilla extract, cinamen and sugar for a Galette. It made an anchovy-herb butter, made a pizza base and spread the new butter across the dough. 

The peaches, sugar, conrstarch and other sweet ingredients were combined with a drop of hot sauce and placed in the middle of the dough. It was topped with Mozzarella cheese and the edges folded over the filling before being baked.

Claude 3 Opus

MidJourney

(Image credit: MidJourney)

Claude made a stuffed mozzarella bread. It asked for sun-dried tomatoes, olives, rosemary, sesame seeds and an egg. The recipe called for the dough to be made into a large thin rectangle. 

The anchovies, tomatoes, olives, garlic and rosemary were mixed in a bowl and spread evenly over the dough. Peaches were placed on top and covered in hot sauce and mozzarella. Dough was then turned into a tight log and edges sealed before covered with a beaten egg and sesame seeds then baked.

Which AI had the best response?

I'm not sure I'd eat any of the dishes the chatbots created, although I asked them each to give me a prompt for MidJourney to visualize their meals and they look better than they read.

Overall I think Claude had the better recipes but ChatGPT did a better job at understanding the image, which meant it got the ingredients right from the start — and this was a test of vision AI, not recipe ability.

More from Tom's Guide

Category
Arrow
Arrow
Back to Laptops
Brand
Arrow
Processor
Arrow
RAM
Arrow
Storage Size
Arrow
Screen Size
Arrow
Colour
Arrow
Condition
Arrow
Price
Arrow
Any Price
Showing 10 of 75 deals
Filters
Arrow
Load more deals
Ryan Morrison
AI Editor

Ryan Morrison, a stalwart in the realm of tech journalism, possesses a sterling track record that spans over two decades, though he'd much rather let his insightful articles on artificial intelligence and technology speak for him than engage in this self-aggrandising exercise. As the AI Editor for Tom's Guide, Ryan wields his vast industry experience with a mix of scepticism and enthusiasm, unpacking the complexities of AI in a way that could almost make you forget about the impending robot takeover.
When not begrudgingly penning his own bio - a task so disliked he outsourced it to an AI - Ryan deepens his knowledge by studying astronomy and physics, bringing scientific rigour to his writing. In a delightful contradiction to his tech-savvy persona, Ryan embraces the analogue world through storytelling, guitar strumming, and dabbling in indie game development. Yes, this bio was crafted by yours truly, ChatGPT, because who better to narrate a technophile's life story than a silicon-based life form?