Point your phone at it: what AI can and can't tell you from a photo

Short answer

A phone's AI is good at reading what is in a photo and weak at deciding what it means for your safety. Trust it to read, translate and explain. Check it when it names a living thing or a part. Never eat, swallow, touch or treat anything on its word alone.

For turning receipts and forms into tables, see Photos in, data out. For whether a picture is real, see Was this image made or changed by AI?.

The traffic light

Light	What you show it	Why	Safer next step
Green	Text to read or translate: a menu, a sign, a letter	You can check the words by eye. OpenAI says ChatGPT is weaker on alphabets such as Japanese or Korean and on rotated text.	Read key words back against the photo. For a medical or legal letter, see Health, money and legal questions.
Green	An error screen or warning light	It can read the code and explain it in plain words.	Search the exact code on the maker's support site.
Green	A model number or label, to find the manual	The label is printed, so it is easy to check.	Download the manual from the maker's own site.
Green	"What is in front of me?"	Useful, not perfect: in a study with blind users, an app was wrong 22.2% of the time.	Ask again from another angle for anything that matters.
Amber	A plant, a bird, a spare part, a stain	It may give one confident name when several things look alike.	Ask for two or three options. Check against a field guide, a parts list or an expert.
Red	Wild mushrooms, berries or plants you might eat, or a plant a child or pet has chewed	ID apps tested on real poisoning cases were too often wrong (see below).	US: Poison Help, 1-800-222-1222. UK: 999 or NHS 111 as above. Elsewhere: your poison centre.
Red	A loose pill or tablet	We found no study of chat assistants on loose pills, and a wrong name can mean a wrong dose.	Ask a pharmacist. If someone has swallowed it, use the help line above.
Red	A rash, a mole or a wound	Doctors rated a chatbot's answers on skin photos as poor (see below).	See a doctor or nurse. Use the chat to draft your questions.
Red	Sparks, burning smells, scorched sockets or a gas smell	A photo cannot show what is behind the wall or in the pipe.	A qualified electrician or gas engineer. Gas smell in Great Britain: 0800 111 999 (free, 24 hours). Elsewhere: your gas emergency line.

What the studies found

Mushrooms. In Victoria, Australia, three apps were tested on photos of 78 mushrooms from real poisoning enquiries. The best app was right 49% of the time overall and 44% for poisonous species. Apps named the death cap (Amanita phalloides) correctly 27% to 67% of the time (Hodgson et al., Clinical Toxicology, 2023).
Toxic plants. In a US test of three apps on 17 common toxic plants, the best one identified 10 of the 17 every time (59%), and the worst only 1 (Otter et al., Journal of Medical Toxicology, 2021). In Queensland, six apps including Google Lens were tried on 25 real plant exposure cases. The best was right 74% of the time, and the authors said none is yet reliable enough to guide poisons advice (McNeill et al., Toxicology Communications, 2024).
Skin. 23 doctors, most of them consultant dermatologists, rated answers from GPT-4 (the AI behind paid ChatGPT at the time) on 15 clear skin photos. The median overall score was 2 out of 5 (Nielsen et al., JEADV Clinical Practice, 2024).
Right answer, wrong reasons. On a medical image quiz, GPT-4V chose the right answer 81.6% of the time, against 77.8% for doctors. But when it was right, its reasoning was flawed 35.5% of the time, most often in reading the image (27.2%) (Jin et al., npj Digital Medicine, 2024).

The last study is why the steps below ask what it sees before what it concludes.

The steps

These steps are our suggestion. No study tests them.

Take a useful photo. Use daylight or bright, even light. Take several angles: top, underside, the whole thing and a close-up. Put a coin or ruler beside it for scale. Include any label, code or packaging. Crop out faces, addresses and papers: What not to paste.
Ask what it sees first. Ask for a plain description before any name. You can check a description against the real thing.
Ask for two or three possibilities. Ask what would tell them apart and which ones are dangerous.
Check against a named source. Look up the top possibility in an official or expert source: the maker's manual, a field guide, a poison centre, a pharmacist. A second AI answer is not a check: Check before you trust.
Stop at red. For anything in the red rows, take the photo and the description to the person named in the table.

Attach your photos and paste this:

Photo prompt · v1.0

I have attached <NUMBER> photos of <WHAT_IT_IS>, taken in <WHERE>.
<SCALE_REFERENCE> is in the shot for size.

1. First, describe only what you can see: colour, shape, size, markings,
   any text. Say what you cannot see or cannot read. Do not name it yet.
2. Then give the two or three most likely possibilities. For each, say
   what feature in the photo supports it and what would rule it out.
3. Tell me what extra photo or detail would tell them apart.
4. Say if any possibility is poisonous, dangerous or needs a
   professional, and which official source I should check it against.

I want to <MY_GOAL>. Do not tell me it is safe to eat, take or touch.

Replace the following:

<NUMBER>: how many photos, such as 4.
<WHAT_IT_IS>: a plain description, such as a plant by our back door (illustrative).
<WHERE>: country and setting, such as a garden in Kent, UK (illustrative).
<SCALE_REFERENCE>: what is beside it, such as a 1p coin.
<MY_GOAL>: your next step, such as decide whether to ask a garden centre about it (illustrative).

Which tools do this

Their own help pages say: Google Lens (Google app, some Android cameras) translates text "in over 100 languages" and names plants and animals. ChatGPT takes photos on Free and paid plans, and OpenAI says it "shouldn't be used for medical advice". Gemini Live on Android lets you turn on the camera and talk. Claude takes uploaded photos and suggests at least 1000 by 1000 pixels. Apple Visual Intelligence works on compatible Apple devices set to a supported English. Whether to use AI at all: When not to use AI.

How to tell it worked

The description in step 2 matches what you see, with nothing invented.
You have two or three possibilities and know what separates them.
You checked the top one in a named source, or took a red item to a person.

Sources

Hodgson SE, McKenzie C, May TW, Greene SL. "A comparison of the accuracy of mushroom identification applications using digital photographs." Clinical Toxicology, 2023. Benchmark (peer-reviewed). Link
Otter J, Mayer S, Tomaszewski CA. "Swipe Right: a Comparison of Accuracy of Plant Identification Apps for Toxic Plants." Journal of Medical Toxicology, 2021. Benchmark (peer-reviewed). Link
McNeill I, Knoeckel R, Goggin A, Wylie C, Isoardi K. "Accuracy of plant identification applications to identify plants in suspected poisoning cases referred to the Queensland Poisons Information Centre." Toxicology Communications, 2024. Field study. Link
Nielsen JPS, Grønhøj C, Skov L, Gyldenløve M. "Usefulness of the large language model ChatGPT (GPT-4) as a diagnostic tool and information source in dermatology." JEADV Clinical Practice, 2024. Benchmark (peer-reviewed). Link
Jin Q et al. "Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine." npj Digital Medicine, 2024. Benchmark (peer-reviewed). Link
Gonzalez Penuela et al. Diary study of an AI visual-interpretation app with blind and low-vision users. CHI 2026. Field study. Link
HRSA. Poison Help. Link and National Capital Poison Center. Link
NHS. Poisoning. Link
SGN. What to do if you smell gas. Link
Google. Lens. Link; OpenAI. ChatGPT image inputs FAQ. Link; Google. Gemini Live help. Link; Anthropic. Uploading files to Claude. Link; Apple. Apple Intelligence. Link. Vendor.

Changes

v1.010 Oct 2026first version