An older image AI beat frontier models in a 200-lesion skin test
CNN(see-en-en)
An AI system designed to find patterns in images.
LLM(el-el-em)
A broad AI model that works with language and other inputs.
biopsy(bye-op-see)
A test that examines a small piece of body tissue.
What happened
On October 25, dermatologist Magnus Lynch published a small AI comparison. His original report tested GPT-6 Astra, Claude Fable 5.1, and a fine-tuned Inception v3 CNN. All three systems examined the same 200 skin lesions.
Each lesion had a clinical photograph and a dermoscopic photograph. The systems chose among six diagnoses. The CNN named the correct diagnosis first in 125 cases, or 62 percent. GPT-6 Astra got 84 cases right, or 42 percent. Claude Fable 5.1 got 85 right, also 42 percent.
How the test was built
Every lesion had a diagnosis confirmed by biopsy. The sample included 100 lesions that should not be discharged. It also included 100 lesions considered safe for discharge. The first group included 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas, and 20 actinic keratoses. The second group included benign moles and seborrhoeic keratoses.
Inception v3 was first published by Google in 2015. Lynch fine-tuned all its layers on 68,011 clinical and dermoscopic images from the ISIC Archive. He removed the test lesions, and other images from the same lesions or patients, before training. The fine-tuning took about 20 minutes on one H100 GPU.
The two LLMs received both images and the same prompt. They supplied a first diagnosis, confidence, second choice, and a recommendation about discharge, biopsy, or follow-up. Each LLM case ran once, using default settings.
Why it matters
The result is not simply old AI versus new AI. It is a comparison between a model tuned for one image task and broad models used without that specialized training. A system with wide abilities may still be a poor fit for one narrow decision. The test therefore challenges the idea that newer, larger models automatically win every task.
The story also drew strong attention on Hacker News. The supplied tally lists 435 points and 206 comments. Those numbers show community interest, not scientific validity. The Hacker News thread is separate from the experiment itself.
What the results show
Allowing a second choice, the CNN included the correct diagnosis in 173 cases. GPT-6 Astra reached 134, and Claude Fable 5.1 reached 138. For melanoma, the CNN named 13 of 30 cases first. GPT-6 Astra named eight, and Claude Fable 5.1 named 11.
The safety numbers complicate the headline. At its original threshold, the CNN marked 18 of 30 melanomas as safe to discharge. When its threshold was matched to the comparison, the article reports two or one melanomas discharged. Accuracy and safety were therefore not the same measure.
What remains unclear
The sample was small, especially the 30 melanomas. It cannot settle close rankings. Each LLM answered each case once, and answers can change between runs. The MILK10k dataset became public in 2025, so the LLMs might have encountered it during training. The author says that possibility would favor the LLMs, but it still affects interpretation.
Inception v3 is also a baseline, not the best current skin-cancer system. The comparison was not a prospective clinical trial. It does not show that any of these systems should decide care for real patients.
What to watch next
Lynch released the cases, prompts, model replies, and scoring code. The next useful test would use unseen patients, repeated LLM runs, and matched safety thresholds. Independent groups could then check whether the pattern survives outside this small sample. The durable lesson is about matching a model to a task and measuring the cost of its mistakes.
An older image AI beat newer models in a skin-photo test
📰 Full story: An older image AI beat frontier models in a 200-lesion skin test
A dermatologist compared three AI systems on 200 confirmed skin lesions.
CNN(see-en-en)
An AI system made to study images.
melanoma(mel-uh-NOH-muh)
A type of skin cancer.
biopsy(bye-op-see)
A test using a small piece of tissue.
💡 The gist
- Three AI systems studied the same 200 skin lesions.
- The older image model got 125 diagnoses right.
- Hacker News attention showed interest, not proof.
What did they compare?
Dermatologist Magnus Lynch ran the comparison. He used GPT-6 Astra and Claude Fable 5.1. They are broad AI models that can inspect images. He also used Inception v3. It is a CNN, or image-focused AI system. Inception v3 first appeared in 2015. It later learned from many dermatology images.
Each lesion had two photos. One showed the skin normally. The other used a skin microscope. Every result was already checked by biopsy. The systems chose from six possible diagnoses. They also gave a second choice.
What happened?
Inception v3 got 125 of 200 diagnoses right. That equals 62 percent. GPT-6 Astra got 84 right. Claude Fable 5.1 got 85 right. Both newer models scored 42 percent. The specialist model had more practice with this task. The broad models had wider jobs. So broad ability does not guarantee the best narrow result. This test does not prove older AI always wins.
Why be careful?
The study used only 200 lesions. Only 30 were melanomas. The LLMs answered each case once. Their answers can change between runs.
At its original setting, the CNN had a serious problem. It marked 18 of 30 melanomas as safe to discharge. That means the score alone is not enough. AI should not decide whether a skin spot is safe. A doctor must make that decision.
The author released the cases, prompts, replies, and scoring code. Other researchers can test them. The next question is whether new pictures produce the same result. This was a small comparison, not a clinical trial. Read the original report for the full methods. The Hacker News discussion shows attention, not accuracy.
💬 Specialist model or LLM: who is better at skin images?
The comments give both sides: a model trained for one narrow job may win at that job, but a small, uneven comparison cannot settle the question. They also discuss tool use and why medical decisions need care.
- A CNN is a model made for images, while an LLM is built to handle many kinds of work. A specialist may win at one job, but commenters warned that the dataset was small and the model types were different, so the result should not be generalized widely.
- An LLM can act as a coordinator and call the specialist CNN or a calculation program. A harness is the surrounding setup that picks tools, orders steps, and checks the answer; its quality matters a lot.
- One commenter’s self-report said that one AI froze in a loop and crashed on a puzzle, while another setup used examples and code to solve a level. This was personal experience, not a fair benchmark.
- Another commenter’s self-report said code cut a task from about 30 minutes to 40 seconds, but a 1,000-step version took 11 hours; saving one second in the short loop meant about 20 minutes saved overall. Speed, electricity, and large-scale cost are different things.
- For medicine, some favor more visits or tests to avoid missing cancer, while others warn that unnecessary procedures and blind AI decisions can also hurt. A separate report described engineers using an LLM on spreadsheets instead of regression, while a reply said the LLM could write the regression program; either way, the tools and the checks matter.
initial digest at 206 comments (revision 1). We fetched 100 comments and sampled 100 across the thread. These are HN users’ reports, not independently verified facts.
An old picture AI guessed more skin pictures correctly
📰 Full story: An older image AI beat frontier models in a 200-lesion skin test
Three AIs looked at the same pictures. The older specialist guessed more correctly.
Inception v3(in-SEP-shun vee three)
The name of the older picture AI.
melanoma(mel-uh-NOH-muh)
A kind of skin cancer.
Dr. Magnus Lynch is a skin doctor. He tested three AIs. GPT-6 Astra is a big AI that can look at pictures. Claude Fable 5.1 is another big picture-looking AI. Inception v3 is an AI trained to notice picture patterns.
First, the doctor checked 200 skin spots. Then the AIs looked at the same spots. Inception v3 got 125 answers right. GPT-6 Astra got 84 answers right. Claude Fable 5.1 got 85 answers right.
That sounds like a win for the old AI. But one big warning matters. At its first setting, it said 18 of 30 melanomas could go home. A melanoma is a kind of skin cancer. So the old AI was not safe enough. People should ask a doctor about worrying skin spots.
Hacker News had many points and comments. That shows interest, not truth. More tests must use new pictures. You can read the doctor’s comparison. The Hacker News post shows how much attention it received.
💬 A machine that practices one thing, and an AI that does many things
The comments say the one-job machine may be better at skin pictures, but the test was small. They also say a general AI can ask other tools for help, while people must check medical answers.
- A machine that practices only skin pictures may be better at that one job. But if there are not many pictures in the test, we cannot know if it works well everywhere.
- A general AI can ask an image machine or a calculator for help. One person’s story said one AI got stuck on a puzzle, while another setup wrote code and solved it.
- Being fast is also about electricity and big jobs. For illness and numbers, people and reliable tools should check the answer, while considering both missed illness and unnecessary tests.
initial digest at 206 comments (revision 1). We fetched 100 comments and sampled 100 across the thread. These are HN users’ reports, not independently verified facts.
💬 When a specialist CNN beats a general LLM
The HN discussion around the [source article](https://lynchlab.net/llm-skin-cancer) weighs the apparent advantage of a skin-image-tuned CNN against the limits of comparing it with a general-purpose frontier LLM. The comments cover specialization, harnesses, efficiency, numerical work, and medical risk.
initial digest at 206 comments (revision 1). We fetched 100 comments and sampled 100 across the thread. These are HN users’ reports, not independently verified facts.