# SNOMED CT Quality Investigation using AI

**URL:** <https://forums.snomed.org/t/snomed-ct-quality-investigation-using-ai/1679>\
**Category:** AI & SNOMED CT\
**Created:** [October 18, 2026, 6:07am UTC](https://forums.snomed.org/t/snomed-ct-quality-investigation-using-ai/1679 "2026-10-18T06:07:39Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pwilliams](https://dub1.discourse-cdn.com/flex014/user_avatar/forums.snomed.org/pwilliams/32/279_2.png) [@pwilliams](https://forums.snomed.org/u/pwilliams)\
**Post date:** [October 18, 2026, 6:07am UTC](https://forums.snomed.org/t/snomed-ct-quality-investigation-using-ai/1679/1 "2026-10-18T06:07:39Z")

</div>

I was tempted to make the title “SCT AI QA” there 😄

AI offers opportunities to identify potential defects in SNOMED CT content, both retrospectively across existing editions, and incrementally as new changes are authored.

There are several aspects we could explore together:

- Detection methods: Using AI and other automated processes to identify potential quality issues in SNOMED CT content.

- Early detection: Identifying patterns that could be incorporated into authoring tools to catch issues earlier, or flag them for review.

- Human validation and feedback: Using expert review to confirm or dismiss potential issues, and feed those decisions back into the detection process.

- Operational approaches: Exploring how these techniques could be applied at scale, including across historical content and new deltas.

I’d be interested to hear from anyone experimenting with these approaches, or with ideas for other ways AI and automation could support SNOMED CT quality assurance.

If you’ve been doing work in this area, please share your methods, experiences, and findings here. In particular, it would be useful to understand _how_ the approaches work, what kinds of issues they can detect, and how we might build on them collectively.

---

<div class="post-metadata">

**Author:** ![oftalmos-org](https://avatars.discourse-cdn.com/v4/letter/o/ac91a4/32.png) [@oftalmos-org](https://forums.snomed.org/u/oftalmos-org)\
**Post date:** [October 22, 2026, 11:52pm UTC](https://forums.snomed.org/t/snomed-ct-quality-investigation-using-ai/1679/2 "2026-10-22T23:52:15Z")

</div>

Not quite SNOMED CT’s own content, but a closely related QA case that might interest this thread: I’m the author of an open-source DICOM pipeline ( [GitHub - oftalmos-org/transducin: Open-source Python pipeline for parsing the Optopol Revo FC130 .OPT and Zeiss OCT format and generating DICOM SR TID 1500 with SNOMED-CT coded ophthalmic measurements. · GitHub](https://github.com/oftalmos-org/transducin) ) that generates Structured Reports with SNOMED-CT coded findings from OCT measurements. While auditing the codebase against the real terminology server (via FHIR `$lookup`/`$validate-code`), an AI agent systematically checked ~20 hardcoded concept IDs the software claimed were SNOMED-CT codes. Result: most didn’t exist at all, and — more concerning — one that DID exist and was syntactically well-formed resolved to a completely unrelated concept (“Skin ulcer due to diabetes mellitus” instead of the intended OCT procedure code). That second category is the dangerous one: it would pass any “does this code exist” check and only surfaces if something actually compares the returned display name against the intended meaning.

Workflow that caught it: `$validate-code` for existence + `$lookup` for the returned display/FSN, diffed against the code’s inline comment/claimed meaning in source — flagged any mismatch for human review. All candidate fixes were then cross-checked by a human against the primary source (in this case DICOM PS3.16’s own CID tables, since several of the “findings” were recently-ratified DICOM concepts, not generic clinical findings).

This feels squarely in your “detection methods” + “human validation and feedback” buckets — happy to share more detail on the exact query pattern if useful, or to hear if others have run into the “valid code, wrong meaning” failure mode elsewhere.

---

<div class="post-metadata">

**Author:** ![jrogers](https://avatars.discourse-cdn.com/v4/letter/j/4491bb/32.png) [@jrogers](https://forums.snomed.org/u/jrogers)\
**Post date:** [October 23, 2026, 3:35pm UTC](https://forums.snomed.org/t/snomed-ct-quality-investigation-using-ai/1679/3 "2026-10-23T15:35:01Z")

</div>

I have seen the “_code provided by LLM is valid but does not match the term also claimed by LLM_” failure pattern many times.

The early LLMs were fairly obviously mostly inventing fake SNOMED conceptIds whenever you invited them to annotate clinical texts with SNOMED codes. Back in 2024, most of the “codes” provided didn’t even pass a Verhoeff check.

About 18 months ago LLM behaviours “improved” : the generated codes were usually real and valid codes: they passed a Verhoeff check. But I have seen cases where a “verhoeff valid” code generated by an LLM was in fact not yet part of any externally published SI or NRC release; an exceptionally advanced kind of hallucination! In roughly a quarter to a third of the remaining cases the term that the LLM _claimed_ for the code did not in fact lexically match any valid published description on that specific code. Usually it had a completely different meaning, but sometimes the meaning of the LLM-hallucinated term was quite close to that of the proper fully specified name for the conceptId .. but still did not lexically exactly match any published description on it.

For example, as part of its response to the simple prompt:

“_Please generate complete clinical data for a fictitious patient including all the fields that would be expected in a UK general practice clinical system_”

…these six code-term pairs all came back in one reply from Gemini in June 2025:

195967001 Asthma (diagnosis confirmed) (code is valid but term is not)  
235595009 Gastro-oesophageal reflux disease (GORD) (code is valid and term _almost_ is)  
37796009 Recurrent migraine without aura (code is valid but term is not)  
38907003 Chickenpox (code and term are valid)  
17296009 Appendectomy (code not valid : fails verhoeff)  
72777002 Fractured left radius (code not valid : fails verhoeff)

Today, I tried a similar prompt on Claude (Sonnet 5.5 Medium), asking it to generate 6 fictitious problem lists. The combined response shows significant further improvement:

| Problem | SNOMED CT | Code Validity | Code+Term Validity | Actual preferred term |
| --- | --- | --- | --- | --- |
| Type 2 diabetes mellitus | 44054006 | VALID | VALID | |
| Essential hypertension | 59621000 | VALID | VALID | |
| Chronic kidney disease stage 3A | 700378005 | VALID | VALID | |
| Hypercholesterolaemia | 13644009 | VALID | VALID | |
| Primary hypothyroidism | 40930008 | VALID | INVALID | 40930008 Hypothyroidism |
| Osteoarthritis of right knee | 239873007 | VALID | INVALID | 239873007 Osteoarthritis of knee |
| Obesity | 414916001 | VALID | VALID | |
| Depressive episode (single, mild) | 35489007 | VALID | INVALID | 35489007 Depression |
| Asthma | 195967001 | VALID | VALID | |
| Peanut allergy | 91935009 | VALID | INVALID | 91935009 Allergy to peanut |
| Atopic eczema | 24079001 | VALID | VALID | |
| Allergic rhinitis | 61582004 | VALID | VALID | |
| Otitis media with effusion | 65363002 | VALID | INVALID | 65363002 Otitis media |
| Bronchiolitis | 4120002 | VALID | VALID | |
| Type 1 diabetes mellitus | 46635009 | VALID | VALID | |
| Coeliac disease | 396331005 | VALID | VALID | |
| Generalised anxiety disorder | 21897009 | VALID | VALID | |
| Migraine | 37796009 | VALID | VALID | |
| Iron deficiency anaemia | 87522002 | VALID | VALID | |
| Chronic obstructive pulmonary disease | 13645005 | VALID | VALID | |
| Alcohol dependence syndrome | 7200002 | VALID | INVALID | 7200002 Alcoholism |
| Depressive episode | 35489007 | VALID | INVALID | 35489007 Depression |
| Essential hypertension | 59621000 | VALID | VALID | |
| Obstructive sleep apnoea | 78275009 | VALID | VALID | |
| Low back pain | 279039007 | VALID | VALID | |
| Current smoker | 77176002 | VALID | VALID | |
| Atrial fibrillation | 49436004 | VALID | VALID | |
| Heart failure | 84114007 | VALID | VALID | |
| Chronic kidney disease stage 3B | 700379002 | VALID | VALID | |
| Alzheimer’s disease | 26929004 | VALID | VALID | |
| Osteoporosis | 64859006 | VALID | VALID | |
| Essential hypertension | 59621000 | VALID | VALID | |
| Age-related macular degeneration | 267718000 | VALID | VALID | |
| Urinary incontinence | 165232002 | VALID | VALID | |
| Intellectual disability | 110359009 | VALID | VALID | |
| Epilepsy | 84757009 | VALID | VALID | |
| Autism spectrum disorder | 35919005 | VALID | VALID | |
| Dysphagia | 40739000 | VALID | VALID | |
| Gastro-oesophageal reflux disease | 235595009 | VALID | VALID | |
| Constipation | 14760008 | VALID | VALID | |
| Scoliosis | 20944008 | VALID | INVALID | 20944008 Congenital postural scoliosis |
| Obesity | 414916001 | VALID | VALID | |

However, a follow-on prompt perhaps reveals some of the remaining limitations: I asked Claude to generate three further problem lists, but specifically for imaginary cancer patients with complications of the disease and its treatment. Here is the combined result:

| Invasive ductal carcinoma of left breast, ER+ HER2+ | **254837009** | **VALID** | **INVALID** | **254837009 Malignant neoplasm of breast** |
| --- | --- | --- | --- | --- |
| Lymphoedema of left arm | 234097001 | VALID | INVALID | 234097001 Lymphoedema |
| Osteopenia | 312894000 | VALID | VALID | |
| Cancer-related fatigue | 84229001 | VALID | INVALID | 84229001 Fatigue |
| Chemotherapy-induced alopecia | 56317004 | VALID | INVALID | 56317004 Alopecia |
| Adenocarcinoma of sigmoid colon | 363406005 | VALID | INVALID | 363406005 Malignant neoplasm of colon |
| Secondary malignant neoplasm of liver | 94381002 | VALID | VALID | |
| Anaemia of chronic disease | 271737000 | VALID | INVALID | 271737000 Anaemia |
| Deep vein thrombosis of left leg | 128053003 | VALID | INVALID | 128053003 Deep venous thrombosis |
| Cancer-related fatigue | 84229001 | VALID | INVALID | 84229001 Fatigue |
| Weight loss | 89362005 | INACTIVE | INACTIVE | |
| Low mood | 366979004 | VALID | INVALID | 366979004 Depressed mood |
| Adenocarcinoma of prostate | 399068003 | VALID | INVALID | 399068003 Malignant neoplasm of prostate |
| Secondary malignant neoplasm of bone | 94222008 | VALID | VALID | |
| Osteoporosis due to androgen deprivation therapy | 64859006 | VALID | INVALID | 64859006 Osteoporosis |
| Erectile dysfunction | 397803000 | INACTIVE | INACTIVE | |
| Urinary incontinence, post-prostatectomy and radiotherapy | 165232002 | VALID | INVALID | 165232002 Urinary incontinence |
| Cancer-related fatigue | 84229001 | VALID | INVALID | 84229001 Fatigue |
| Anaemia of chronic disease | 271737000 | VALID | INVALID | 271737000 Anaemia |
| Type 2 diabetes mellitus (steroid and ADT related) | 44054006 | VALID | INVALID | 44054006 Type 2 diabetes mellitus |
| Hypercalcaemia | 66931009 | VALID | VALID | |

Back in Jan/Feb of 2025 we (the UK NRC) tried to engage with the likes of Microsoft on this problem, suggesting that LLM performance in this area could be improved by building in validation calls out to a reference FHIR server, but we did not manage to get any traction.