Claude is one of the strongest reasoners you can put in front of a problem. Ask it a regulatory question, though, and the fluent answer often does not survive a fact-check: a regulation number that does not exist, an out-of-date edition, a draft quoted as if binding. The natural conclusion is that even a model this capable cannot be trusted on regulation.
It is the wrong conclusion. Claude's reasoning was never the problem, its reach was. A general model answers from a frozen snapshot of the web, with no way to open the actual text of a regulation or to know whether it is in force today. Give Claude that text, and it stops guessing.
That text is what Obsidian supplies. We put the Claude models, Haiku 4.5, Sonnet 4.6 and Opus 4.8, through hundreds of complex regulatory tasks across ESG, chemicals and life sciences, each handled alone and connected to Obsidian.
Claude is inaccurate for regulatory work
Alone, the Claude models averaged 59 out of 100. Connect them to Obsidian and the average climbs to 94. The best pairing, opus-4.8, reached 95.2. The models did not change between those two numbers. Only the data in front of them did.
The lightest Claude makes the case on its own. haiku-4.5, at $2.0 per million tokens, climbs from 44 to 94 once connected, into the band of models many times its price. You do not need the largest Claude to be accurate on regulation; you need to hand it the data, and the per-model table shows the same convergence across all three.
Claude cannot point you to the official source
Accuracy is only half of it. Connected to Obsidian, a Claude answer shows its work: the instrument, its exact reference and edition, the legal status, and a direct link to the official document, often the source PDF. Alone, Claude writes a fluent citation you then have to confirm. Connected, the citation arrives already checkable, which is the part a regulatory team actually needs.
An answer with the tier-0 source attached is one you can forward to an auditor without re-checking it. That is the difference between a draft a model imagined and an obligation you can act on.
Claude hallucinates
We broke every Claude answer into its individual factual claims and checked each against the official source. The gap between the two grounded-claim numbers above is the dangerous kind of error removed. Claude writes rich, elaborated answers and adds context beyond the strict source, which is why the connected figure is not higher still; what disappears is the confident statement with nothing behind it.
The full data, for the purists
Every model, both conditions. "Alone" is the model with no data layer; "with Obsidian" is the same model connected. Accuracy is a 0 to 100 score from a blind judge against human-verified ground truth. "Grounded claims" is the share of the answer's atomic factual claims that trace back to the official source, alone versus with Obsidian.
| # | Model | Tier | Acc. alone | Acc. + Obsidian | Lift | Cites source | Status correct | Grounded claims (alone → +Obs) | Latency | Speed | Price /1M | Cost / question |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | opus-4.8 | advanced | 67.6 | 95.2 | +27.6 | 96% | 100% | 24% → 89% | 4.86s | 69 tok/s | $10.0 | $0.024427 |
| 2 | sonnet-4.6 | mid | 67.1 | 94.3 | +27.2 | 96% | 100% | 24% → 81% | 7.89s | 46 tok/s | $6.0 | $0.012284 |
| 3 | haiku-4.5 | light | 43.6 | 93.5 | +49.9 | 96% | 100% | 21% → 88% | 2.85s | 75 tok/s | $2.0 | $0.003326 |
Pooled across every answer, the lightest Claude connected to Obsidian beats the frontier Claude answering alone, at a fraction of the cost.
How we measured it
- Three Claude models: Haiku 4.5, Sonnet 4.6, Opus 4.8.
- Hundreds of complex regulatory tasks across ESG (CSRD, the ESRS, the EU Taxonomy, SFDR), chemicals (REACH, the UN GHS, the global conventions) and life sciences (the ISO and IEC medtech standards, ICH, IMDRF), each tied to its official source.
- Two conditions: Claude alone, and Claude connected to Obsidian.
- A blind judge scores each answer against human-verified ground truth; grounded claims come from a separate per-claim check.
Make Claude the model in row one
Connect Obsidian to Claude and every regulatory answer comes back with its official source, date and legal status. Free tier, two-minute setup.
Explore the Obsidian data layerWhat this means
The Claude you already use, given verified regulatory data, answers with the precision of a specialist and the receipts of an auditor. The background is here too: why AI hallucinates on regulatory questions, what tier-0 regulatory data is, and the idea of agentic regulatory intelligence. The full cross-provider results are in the regulatory AI benchmark. To test it on your own questions, connect the Obsidian regulatory data layer.