Ask ChatGPT a regulatory question and the answer comes back fast and self-assured. Then you check it: a regulation number that does not exist, a superseded edition, a draft quoted as binding. After enough of these, the verdict feels obvious: ChatGPT is not ready for regulatory work.

It is the wrong verdict. The GPT models people use are perfectly capable of regulatory reasoning. What fails them is reach: a general model answers from a frozen snapshot of the web, with no way to open the actual text of a regulation or to know whether it is in force today. Give it that text, and it stops guessing.

That text is what Obsidian supplies. We put the GPT models, GPT-5.4-nano, GPT-5.4-mini and GPT-5.5, through hundreds of complex regulatory tasks across ESG, chemicals and life sciences, each handled alone and connected to Obsidian.

54 → 95
Average regulatory accuracy, the same models alone vs. connected (out of 100)
35% → 96%
Share of an answer's factual claims grounded in the official source
95%
Connected answers that cited the correct official source

ChatGPT is inaccurate for regulatory work

Alone, the three GPT models averaged 54 out of 100. Connect them to Obsidian and the average climbs to 95. The best pairing, gpt-5.4-mini, reached 95.5. The models did not change between those two numbers. Only the data in front of them did.

Regulatory accuracy versus price per 1M tokens
Regulatory accuracy against price. Connected to Obsidian (the wider coins), every model converges near the top.
Regulatory accuracy versus average response time in seconds
The same against response time.

The standout is the smallest one. gpt-5.4-nano, at $0.463 per million tokens, climbs from 42 to 96 once connected, into the band of models many times its price. On regulation the data layer outweighs the model size: you hand the data to a small, fast model rather than buying a bigger one, and the per-model table shows the same convergence across all three.

ChatGPT cannot point you to the official source

Accuracy is only half of it. Connected to Obsidian, a GPT answer shows its work: the instrument, its exact reference and edition, the legal status, and a direct link to the official document, often the source PDF. Alone, you get a plausible citation to verify yourself. Connected, it arrives already checkable, which is the part a compliance workflow actually needs.

An answer with the tier-0 source attached is one you can forward to an auditor without re-checking it. That is the difference between a draft a model imagined and an obligation you can act on.

ChatGPT hallucinates

We broke every GPT answer into its individual factual claims and checked each against the official source. The gap between the two grounded-claim numbers above is the dangerous kind of error removed: the confident statement with nothing behind it. The ungrounded remainder is added context around the source, not fabricated references.

The full data, for the purists

Every model, both conditions. "Alone" is the model with no data layer; "with Obsidian" is the same model connected. Accuracy is a 0 to 100 score from a blind judge against human-verified ground truth. "Grounded claims" is the share of the answer's atomic factual claims that trace back to the official source, alone versus with Obsidian.

#ModelTierAcc. aloneAcc. + ObsidianLiftCites sourceStatus correctGrounded claims (alone → +Obs)LatencySpeedPrice /1MCost / question
1gpt-5.4-minimid69.595.5+26.096%100%38% → 96%1.25s84 tok/s$0.7$0.000966
2gpt-5.4-nanolight42.595.5+53.094%99%28% → 96%1.42s83 tok/s$0.463$0.000551
3gpt-5.5advanced49.894.4+44.696%100%44% → 96%4.89s42 tok/s$11.25$0.0167

Pooled across every answer, even the smallest GPT connected to Obsidian beats the frontier GPT answering alone, for less than a cent a question.

How we measured it

  • Three GPT models: GPT-5.4-nano, GPT-5.4-mini, GPT-5.5.
  • Hundreds of complex regulatory tasks across ESG (CSRD, the ESRS, the EU Taxonomy, SFDR), chemicals (REACH, the UN GHS, the global conventions) and life sciences (the ISO and IEC medtech standards, ICH, IMDRF), each tied to its official source.
  • Two conditions: GPT alone, and GPT connected to Obsidian.
  • A blind judge scores each answer against human-verified ground truth; grounded claims come from a separate per-claim check.

Make ChatGPT the model in row one

Connect Obsidian to ChatGPT and every regulatory answer comes back with its official source, date and legal status, often for less than a cent a question. Free tier, two-minute setup.

Explore the Obsidian data layer

What this means

The GPT you already use, given verified regulatory data, answers with the precision of a specialist and the receipts of an auditor. The background is here too: why AI hallucinates on regulatory questions, what tier-0 regulatory data is, and the idea of agentic regulatory intelligence. The full cross-provider results are in the regulatory AI benchmark. To test it on your own questions, connect the Obsidian regulatory data layer.