In regulated industries, an AI answer that cannot be explained is an AI answer that cannot be used. This post explains why knowledge graphs are the architecture that makes AI explainability possible by design rather than by retrofit and what that looks like in practice across financial services, life sciences, aerospace, and energy.
Regulated industries are deploying AI faster than the governance frameworks to support it are maturing. Organizations in financial services, pharmaceuticals, aerospace, and energy are building AI copilots, deploying automated decision support, and integrating AI into operational workflows, often before compliance teams have agreed on what “explainability” means in each context.
The regulatory direction of travel is clear. The EU AI Act, sector-specific guidance from the FCA, EMA, EASA, and equivalents in other jurisdictions, and emerging requirements in the United States all converge on a common requirement: AI systems used for consequential decisions must be able to explain those decisions in terms that can be reviewed by a human, traced to source data, and audited by a regulator.
The architecture challenge this creates is significant. Most AI systems including those built on retrieval-augmented generation (RAG) and large language models do not produce explanations as a native output. Traceability is typically retrofitted: a separate system records what data was retrieved, a separate layer attempts to reconstruct the reasoning path, and compliance teams accept a partial picture because the full picture is not available. In high-stakes decisions, partial explainability is not enough.
|
Explainability is not a layer you add to an AI system. It is a property of the architecture the AI operates on. Knowledge graphs make every decision traceable because traceability is built into how they generate answers. |
Why most AI architectures make explainability hard
The explainability gap in regulated AI is not primarily a model problem. It is a data architecture problem. Most AI systems generate answers from data that carries no formal declaration of what it means, how the entities involved relate to each other, or which rules govern their interaction. The model makes inferences from patterns. Those inferences may be correct. But they cannot be verified by tracing a reasoning path, because no formal reasoning path exists.
Consider a credit risk AI that flags a loan application as elevated risk. To explain that decision to a regulator or to the applicant, the system must be able to answer: which data contributed to this assessment, through which relationships, validated against which business rules, derived from which authoritative sources? A model that has inferred risk from statistical correlations in training data cannot answer those questions. It can provide feature importance such as a statistical summary of what the model weighted but it cannot provide a traceable chain from decision to governed source data.
|
AI approach |
How it generates answers |
Explainability limitation |
|
LLM (ungrounded) |
Statistical inference from training data patterns |
No traceability to source data. Cannot identify which data contributed or through what reasoning. |
|
RAG (document-based) |
Retrieves similar documents; LLM synthesizes answer from retrieved context |
Can cite which documents were retrieved. Cannot trace reasoning through entity relationships or validate against declared rules. |
|
Text-to-SQL |
Generates SQL from natural language; executes against database |
Query is inspectable. But limited to single-domain questions; no entity relationship traversal; no declared business rule validation. |
|
Knowledge graph+ GraphAI |
Matches question to validated ontology query; executes deterministically; returns full semantic lineage |
Every answer includes the complete chain: semantic query → entity traversal → relationship path → source data record → governing rule. |
Why knowledge graphs make explainability structural
A knowledge graph does not generate answers probabilistically. It resolves answers deterministically through declared entity relationships, governed by a formal ontology, executed on source data. Every component of that process is explicit and inspectable, which is precisely what makes the resulting answers explainable.
Entities and relationships are declared, not inferred
In a knowledge graph, the entities that matter to the business — counterparties, assets, components, clinical subjects, regulatory requirements — are formally defined. The relationships between them are declared: a counterparty is exposed to a financial instrument through a contract, governed by a regulatory classification, subject to a jurisdiction’s reporting rules. These are not statistical patterns. They are explicit facts.
When an AI system reasons over those declared facts rather than over statistical patterns, the reasoning path is the set of declared facts traversed. It is inspectable, auditable, and reproducible. A regulator asking why an answer was produced can be shown the exact entities, relationships, and rules that contributed to it.
Every answer carries graphical lineage
Kobai’s Episteme AI module produces what Kobai calls graphical lineage: a complete, visual trace from the AI answer back through the semantic query, through the entity traversal, through the relationship path, to the specific source data records in the governed Databricks Lakehouse. This lineage is not a reconstruction or an approximation. It is the actual reasoning path that produced the answer, captured as a by-product of the deterministic execution.
For compliance teams, this means the AI answer and its audit trail are produced simultaneously, not assembled after the fact. For regulated workflows where the audit trail is a prerequisite to using the answer, this changes what is operationally possible.
Business rules are explicit and auditable
Regulated decisions are not just data questions. They are rule questions: does this transaction fall within the exposure limits defined in this policy? Does this component meet the certification requirements for this asset class? Does this clinical finding meet the threshold for this regulatory submission? In a knowledge graph, those rules are declared in the ontology as constraints, thresholds, or inference rules. When an AI answer depends on a rule, the rule is part of the lineage. It can be inspected, challenged, and updated.
|
The difference between explainable AI and AI with an explanation is the difference between an architecture that produces traceability as a native output and one that attempts to reconstruct it afterwards. Knowledge graphs produce traceability by design. |
What regulators are requiring and where AI systems are falling short
The regulatory requirements for AI explainability are becoming more specific across multiple jurisdictions and sectors. Understanding what is being required helps clarify why the architecture matters, not just the model.
|
Regulatory context |
What is being required |
The gap in current AI architectures |
|
EU AI Act (High-risk AI) |
Providers of high-risk AI must enable meaningful explanation of outputs to affected individuals and competent authorities. Systems must be designed for human oversight. |
Most LLM-based systems cannot provide a deterministic, verifiable reasoning path from output to source data. Feature importance scores do not meet the meaningful explanation standard. |
|
Financial services (FCA, DORA, SR 11-7) |
Model risk management frameworks require model explainability, documentation of decision logic, and the ability to trace outputs to data lineage. |
RAG and text-to-SQL systems can cite sources but cannot trace through entity relationship chains. Cross-domain decisions (customer + counterparty + exposure + jurisdiction) are particularly difficult to explain. |
|
Life sciences / pharma (EMA, FDA 21 CFR Part 11) |
AI used in regulatory submissions, clinical decision support, or quality systems must demonstrate validated, auditable, and reproducible reasoning. |
LLM outputs are probabilistic and non-reproducible. The same question asked twice may produce different answers — which is incompatible with validated system requirements. |
|
Aerospace (EASA, FAA, DO-178C) |
Safety-critical systems require deterministic, verifiable behaviour. AI-assisted maintenance, design, or certification decisions must trace to authoritative source data. |
Unstructured AI inference is not compatible with safety-critical certification requirements. Deterministic, rule-based execution with full auditability is required. |
|
The reproducibility requirement Many AI systems, including those built on RAG, produce non-deterministic answers: the same question asked twice may return different answers depending on retrieval conditions, model temperature, and context window composition. In regulatory environments that require validated, reproducible AI outputs — pharmaceutical quality systems, aerospace certification, financial model validation — this non-determinism is not a theoretical concern. It is a compliance disqualifier. Deterministic execution on a governed knowledge graph produces the same answer to the same question every time, which is the baseline requirement for a validated system. |
Explainable AI by sector: what changes in practice
|
Financial services: explainable credit, risk, and compliance decisions In financial services, AI is increasingly used to support credit decisions, counterparty risk assessment, anti-money laundering investigations, and regulatory reporting. In each of these workflows, the output must be explainable to multiple audiences: the affected customer (under consumer protection requirements), the model risk management function (under SR 11-7 or equivalent), and the regulator (under applicable reporting frameworks).A knowledge graph grounded in the Financial Industry Business Ontology (FIBO) or an organization-specific equivalent models the entities — counterparties, instruments, exposures, jurisdictions, obligations — and the relationships between them. When an AI system assesses counterparty risk, it traverses those declared relationships: counterparty → legal entity → exposure by instrument class → jurisdiction → applicable reporting requirement. The answer carries the full traversal as its explanation. Every component can be audited, and the answer is reproduced identically on re-run. |
|
Life sciences and pharma: compliant AI across the product lifecycle Pharmaceutical organizations handle data across clinical trials, manufacturing, regulatory submissions, and post-market surveillance. Each of these domains has strict data governance requirements — 21 CFR Part 11, GxP, EU Annex 11 — and AI systems that operate across them must demonstrate that their outputs are traceable, reproducible, and derived from authorized source data. Kobai has deployed this architecture for a pharmaceutical organization operating on the Databricks Lakehouse. Clinical, manufacturing, and regulatory data was unified in a semantic knowledge graph, with AI answers traced through the graph back to source Delta records governed by Unity Catalog. The compliance team accepted the AI assistant for a regulated workflow because every answer included its complete audit trail. This was the first explainable AI system approved for a regulated workflow at this organization. |
|
Aerospace and defence: traceable decisions across the digital thread In aerospace, the digital thread — the connected chain of data from design through manufacturing, maintenance, and service — is the foundation of airworthiness and compliance. When an AI system assists with a maintenance decision, a configuration change, or a failure investigation, the decision must be traceable to the specific parts, aircraft, maintenance records, and certification data that informed it. A knowledge graph connecting design specifications, manufacturing records, component genealogy, maintenance events, and regulatory requirements provides the structured foundation for traceable AI decisions. When an in-flight sensor flags an anomaly, the AI can traverse from the specific component to its manufacturing batch, to other aircraft sharing components from the same batch, to the applicable maintenance requirement, to the available certified engineers — with every step of the traversal recorded as part of the answer’s lineage. |
|
Energy and utilities: regulatory compliance and operational safety Energy operators face dual explainability requirements: regulatory reporting obligations (environmental, safety, market) and operational safety decisions where AI-assisted recommendations must be traceable to authoritative data. A recommendation to extend an inspection interval, defer a maintenance intervention, or approve a configuration change carries significant liability if it cannot be shown to have been derived from authorized, current, and accurate data. A semantic knowledge graph connecting asset technical specifications, inspection history, regulatory requirements, operating conditions, and workforce certifications provides the traceable foundation for AI-assisted operational decisions. Every recommendation carries its reasoning chain — the specific asset state, the applicable regulatory threshold, and the supporting data — as an auditable output rather than a model confidence score. |
How graphical lineage works in practice
Kobai’s Episteme module provides what is described internally as GraphAI — a deterministic, ontology-grounded approach to AI question answering that produces graphical lineage as a native output rather than a feature to be enabled.
When a user asks a question, Episteme:
Because the answer is the product of a deterministic execution against declared facts rather than probabilistic synthesis from retrieved context, it is reproducible. The same question, asked on a subsequent occasion with the same underlying data, produces the same answer through the same reasoning path. This reproducibility is what makes Episteme answers acceptable for validated regulatory workflows.
|
Episteme graphical lineage chain User question → Matched semantic query → Ontology validation → Entity traversal path → SQL execution on Databricks → Answer + full lineage Every component of this chain is captured, inspectable, and exportable. The lineage is not a summary of what happened — it is the actual execution path, available for review at any point. |
What explainability enables beyond compliance
The immediate driver for AI explainability in regulated industries is regulatory compliance. But the organizations that have deployed explainable AI architectures report that the operational benefits extend well beyond satisfying a regulatory requirement.
Compliance teams accept AI-assisted workflows they would otherwise reject
The most common obstacle to AI adoption in regulated organizations is not technical capability, it is trust. Compliance teams that cannot inspect how an AI answer was produced will not accept it in a regulated workflow, regardless of how accurate it is. When every answer carries its reasoning chain as a native output, the compliance team becomes a reviewer of AI outputs rather than an obstacle to AI adoption.
Model risk management becomes tractable
Financial services organizations operating under model risk management frameworks (SR 11-7 in the US, equivalent frameworks elsewhere) must document, validate, and monitor every model used for consequential decisions. An AI system that produces deterministic, traceable outputs is substantially easier to validate than one that produces probabilistic answers from a process that cannot be fully described. The audit trail that graphical lineage produces is also the model documentation that model risk management requires.
Disputed decisions can be explained and defended
When an AI-assisted decision is challenged — by a customer, a counterparty, a regulator, or an internal review — the ability to produce the complete reasoning chain from decision to source data is the difference between a defensible position and an exposed one. Knowledge graph-grounded AI produces that chain as a standard output, not as something to be reconstructed under pressure.
|
The pharmaceutical organization that deployed Kobai’s Episteme AI for a regulated workflow described it as “the first explainable AI our compliance team actually accepted.” That acceptance was not a concession, it was the natural result of an architecture that produces traceability by design. |
Explainable AI that compliance teams trust
Kobai is purpose-built for complex, regulated industries where enterprise data is fragmented across many systems and AI decisions must be explainable. The semantic knowledge graph runs directly on the Databricks Lakehouse — on governed Delta tables, under Unity Catalog access controls, with graph traversals executed on Databricks compute. Domain experts define the ontology in Kobai Studio. Episteme delivers AI answers grounded in that ontology, with graphical lineage from answer to source as a native output.
For regulated industries, this architecture means AI that compliance teams can review, model risk management functions can validate, and regulators can audit without a separate explainability layer bolted on after the fact. Explainability is not a feature of Kobai’s AI. It is a property of how the AI is built.
To explore how a knowledge graph architecture on Databricks can support explainable AI in your regulated environment, speak with us at kobai.io or contact us at contact@kobai.io.