When the Data Is Right and the Truth Is Still Missing
Most organizations invest heavily in data quality. Far fewer ask whether their data models actually represent the business.
Some of the most sophisticated data environments I have ever worked in belong to large financial institutions. Two of them, the two largest Swiss banks, stand out because both organizations invested seriously and successfully in data governance, with documented lineage, validated values, and audit trails that would satisfy any regulator. They had genuinely achieved a level of operational data quality that is aspirational for many organizations.
And yet they were no better positioned than those aspirational organizations when tackling significant analytical efforts. Those projects required something that no amount of quality and governance investment could provide: a working knowledge of how the data had been constructed, what edge cases the models had silently absorbed, and which institutional practices were encoded not in any system but in the heads of the people who had been there long enough to know. Deriving an accurate picture of what was actually happening in the business required extensive enrichment, careful transformation, and a kind of expert interpretation that felt more like archaeology than analysis.
For a long time I assumed this reflected a governance gap, that more documentation, better metadata, and stronger standards would close it. The more I worked through these environments, though, the more a different diagnosis emerged: the underlying models were not faithfully representing their domains, and all the governance in the world could not compensate for that.
Data quality vs. representational adequacy
Data quality, as the field usually defines it, concerns the accuracy and reliability of individual values: whether a field is populated, whether a number is correct, whether a record reconciles to another. Representational adequacy is a different kind of question. It asks whether the model, the collection of tables, fields, and relationships that defines what gets stored, actually captures the structure of the business reality it is meant to describe. A system can achieve impeccable data quality by every conventional measure and still fall well short of this.
The clearest way to see this is through an example. A trading system records each trade as a single record, and among the fields it captures is the salesperson who originated the deal. This is a natural and useful thing to store, and for years it works exactly as intended. Then the business evolves: salespeople begin collaborating on larger transactions and splitting the commissions, and the practical solution is to book two trade records rather than one, crediting each salesperson individually. Each record is accurate. The salesperson is correctly identified, the commission correctly calculated, and an auditor reviewing either record would find nothing to question.
What has quietly broken is the model’s correspondence with the underlying reality. A trade record now serves two purposes simultaneously: it is both a record of a transaction and a unit of compensation credit. Anyone who later wants to count transactions with a given counterparty, or analyze trading volume in a particular product, will find numbers they cannot trust without first knowing this history and working around it manually. No individual value is wrong, but the model has been stretched past the point where it reliably represents what it was built to represent.
When the transaction disappears
A structurally more complex version of this problem appeared at one of those Swiss banks in the treatment of repurchase agreements, or repos. A repo is a short-term financing transaction in which one party sells securities (typically government bonds) to another, with a simultaneous agreement to repurchase them on a specified future date at a slightly higher price. The difference between the sale price and the repurchase price is effectively interest, and the securities serve as collateral for what is, in economic substance, a short-term loan.
The bank’s system recorded repos as two separate trades: the initial sale and the forward repurchase. Both legs were captured faithfully and accurately. But the system held no representation of the repo itself as a unified transaction. There was no link between the two legs, no encoding of the agreed repurchase price as a rate of return, and no concept of a collateral relationship connecting them. To calculate the implied interest rate on any given repo, an analyst had to know to locate both legs and perform the arithmetic manually. To track whether collateral had been substituted during the life of the transaction, which is a common event in active repo books, required institutional knowledge of how substitutions would have been recorded, because the system had no model of substitution at all. The individual values were accurate in every respect, but the repo itself, as a coherent object with a term, a rate, and a collateral relationship, had no place in the system.
What these two examples have in common is that competent people made reasonable decisions. Recording two trades for a split commission is a pragmatic solution to a genuine business need. Recording both legs of a repo is technically accurate as far as it goes. The failure in each case is more subtle, and harder to catch in the moment: neither approach considered whether the model, as extended, would continue to correspond to the underlying reality it was meant to describe. The commission-split convention turned trade records into something else, and the two-legged repo recording preserved all the arithmetic while discarding the transaction itself.
Representational adequacy, as I have come to understand it, is the quality of that correspondence. A representationally adequate model captures the entities that actually matter in the domain, along with the relationships and events that define their behavior. Where the business has exceptions or non-standard processes that arise regularly enough to matter, it models those explicitly rather than absorbing them through workarounds that gradually obscure the underlying structure. Its absence rarely triggers a data quality alert, because the individual values are accurate and lineage holds up under scrutiny; the problem surfaces only when someone tries to answer a question the model was never designed to support.
This doesn’t fix itself downstream
The practical urgency is that representational problems do not get easier to address once data has entered the warehouse. There is a reasonable intuition that these issues can be resolved during the transformation process — what practitioners call the bronze-to-silver stage, where raw ingested data gets cleaned, standardized, and aligned. And transformations can do a great deal: inconsistent formats, naming conflicts, identifier mismatches are all addressable there.
What they cannot do is reconstruct meaning that was never recorded. If split-commission trades enter the raw layer as independent records with no indication that they belong together, there is no transformation that can recover the connection without someone encoding the institutional knowledge that these records represent a single transaction. If repos arrive as two unlinked trade legs, any downstream analysis that depends on treating them as a single instrument must carry that reconstruction logic forward indefinitely, updated as the business changes and explained to every new team member who inherits the pipeline. The transformation layer gradually accumulates business rules that should have lived in the source system, and the burden of interpretation compounds quietly over time.
This is why the representational work has to happen before data enters the warehouse — not necessarily by overhauling the system of record, which is often impractical, but by ensuring that whatever feeds the warehouse correctly represents the domain. Sometimes that means a tactical reconstruction: a purpose-built feed that merges split-commission records into unified trades, or links repo legs into single transaction objects with their associated terms and rates. It is unglamorous work, and it lives upstream of where most data teams focus their energy. Without it, every layer built above inherits the same interpretive burden.
What strikes me, looking across the organizations I have worked with, is that neither of these failures is unusual. The commission-split problem and the two-legged repo are instances of patterns that appear, in different forms, across industries and systems of every kind. Once you know what to look for, you start seeing them. And once you see them, you find yourself asking different questions — not just whether the data is accurate, but whether the model behind it actually represents the thing you are trying to understand. That shift in framing is, I think, where the more durable fixes begin.