Different businesses have different versions of the same data. A universal data truth would be nice but as companies’ legal infrastructures become more complex, new financial products continue to be developed and data architectures are squeezed into older technologies, this creates data discrepancies and inconsistencies that make it harder to aggregate, assimilate and normalize enterprise information.
In our very complex, interconnected and fragile world, these complexities and the data they represent do matter. But how can anyone at a senior level assess this inter-connectedness and inter-dependencies when an issuer underwrites securities composed of hundreds of loans originated by various subsidiaries that are guaranteed by third parties that obtain financing and insurance from the original issuer?
It’s safe to say that many Lehman Brothers counterparties had no idea of the extent of their exposure -- until it was too late. Unfortunately, little has changed in the race for clean data since 2008 and Lehman’s fateful demise.
Firms, especially larger ones, have heavily invested in their data infrastructure. It is not unusual to hear of investments of $50 million to $100 million to solve these problems. However, most of the firms that have made these investments do not see the returns they are looking for and most of these initiatives result in an enterprise-wide bureaucracy, hindering growth, stifling innovation and increasing costs while doing little to solve the problem.
This is because most of these large-scale data initiatives start with a faulty premise: Given unlimited centralized resources, any problem can actually be solved. But the challenge is that this data issue cannot be solved by an enterprise or even a divisional solution -- because the problem is simply too big.
Today, centralized solutions require a comprehensive view of an enterprise’s data usage. We need to believe the enterprise can understand, agree and manage not only how the data should be represented, but how it should be stored and normalized. Even if one internal team could fully understand the difference between an equity, convertible bond, debenture, securitized note, future, option and swap, they would also have to know how each product gets calculated and used and the tangential risks associated with each product, not to mention the issuer chain and various product dependencies.
This is a Herculean task, which is why enterprise data projects extend over years, cost tens if not hundreds of millions to implement, and often never reach fruition. Furthermore, most of the cost involved isn’t software-specific – it is spent on governance, data model rationalization and integration.
There is a better way to manage these projects. It is not through unification but through a federated strategy.
A federated solution pushes the processing of data down toward the processing system layer. Each system manages the data that drives the business and then reports up to a central repository.
In a federated model, the endpoints are the masters, with the hub serving as a guide pointing to where the data can be found. In a centralized system, the central view of the pristine data gets pushed or propagated throughout the enterprise. In the federated model, the system of record determines the product characteristics and reports to a centralized aggregator.
The centralized view assumes that there is only one proper view of the data held centrally, while federation assumes that processing systems are right. Centralizing can remove data integrity from the business, creating the most to lose if the data is wrong, while giving control of it to data groups that care about cleanliness but have nothing personally at stake for getting it right.
Centralization also requires very tight integration between processing and data-management platforms, creating very expensive enterprise projects to modify virtually all organizational systems to seamlessly obtain, vet and compare enterprise data to each and every processing system.
Even with federation, we can’t completely sever the tie between local and enterprise information. There are times when conflicts arise and the multiple versions of the truth need to be reconciled. So, while federated information does need to flow between local systems and a centralized repository, there does need to be logic to reconcile, highlight, track and balance conflicts in near real time to ensure that operating groups have not gone rogue.
Giving users control of their data can cause conflict and increase management complexity of federated data strategies. But once you consider the cost and lack of a success history of centralized data projects, it makes sense to start thinking of a better way where businesses can manage infrastructure; managers can get a better view of risk; audit and control can more easily track conflict; and senior management can focus on the company -- instead of investing hundreds of millions of dollars with an uncertain and (most likely) negative return.