Data and workflows don't work in isolation, so system testing can't either

In fast-moving investment and asset management environments, system change is a fact of life, whether that is to meet evolving security needs, upgrade aging systems, or respond to wider business evolution. The foundations that are laid during the transformation of investment management platforms determine the long-term operational success of the new or upgraded platform.

‍

The testing that is conducted during the transformation process is critical, but it too often focuses too narrowly on the single system that is undergoing change.  The problem is that real business operations are not isolated, and neither are the systems that enable them. By their nature, investment workflows aren’t confined to one system, but cross several. A workflow that begins in one place is completed across others, and the data is passed and translated at each handoff. Testing built around a single system can only ever see part of what is happening.

‍

This piece sets out why generic testing tools rarely bring the whole picture together, and why the problem sharpens rather than eases post-implementation.

‍

Where the end-to-end testing foundations are strong, validation carries all the way through to production. Where they are weak, blind spots are built in early and tend to surface at the point they are most costly to fix. And the larger the environment, the more this matters, because the risk only compounds as the IT estate grows.

Testing one system at a time is where the gaps begin

The challenge in any investment management transformation or large-scale implementation is being able to look at all the touch points in an environment. Engineers focused on changing a specific platform aren't usually thinking widely across a business whose workflows touch many other systems too. As an example: if you are primarily generating orders, you build your validations within the order management system as they relate to that endeavor. Systems don't exist in isolation but are connected by multiple workflows and share data that flows continuously between them.

‍

Orders or ideas are generated in an outside system, then validated and executed in the order management system. From there the order management system hands orders to an execution management system for execution, receives data back for allocations, and passes information down to the accounting system or the custodian for completion.

‍

Each of these results in an end-to-end workflow that does not live in any one system specifically but flows through a series of them holistically. Even in the best case of a front-to-back system, data is still being moved between components within that system.

The half of the story most testing misses: the data

This brings a real challenge, and a blind spot in most testing environments, because testing is so often done system by system. What makes it harder is that as data moves between systems it is translated and transformed into different forms. Whether the workflow works is only part of the problem. You can click the button, make the request in a CRM to invoke a cash flow or a trade request into the wealth management or trade order system, and see it end up where it is supposed to. The data has to arrive correctly too, whichever systems it passes through. Testing that only looks at whether something happened and something reached the endpoint is really only giving you half the story.

‍

That gap is hard to close with generic testing systems, of which there are many. One can test a UI invocation, another can test an API call with a different set of tools, and another can look at data reconciliation as data moves between systems. Few testing providers have put it together to look at all of those points holistically and affirm that everything is happening as expected.

‍

When you are looking at testing and QA platforms, it is important to look at how comprehensive the platform is and whether the domain expertise is baked into the tools themselves. That means understanding the intricacies of data as it moves through the investment life cycle, the touch points along the way, and above all the data structures that change from system to system.

‍

Start with generic tools and people who do not completely understand the workflows, and you are left trying to piece things together as they happen.

Change doesn't stop at go-live

Once you have end-to-end workflows across multiple systems with data validation in place, the next question is how you assure that the quality checks done in QA translate to how the data actually moves in production.

‍

Separate systems are often built to do that, which is wasteful once the validations and reconciliations have already been worked out. Testing that is confined to lower environments and does not translate into production leaves you disjointed between how you evaluated things below and how you reconcile them once you are live. The work should be done in a singular way and translate across every environment.

‍

Things do not stay static once you are live. As BAU updates are introduced, systems are upgraded and new desks come on, what matters is the residual and regression capability. That capability has to assert that what has been done before is not lost and does get revalidated. The routine patches and incremental updates of running a live environment are quick to apply, but in an environment this interconnected even a small change can carry outsized risk, which is why the validation that follows matters far more than the change itself. That validation asserts that nothing has broken and everything continues to work as expected, while any new efforts are evaluated alongside it. This becomes a self-reinforcing loop, a continuous regression that carries forward everything done before and newly evaluates anything new. The continuous regression runs from Dev through SIT and UAT to production.

‍

Change never really stops, so the testing that underpins it cannot stop either.

Setting the Bar: What Good Testing Looks Like

Handling all of this well, the cross-system workflows and the constant change, comes down to two things.

‍

The first is deep domain expertise to understand the intricacies of the investment management life cycle. That means the account structures, instruments and positions that may look different in different parts of the environment and carry a translation or transformation that is non-trivial.

‍

The second is understanding the full workflows of what wealth managers, hedge funds and investment managers do, and translating those workflows into actionable quality assurance without the client having to learn the domain.

‍

When you look at a solution, the questions are whether the domain expertise is baked in, whether the workflows are understandable, whether the tool provides for cross-system evaluation rather than singular system evaluation, and whether the data stays high quality and reconcilable as it moves.

‍

Without all of that, you are constantly struggling to piece together something that was never integrated to start with.

Fit-for-purpose holistic end-to-end testing

This is the specific need that our end-to-end testing platform Auto XLR8 was built to meet. The team at Velocity built it after seeing the problem in a very large-scale wealth transformation, where a client was moving from a largely homegrown system to many systems at once, including a new CRM, a new order management system, a new portfolio order generation system and a new accounting system. The client needed to see the workflows across all of them, validate them coherently across that communication chain, and assert that the data maintained its integrity as it moved.

‍

The platform validates the full cycle rather than a system at a time. It runs across development, UAT and production, reaching the controlled production monitoring that most tools cannot. Working alongside it is a distinct data quality tool, built specifically for the data challenge this piece has described. It handles data that arrives in different forms from different vendors, comparing it, mapping it to a common model and reconciling it at record level once the volumes climb past anything a spreadsheet can hold.

The stakes only rise with scale

None of the problems we’ve outlined here ease as a firm or its IT estate grows. As you bring in more desks, more systems and more people interacting with them, the interconnection issues multiply, and the cost of a limited testing view multiplies with it.

‍

A blind spot that was barely manageable in a smaller environment becomes a systemic risk across a larger one. One unnoticed data error after a new implementation or upgrade can propagate through connected workflows and into other systems – and the further it goes, the harder and more expensive it is to find and unwind the issue at source.

‍

This is exactly where a testing foundation built to work end-to-end and across systems proves its value, because strong cross-system validation that protects one transformation holds just as well across a far larger and more complex estate. The bigger the footprint, the greater the exposure, and the more valuable it becomes to test the whole rather than the parts from development through to production and continually on from there.

‍

Get that foundation right, and making investment management platform changes at scale becomes something a firm can absorb with confidence rather than fear.