Avoiding Icebergs in IT with Third-Party Assurance

Imagine a scenario where your favorite sports team gets to handpick the referee for each match or you can personally review and approve your tax returns. Sounds appealing, doesn't it? Guaranteed victories and significant tax savings!

This idea probably crossed the minds of a certain aircraft manufacturer when they decided to cut corners to boost stock prices, essentially manipulating third-party inspections to overlook defects. While this strategy might have seemed beneficial initially, passengers didn’t agree, especially the ones sitting next to the exit door when its hinges failed mid-flight.

If you were to ask about their stock value now, the business likely regrets those shortcuts. Whatever short-term gains they made by doing the testing themselves have undoubtedly led to long-term repercussions.

In this article, we explore the importance of third-party assurance, and why cutting corners can have severe consequences.

Self-Evaluating SIs are an IT & Business Risk

Why do we apply common sense to recognize flaws in certain scenarios, yet seem to overlook these issues in IT? More specifically, why is it acceptable to cut costs by allowing System Integrators (SIs) to self-evaluate their work during significant transformation projects?

The parallels should be clear. While SIs may promise to streamline work, enhance efficiency, and speed up time to market by taking control of quality assurance, this approach often serves their interests more than yours. The reality is, relinquishing quality control tends to benefit the SI by reducing focus on complete quality assurance, not your business.

A common rebuttal, "We conduct our own User Acceptance Testing, so everything should be fine," does not hold up against the reality of the SI's vested interests. How many business users possess a deep understanding of structured testing to accurately assess the SI's work? And even if they have the knowledge, do they truly have the time for rigorous testing with such a packed schedule?

This highlights a critical oversight in trusting SIs with too much control over your quality assurance, underscoring the need for your business to maintain vigilance and involvement in the testing process to ensure your interests are fully protected.

Overlooking the Assurance Iceberg

If companies undergoing digital transformation or large-scale system implementations truly grasped the role of testing within the Software Development Life Cycle (SDLC), they'd recognize that User Acceptance Testing (UAT) is simply the visible portion of a much larger iceberg. Relying solely on UAT to assess the quality of the delivery is a profound mistake.

Experienced testers will confirm that UAT primarily focuses on assessing end-to-end business processes through a more 'benign' lens, emphasizing 'happy flow' scenarios. In contrast, System and Integration testing, which precedes UAT, adopts a more rigorous approach. It aims to thoroughly test the system by exposing it to as many scenarios as possible to identify where it could fail.

For example, look at UAT as a leisurely test drive on a well-paved road. This is where basic functionalities are checked – ensuring the engine is running, the speakers sound clear, and the air conditioning works. On the other hand, System and Integration testing is like the rigorous, all-encompassing torture tests on specialized tracks that simulate challenging conditions, ensuring the vehicle performs reliably under any circumstance.

Delaying the identification of issues until User Acceptance Testing (UAT) inevitably leads to substantial rework, directly contributing to time and budget overruns. If your System Integrator (SI) has not prioritized testing, and you still manage to execute a successful UAT, be prepared to encounter numerous problems. These will likely result in considerable time and financial resources spent on discussions, potential redefinition of original requirements, impact analysis, redevelopment, re-release, retesting, and more. This creates an excessive number of "re" activities, exposing your project to unnecessary risk and inefficiency.

Here, a common counterargument emerges: "Our contracts with the SI protect us from any errors, so they are responsible for any mistakes." This argument, frequently used by SIs themselves, aims to undermine the importance of independent validation through UAT. The rationale is that, with strong Service Level Agreements (SLAs) in place, UAT becomes redundant.

SIs might offer to handle UAT as part of their services, suggesting this makes the client's life easier. However, this approach primarily benefits the SI by eliminating grounds for dispute. Once UAT is signed off based on unreviewed reports, any issues that surface during production – which were not identified during UAT – result in additional costs for "out of scope" change requests, which could have been avoided if the project had been fault-free from the beginning.

Retail Banking Upgrade Gone Wrong: A Case Study

A clear example of what can go wrong without enough testing and too much trust in System Integrator (SI) self-reports is seen in a major retail bank's experience.

The bank started a big project to upgrade its consumer banking portal, and everything looked good at first. The bank's SI promised a smooth development and plenty of internal testing. However, when it came time for User Acceptance Testing (UAT), big problems with transaction processing and customer data security came to light – problems that should have been caught and fixed much earlier.

The consequences were immediate and serious. The launch was delayed for months, and fixing these issues cost a lot more than expected. Even worse, customer trust took a hit when news about the project's problems got out. This not only shows the dangers of not doing enough independent testing and validation, but it also makes the case for a thorough and integrated approach to quality assurance throughout the entire project, not just at the end.

Building Trust in Your Assurance Process

Even with the best contract in the world with your Systems Integrator (SI), do you want to risk a reputational disaster from a catastrophic failure in production? In other words, is it worth taking the risk and waiting for a critical failure to be exposed publicly mid-flight? Or would it be wiser to engage a third-party, independent validator to collaborate with you and your SI, ensuring every detail is meticulously checked?

The stakes are too high to leave the quality and integrity of your digital transformation to chance or blind trust in a Systems Integrator's self-assessment. Your strategy should include integrated, rigorous, independent validation at every step of your project's lifecycle to safeguard your investment and strengthen your brand's reputation against unforeseen embarrassments and the expenses of "re" activities. After all, when it comes to the success of your transformation project, hope is not a strategy, and trust must be validated.

CPT Global is the independent, third-party assurance partner of choice for leading brands. Get in touch to assure your IT.