When HealthCare.gov, the online health insurance marketplace launched, it promised to give residents of 36 states the opportunity to shop for health insurance as required by the Affordable Care Act (ACA). Yet, from the moment it went live, the problem-ridden application was effectively unusable for many, even as the deadline to buy healthcare under the ACA loomed. Any application that buckles under the strain of fulfilling its own purpose will not be well-received by the public. However, it was quality assurance engineers everywhere breathing deep, knowing sighs as officials explained that an effort to release the application on a certain date lead to “insufficient quality assurance testing” as a root cause of the trouble.
When a certain deadline needs to be met and development hours are running long, it may be tempting to see quality assurance testing as an expense to be minimized. Hoping that releasing on time with a few pending issues, is preferable to releasing later than promised. Yet, as demonstrated by the troubled launch of HealthCare.gov and many other infamous software disasters, cutting back on testing time and effort can be costlier than employing a testing strategy that will prevent a catastrophe.
Misunderstanding the Problem
If you believe that functionality and feature completion are the key benchmarks to be achieved before releasing, then, to quote the cinema classic The Princess Bride —“You fell victim to one of the classic blunders!”.
It’s very easy to get caught up in the new features you want to present to users — the things that will give your software the competitive edge — and ignore the seemingly mundane questions like:
- How will it respond under load?
- How secure from intrusion or manipulation is our product?
- Are we confident it is managing data correctly?
However, these are the areas where the exclusion of quality assurance testing can lead to hidden costs. Let’s take a closer look at them.
Hidden Cost #1: Underperforming Software
A piece of software can seem very useful in development, but all the finest features will seem foolish if the application does not respond well in production. There are many questions to ask here:
- How well does the application manage memory?
- Does it interact with a database in manner that is efficient and scalable?
- If it’s a mobile app, what impact does it have on resources like storage space and battery usage?
Answers to these questions, and others, are essential to success and will make a big difference in how useful users find your application. The last thing you want to hear users say is “it’s too slow” or “it’s a resource hog”, even if it’s feature-rich and visually appealing. Ignoring this can mean painful losses in your users time, their productivity and your reputation.
Hidden Cost #2: High Security Risk
Despite the heavy toll a poorly performing piece of software can have on your user’s experience, the potential that your software is vulnerable to intrusion is an issue of much greater magnitude. This is not an area your users will notice, until something goes terribly wrong. It’s important here to assume one thing: someone out there wants to crack your application. This is something that hard-working, honest people have trouble conceptualizing. However, there are people everywhere that want to break down what you are building. Their goal may be exposing user data, embarrassing your company, sharpening their skills before taking on some larger prize, or the simple thrill of causing chaos and headaches for others.
If your application sends any type of user-entered data to a server—and almost all do—a set of thoughtful security tests executed by competent QA engineers should be a normal part of a development lifecycle. Not something that get’s skipped. Similar to issues with performance, vulnerabilities in your software put your company at risk of losing money, time and the trust of your clients.
Hidden Cost #3: Data Loss and Corruption
If your application is performing well and it’s secure, there is still the matter of data. Perhaps the most precious asset a person or a company has is its data. It would be silly to put any of it at risk. We’ve talked about keeping the application and by extension, the data, secure. But, a threat of equal magnitude is the possibility that the program itself loses or corrupts the data. A variety of data validation tests can be implemented to ensure data quality is being maintained by your application and all assets are safe. Data failure can effectively negate the time and money spent to enter and secure the data in the first place, making it an important consideration for conducting QA in a software development project.
The Moral of the Story
Releasing software that is slow, insecure, and unreliable is almost entirely preventable. Quality assurance testing doesn’t have to take a great deal of time. And, with some advanced planning, much of it can be done along the way. The mistake is seeing quality assurance testing as nonessential, a “nice to have”, rather than a requirement for your project. There is simply no comparison between the cost of adding quality assurance testing to your project and the hidden costs of releasing problematic software. This is true of large scale failures, and the smaller ones. While it’s easy to criticize the mistakes of those behind the HealthCare.gov fiasco, it’s even easier, and a great deal wiser to learn from them.