Like many of us, my email inbox is peppered with messages from software vendors who would like to “let me in” on an opportunity to purchase their product. These are not spam, necessarily. They come to me because some time ago, when I was on the hunt for a software testing tool, I hastily provided my email address in exchange for a free trial of their product. I was looking to find an application that would enable our quality assurance personnel to create and organize test cases for software testing. However, what I discovered during this search was that there is much more to selecting test management software than I had imagined.
Let the Search Begin…
Once you have decided to seek out a new testing tool, it is tempting to dive into online literature, screenshots, demos, webcasts and of course, free trials of the most promising candidates. This would be a mistake. Like so many of life’s endeavors, you will need to know what you are searching for before you can find it. It is vitally important to put the idea out of your mind that any software tool is the “answer” to your teams’ needs. What your team needs—what every team needs—is not an application per se, but a workable process for getting QA done. The application should support the process, not the other way around. Therefore, the search should begin there.
Step One: Identify What Your Team Needs
All that may be needed at the start is for you and the QA team to identify how you do your best work. You may already have a way of doing things that is effective. If not, it’s time to engineer a new process for how testing should proceed in most situations. Perhaps you could even “go old school” with it. Work up some spreadsheet templates to execute your process for a while. Make sure you can trust your process first and then select the best software tool to facilitate it. Of course, there is nothing wrong with letting a great testing tool open your minds to some possibilities, and then making a few adjustments to mesh your process with the software. What is important to keep in mind is that you should never let the limits of a piece of software compromise what could work, or is already working for you. Naturally there are some key features you’re likely to be interested in and should keep in mind when evaluating prospective candidates. These include:
- The ability to create test cases which include step-by-step procedures for testing
- A way to indicate how the test case was derived (i.e. by linking requirements, user stories, etc.)
- The ability to mark test cases or even individual testing steps as passing or failing
- A way to group and organize test cases for repeated use
- The ability to track and report test case passes and failures
- A way to facilitate and/or execute automated testing
Step Two: Everybody Should Try It Out
Among testing tool offerings, a lot is said about the ability to create status reports, and this information dominates most of the online literature. Of course, reporting on test progress is absolutely essential and an application that does a poor job of reporting may not be valuable to you. There are some tools which do a great job of reporting results, but a poor job of enabling people to do the work that the reports are supposed to reflect. My own cynicism leads me to suspect that some companies put reporting at the forefront of their offering to impress decision makers who make selections for their staff based on their own needs. Namely their desire to get a high-level view of what is happening on projects. Whether this theory holds true or not, choosing an application that enables managers to do their work while leaving the testing staff unproductive would not be in anyone’s best interest. While you won’t need everyone’s input, the real evaluation of any software will happen when you can put a trial version of the product in the hands of at least one person at every level of the software development lifecycle and ensure that it works well no matter how you use it.
Step Three: Think Long Term Commitment
Finally, one uncomfortable truth about selecting any software, even after conscientious evaluation, is that you may find the solution you chose does not work for your team or company. Or it only works in the short term. Considering this, it may be wise to select an application that can be configured to your needs. Not all tools offer this kind of flexibility, but your company’s plans to grow may well need applications that can grow with it. In this case, you will want to get input from those who would be able to anticipate how this growth may occur and determine if the software you are choosing can grow with you.
All Together Now
If you’ve been keeping score, we have now involved not just quality assurance engineers and testers, but also management and anyone who takes the role of casting vision for your organization into this decision. However, don’t let this dash your hopes of coming to a quick or easy decision on a new testing tool. The main point here is not to involve too many people, but to have a clear strategy and set of goals in place before you invest in any product. A little forethought now could save money and frustration later. Of course, no matter what you do you may still have the frustration of unwanted emails haunting you now and in the future. Then again, perhaps you, unlike me, will take a few seconds and hit “unsubscribe” before it gets out of hand.