Performing Regression Testing effectively.
I often work with software teams and historically have led testing teams myself, so as I’m recently operating in the same world as a bunch of testers, it reminded me to jot some thoughts down, from a business perspective, into the importance of regression testing. This is especially important when you’re building F**koff websites.
Regression testing is important for complex software releases. New features are one thing, but if that sprint of new features breaks something else that used to work perfectly, your users are going to be pissed pretty quickly.
It does however surprise me how much even to this day companies don’t prioritise regression testing often due to their being no clear ROI attached to it. Some of the key issues around getting a business to understand the value of regression testing include (I’ve tried to be fairly exhaustive here);
It can be expensive as f***
It can take so much time to do it right (like anything worthwhile, right?) and testing & re-testing an application or website is often seen as unesscary to a business when it’s already ploughed money & resource into the product earlier down the line. Because there’s no clear ROI attached to this activity it’s often difficult to get buy in, seen subsequently as a unesscary evil and not given the priority it deserves. This then means it’s done badly, etc etc and the spiral of shit begins.
It’s hugely time bound.
With the advent of agile thinking, software sprints, development time & release schedules are becoming quicker & quicker. So, when it comes to regression testing, it’s always done to a very tight schedule. Because of this, the temptation to skip tests is massive, and therefore issues are often missed.
Optimisation & Maintenance is a job in itself.
Test scripts for regression require constant maintenance. Something often missed by companies. For every change to the core product, website or software, the regression test suite needs updating and testing in its own right. Managing, purging and re-writing test cases can be a job in itself so it’s easy for a business to ignore this burden & avoid regression testing altogether.
It’s often badly instructed.
Testers are often ill equipped to regression test. Not understanding what they are testing, what the product is ultimately meant to do nor why and nor understand any of it’s history. Anyone new coming in has an impossible task & it’s highly unlikely any time has been given to document things. Therefore, failure is baked in right from day one. Developing an effective testing strategy is almost always missed and I see it time & time again.
So now we’ve looked at what companies do wrong, let’s now look at what you can consider if you want to up your performance of regression testing on your next software project.
Execute the automation of smoke & sanity testing
Let’s clear up a few terms – ‘smoke’ testing usually means testing parts of the platform or product at a very early stage to understand how close to the mark it is in terms of delivering what it should do without digging too much deeper. ‘Sanity testing’ refers to testing a few areas pretty much at random, to test across a broad spectrum of functionality.
So, by creating these cases in advance, and ensuring they are broad enough, it makes them easier to change when base features & functionality are changed. Make sense? You’re testing the application here under normal conditions, not to find bugs. You can automate this part massively and save heaps of time. I must write a post about Selenium/Cucumber soon. My ‘go to’ test software since I moved away from WebLoad.
The right tools for the job
Point one brings me neatly onto point two, the right automated regression testing tools. If you’ve as a business agreed you NEED automation testing and UNDERSTAND the value it can bring in terms of time & cost, then you need to consider the right tool. Often, this is working your face off by testing each one in turn, until you & your team settle on the right one for that particular test. There’s a few here, the pro’s & con’s of each outside the scope of this article. I will say however that you must give time & space to allow the tools to be set up correctly on the network, ensuring computers & infrastructure are adequately spec’ed to really leverage the benefits from automation testing.
Pour over the bug reports in detail
One of the benefits of using a decent tool means the bug reporting is often class. Pouring over these in detail ensuring you’re clear on bugs, and these bugs are documented appropriately makes fixing them so much easier. This is a culture, a mindset you must adopt. The more time you spend documenting a bug, the easier it ultimately comes to fix. If you’re testing team is jaded and just throwing bugs into a bug tracker, you’re mean-cost-to-fix increases for each bug raised.
Culture
Regression testing is a repetitive task and subsequently fairly boring. Remember those fleshy humans at the end of the keyboards & ensure their job is mixed up occasionally. Keeping them fresh means they’ll be more likely and more in the frame of mind to spot issues.
Maintenance
You MUST allow for time to maintain your scripts & regression testing packs. Take the time to look after your scripts will pay dividends in the long run. And if you haven’t considered points 2 and 4 above, you can guarantee you’ll have some disgruntled tester running all your regression scripts on his own laptop leaving your organisation massively at risk. All because you haven’t provided the right equipment, infrastructure or loyalty to this discipline, instead paying it lip service.
Testing is so important. Do it right.
