Some things are inevitable, and one of those is that the time marches on relentlessly. Another is our very human tendency to underestimate work and fall behind on the ideal schedule. Luckily, in software development we have a secret weapon which can be used to defend ourselves against these truths: automated testing.
After finishing up work with my last client project I now have a fresh perspective on the benefits of automated testing which I would like to share with you. To set the scene, our client is a FinTech startup seeking regulatory approval. Deadlines for our software project is therefore therefore at the mercy of this critical process. This brings a unique set of management challenges that can only be overcome with constant and accurate feedback transmitted to the stakeholders.
Making the right decisions in a timely manner is the difference between success and failure for any business, not least those in a start-up in a fast-moving industry. The feedback that automated testing provides allows us to see through the complexity and dynamism of this kind of work in order to make educated decisions. In the case of this client, and my part of the work, we built automated suites which tested the web user interface by going through banking applications from a user point of view. These (many thousands of) scripts were able to quickly enter data sets that simulated a diverse set of customers, which would be impractical through any manual means. This front-end focused regression testing put the back-end microservices and the CRM to work, indirectly testing the system as whole, which then produced a report giving us the ability to keep the stakeholders up to date with the latest readiness information.
The information gained from these regression tests proved to be invaluable for us in the team too, helping us to stay on top of the incoming defects that came with each deployment. When all hands are “on deck” coming up to a go-live date, the familiarity of a periodic development cycle disappears and in any one day there can be incremental changes to any one of several microservices, or pieces of infrastructure. To bring clarity in these moments we can use a subset of our regression suites to quickly smoke test the system so that we can get some confidence on proceeding with the next tasks. This ad-hoc style of testing the system with a group of key tests is a great way of dealing with the unpredictable nature of defects appearing with frequent changes.
This allowed us to test changes end-to-end in hours rather than days, receiving new code in the morning and running all our regression tests within hours, ready for the end of day status report.
A final point to make is that these automated suites are costly to implement and maintain so these costs must always be weighed up with any potential benefits that they bring, with some projects not being suitable for this kind of testing strategy. That being said, the initial effort of implementing the suite can be seen as an investment into creating a test asset for the company that will provide value into the future.
Article written by Ben Hesketh, Test Engineer at Dragonfly
