Continuous Testing Requires Continuous Data

Share on facebook
Share on twitter
Share on linkedin
Share on email

Software development organizations are being pressed to become more responsive to the needs of the business.  As such, there has been a move afoot for the last dozen years or more that is all about doing more, quicker, and better.  From agile, to DevOps to CI/CD, organizations are striving to deliver high quality code into product faster than ever before.  An implied requirement of this move is Continuous Testing. This goes beyond ‘simple’ test automation to a state where the testing is integrated into the CI/CD pipeline.  What is not always realized, but soon becomes evident, is that in order to get it right, you also need “Continuous Test Data.”  Without that, your continuous testing will not be possible, and your results will be limited.

When your team was doing manual testing, they required test data – actually – great test data, in order to achieve great test results on time.  The move to continuous testing does not change that, in fact, it probably puts even more emphasis on it. Automating tests with bad data usually means more testing failure due to bad data, not bad code.  Or, in some cases (false positives) passing because of bad data not good code.  Bad data just makes it that much harder to pinpoint the real issue.  Enough issues caused by bad data causes everyone to begin losing trust in the CI/CD process.  Clearly not a good thing.

Infrastructure-as-code now allows teams to provision complete test environments on demand.  This requires test data on demand.  These environments should be temporary in nature; decommissioned after use; but able to be quickly re-established and reloaded for the next round.  The data needs to be as close to production data as possible.  If it is not, you will encounter the issues noted above.  Making a copy of production databases is cumbersome, slow, and risky if your organization has any sensitive data (e.g. PII, PHI, PCI). 

The solution for continuous test data is test data management (TDM).  Ideally, you will implement a tool that is specially designed for the CI/CD continuous testing pipeline – like Semele’s Test Data Management and Test Data Repository solution.  In the same manner you would create environments on demand you would provision test data on demand.

About the author.

Carl’s insights are grounded by a 30 year career in the technology field.  Starting as a tester for a large system integrator, he has written software; managed software development teams and led large technology organizations.  As a manager and leader Carl enbraces new technology, productivity tools  and processes to help teams exceed customer expectations.  Carl leads the professional services organization at Semele Data; ensuring that our software is deployed to enable our clients provision the best test data possible while eliminating test data bottlenecks.

You can follow Carl on LinkedIn.