Integrated testing - Proving apps work

Continual functional testing is good, but it's essential to check an app integrates into the wider ecosystem and avoids easy to miss bugs.

This article is about the background to creating an IMAP integration test script. Why it was needed, what makes it a good foundation for tests, and how to improve tests to make debugging easier.

Modern tools are great at raising standards and making it easier to code than ever, and integrating them into normal development is essential for reducing bugs. But there's more to an app than its functions and integrated testing ensures an app still works as expected during normal workflow, and also for more expensive (time and resource) testing.

All software has bugs and DBMail is no exception. DBMail has always been reliable thanks to extensive manual tests, but when it's used by so many it's essential to have robust testing in place.

Testing an app as complex as DBMail is challenging as there are many configuration options. By creating a check script to automate these tasks is a

Valgrind is an essential tool for checking memory in a way that facilitates debugging makes it easy to pinpoint any issue.

Although DBMail was always tested, it lacked an automatic integration test script until a few days ago. It was always tested manually then updated on a live server for soak testing. Automation makes it easier to test and test more frequently. It also makes it easier for others to perform the same tests.

DBMail works on a wide variety of Unix and Linux type operating systems so the choice of Bourne shell vs Bash was easy as Bourne shell is more widely available and Bash, a superset, has everything that was needed for testing.

You can checkout the tests on GitHub.