Don't neglect these four types of testing

At Crafting Bytes we believe that quality is everyone's responsibility.

As developers we begin by writing functional (input output driven) unit tests.  These give the added benefit of forcing the code to be more functional in nature, with fewer side effects that can lead to defects.  We then build the running of those tests into the CI pipeline.  If the tests fail for any reason the code cannot be deployed.  These tests give us continuous feedback that the code is healthy, and granularity so that if something goes wrong we can figure out where the issue occurred.

As the number of components grows it will become necessary to start writing some integration tests.  A common example of these are tests which test the interaction with an actual database, or a custom REST client calling the REST server.  Once again these should be integrated into the CI pipeline so if tests fail, the build will not be deployed.

There are certain use cases for your specific product that must never fail.  The product itself would be considered broken if any of these use cases do not work.  For these, we would write end-to-end tests.  These are normally built using some browser or mobile simulator automation, and verify that the behavior that is shown on the screen is correct.  These tests are generally more difficult to write, more time consuming to run, and more expensive to maintain, but they are worth it.  They ensure that those primary use cases are never compromised.  Ideally these types of tests would be written in English (gherkin syntax) so that they can serve as specifications, even to non technical members of the team.

You won't anticipate everything, so don't forget to manually test the new feature before your users do.  Automated tests will not save you. It's very difficult to write a new feature and understand every way it could fail.   When issues are found through manual testing, we need to add to the suite of automated tests, so that the quality is protected against similar mistakes in the future.

​The combination of these three automated types of tests drastically reduces, and in some cases eliminates, the amount of regression that needs to be done upon major releases of the product.   

At Crafting Bytes, we have a saying: Test twice... then test again.

​