Automated Testing (Codeception + Github Actions)

Testing is the only way to find bugs. And bugs are bad. It’s pretty much guaranteed there will always be bugs in any project, but there are ways to make sure that the number of bugs are minimized. One of those ways is automated testing like unit, integration, functional, and acceptance testing.

All DWS WordPress Framework modules use Codeception for writing and running automated tests. Codeception is a framework for writing PHP tests, not WordPress tests specifically. For that reason, it natively doesn’t support integration tests (which in the context of WordPress Plugins are tests for running the code within WordPress). To use Codeception effectively, we’re using the WP-Browser package (all of this should be obvious by looking at the dev-dependencies specified in the composer.json files).

There are basically 4 levels of tests (mentioned in the paragraph above). A good breakdown of what each test stands for is offered by WP-Browser.

Not all levels of testing make sense for all modules. For example, acceptance tests make sense for the Bootstrapper Module, while acceptance tests alone make no sense for the Helpers Module.

Writing good automated tests ensures that our code works as intended and that we don’t accidentally break anything while releasing a new version. It is obviously a rinse-and-repeat process and the tests themselves can be flawed since we are also humans ergo intrinsically flawed, but at the very least, it should let us avoid silly mistakes and most regression bugs.

Test across different environments

If automated tests pass during development, that simply means that everything is good on the PHP version and WP version that the developer is using to run the tests against. That’s great, but it still doesn’t mean it will work everywhere else (for example, you might be running PHP8 and all tests pass but you still aren’t sure that they pass on PHP7.4 which is our minimum guarantee).

For that we use Github Actions. Theye allows us to run all of our tests across different versions of PHP and WordPress by setting up virtual machines to our specifications and … running the tests again in each one.

A new version is never released unless all automated tests pass in all relevant environments!