CI & CD

In this Test Automation series, we investigate ways to optimise code integration and delivery speed without trading off software quality. In a previous article we explored the “when” to run tests during continuous integration. An equally important question to answer is “what” tests to run. What are some industry guidelines we could apply to a codebase with thousands or millions of tests? One line, thousands of tests In a test setup that grew organically, without any optimizations, a developer edits one component file. The CI queues the whole test suite (let’s imagine 10,000 tests). ...

The speed vs quality dilemma Last week I laid out one of the CI/CD (Continuous Integration & Continuous Delivery) engineer dilemmas: improve quality with more test coverage, or improve the speed of delivery. Brute-forcing both by throwing more machines at the problem works for a while, then stops working when the test suite runs for hours and rarely comes back green on the first try. But speed and quality don’t have to trade off against each other, and there are ways to speed up integration and delivery, while aiming for high quality. ...

The Test Automation Engineer dilemma Your company is growing (yay!). You have more customers. You add more features and support more use cases. You can’t afford breakages and downtimes anymore, so you decide to improve the robustness and quality of your testing. In parallel, supporting your growth means hiring more developers, who need to ship fast, with early feedback and limited context-switch, while also sharing a pool of finite test and deploy resources. Integrating and deploying faster vs increasing test coverage is often the dilemma Test Automation Engineers face in such cases. ...