Automation

In this Test Automation series, we investigate ways to optimise code integration and delivery speed without trading off software quality. In a previous article we explored the “when” to run tests during continuous integration. An equally important question to answer is “what” tests to run. What are some industry guidelines we could apply to a codebase with thousands or millions of tests? One line, thousands of tests In a test setup that grew organically, without any optimizations, a developer edits one component file. The CI queues the whole test suite (let’s imagine 10,000 tests). ...

The speed vs quality dilemma Last week I laid out one of the CI/CD (Continuous Integration & Continuous Delivery) engineer dilemmas: improve quality with more test coverage, or improve the speed of delivery. Brute-forcing both by throwing more machines at the problem works for a while, then stops working when the test suite runs for hours and rarely comes back green on the first try. But speed and quality don’t have to trade off against each other, and there are ways to speed up integration and delivery, while aiming for high quality. ...

The Test Automation Engineer dilemma Your company is growing (yay!). You have more customers. You add more features and support more use cases. You can’t afford breakages and downtimes anymore, so you decide to improve the robustness and quality of your testing. In parallel, supporting your growth means hiring more developers, who need to ship fast, with early feedback and limited context-switch, while also sharing a pool of finite test and deploy resources. Integrating and deploying faster vs increasing test coverage is often the dilemma Test Automation Engineers face in such cases. ...

The context An interesting challenge to tackle in the CI world is how to keep up with dependencies. How to stay up to date when your product requires hundreds or thousands of packages, tools, SDKs, and sometimes services that rely on each other. On one hand you could decide to always use the tip (latest), which puts you at risk of a bad surprise on a random Tuesday morning if the new version of a dependency suddenly introduced a breaking change. On the other hand, you could pin all the updates, and hope to never forget to update them manually. ...

In recent years I’ve spent more and more time trying to continuously optimize my daily routine. Fixing the pain points, and improving my productivity. At the end of the day, it’s all about freeing up more time and being able to focus more on what matters and what I like. As I was thinking about it, wrapping up 2022, I realized how aligned this life mindset is with my Automation engineering job’s principles. At the end of the day, my work is all about freeing up internal customers’ time so they can focus on where they provide the highest value (and of course by doing so, saving the company more money). ...