Sneak Peak: How a Stage-Gate Process Can Hinder Your Adoption of Continuous Delivery

Enforcing a stage-gate type development process can significantly hinder the adoption of continuous delivery in large organizations. To succeed, both R&D and product management must be involved and the process re-engineered, researchers at Aalto University discovered. Otherwise, process overhead and conformance to unrealistic schedules hinder adoption and dysfunctional mechanisms such as hiding problems to pass quality gates can flourish. 

An ever growing number of companies are implementing continuous delivery in order to deliver software more quickly to their customers. While the idea has a lot of promise, the implementation can be difficult, in particular in large organizations with an existing strong State-Gate process culture.

In a recent study by the Software Process Research Group at Aalto University in Finland, researchers found that adopting continuous delivery inside a software development unit is unlikely to succeed unless the rest of the organization is involved and the process is re-engineered to fit the demands of a modern, agile process.

Based on a case study in a large telecom organization, the four signs of  dysfunctional continuous delivery practice were identified:

1. Failed builds. Builds failed often, and were not quickly fixed by the people responsible for breaking them.
2. Flaky tests. Test failed randomly, even when the software had no bugs.
3. Low test coverage. Due to a low degree of coverage for the automated tests, the organization did not gain the confidence needed that the product was of releasable quality even if all tests passed.
4. Slow feedback. Due to integration and hardware issues, developers did not gain fast feedback on the quality of their recently integrated code.

The stage-gate process was partly to blame for the above problems though three primary mechanisms:

1. Process overhead. The existing process contained a large amount of tasks that had to be performed before each release, including manual testing, and excessive documentation and reporting.
2. Enforcing unrealistic schedules. As typical in a stage-gate process, schedules were decided upon up-front before development started, often with no slack. As the early set of requirements upon which the schedule was based, the schedule went from unrealistic to absurd. As a way of dealing with this, product problems were hidden in order to meet gate requirements and for personnel to achieve their individual bonus objectives. Furthermore, subsequent schedules were not updated even when encountering problems.
3. Multiple branches. The development process required the use of separate branches for development, after code freeze, and for each release, creating unnecessary complexity, and using scarce hardware resources for building and testing inefficiently.

In addition to the above, problems were created by the distributed nature of the organization, which created communication problems; the fact that the monolithic architecture of the underlying system made it unsuitable for continuous delivery, and the lack of a consistent and well-understood testing strategy.

For the full story, read our paper “Bottom-up Adoption of Continous Delivery in a Stage-Gate Managed Software Organization”, written by Eero Laukkanen, Timo O.A. Lehtinen, Juha Itkonen, Maria Paasivaara, and Casper Lassenius from Aalto University. The paper will be presented at ESEM 2016, and will be available in the IEEE Digital Library in October 2016.