[PDF] A Conceptual Replication of Continuous Integration Pain Points in ...

CI systems require effort to process and can cause unwanted in- terruptions [39], especially without developer buy-in and in the presence of frequent false positives from flaky tests and platform instabilities [37]. Bad experiences or frustration with a specific CI tool can turn developers away from CI as a practice, even when … researchers and tool builders focus on the most severe CI barriers. In this paper we review the CI literature from the perspective of pain points to adoption and usage, and perform a mixed-methods conceptual replication [32, 65] of previously observed findings, on a new population (GitHub open-source developers using Travis … Our main results (Sec. 6), confirming past literature, are that many developers find troubleshooting build failures difficult, desire consistency in CI tools across their projects, find it difficult to use CI with complex tool setups including Docker or to use CI with unsupported languages, find long build times annoying, and find … Testing Deficiencies . Insufficient tests are seen as a significant pain point for adopting CI (O5.a) [28, 39, 40, 52, 64]. Conversely, many see the value of CI, but see writing tests as challenging [27, 39, 40, 52, 64]. The difficulty of automating certain kinds of tests