Nathaniel Ayewah, William Pugh. The Google FindBugs fixit. In Paolo Tonella, Alessandro Orso, editors, Proceedings of the Nineteenth International Symposium on Software Testing and Analysis, ISSTA 2010, Trento, Italy, July 12-16, 2010. pages 241-252, ACM, 2010. [doi]
In May 2009, Google conducted a company wide FindBugs “fixit”. Hundreds of engineers reviewed thousands of FindBugs warnings, and fixed or filed reports against many of them. In this paper, we discuss the lessons learned from this exercise, and analyze the resulting dataset, which contains data about how warnings in each bug pattern were classified. Significantly, we observed that even though most issues were flagged for fixing, few appeared to be causing any serious problems in production. This suggests that most interesting software quality problems were eventually found and fixed without FindBugs, but FindBugs could have found these problems early, when they are cheap to remediate. We compared this observation to bug trends observed in code snapshots from student projects.
The full dataset from the Google fixit, with confidential details encrypted, will be published along with this paper.