1  # Coverity Scan for open source firmware
  2  
  3  ## What’s Coverity and Coverity Scan?
  4  
  5  Coverity is a static analysis tool. It hooks into the build process
  6  and in addition to the compiler creating object files, Coverity collects
  7  information about the code. That data is then processed in a separate pass
  8  to identify common programming errors, like out of bounds accesses in C.
  9  
 10  Coverity Scan is an online service for Open Source projects providing this
 11  analysis for free. The analysis pass is done on their servers and issues
 12  can be handled in their [web UI](https://scan.coverity.com/).
 13  
 14  The Scan service has some quotas based on code size to avoid overloading
 15  the system, but even at one build per week, that’s usually good enough
 16  because the identified issues still need to be triaged and fixed or they
 17  will simply be re-identified next week.
 18  
 19  ### Triage?
 20  
 21  The Web UI looks a bit like an issue tracker, even if it’s not a very
 22  good one. It’s possible to mark identified issues as valid or invalid,
 23  and annotate them with metadata which CLs fix them. The latter isn’t
 24  strictly necessary because Coverity Scan simply marks issues it can’t
 25  find anymore as fixed, but at times it helped identify issues that made
 26  a comeback.
 27  
 28  ### Alternatives
 29  
 30  There’s also clang’s scan-build, which is fully open-source, and
 31  finds different issues. As such, it’s less of an alternative and more
 32  of a complement.
 33  
 34  There’s a regular run of that for coreboot but not for the other projects
 35  hosted at coreboot.org.
 36  
 37  One downside is that it emits a bunch of HTML to report on issues,
 38  but there’s no interactivity (e.g. marking issues solved), no way
 39  to merge multiple builds (e.g. multiple board builds of a single tree)
 40  or a simple way to extract burndown charts and the like from that.
 41  
 42  #### Looking for a project?
 43  
 44  On the upside, it can emit the data in a machine readable format, so if
 45  anybody needs a project, a scan-build web-frontend like Coverity Scan would
 46  be feasible without having to go through scan-build’s guts, just by parsing
 47  text files - plus all the stateful and web parts to build on top.
 48  
 49  ## Logging into Coverity Scan
 50  
 51  Coverity Scan needs an account. It supports its own accounts and GitHub
 52  OAuth.
 53  
 54  Access to the dashboards needs approval: Request and you shall receive.
 55  
 56  ## coreboot & friends and Coverity Scan
 57  
 58  coreboot, flashrom, Chromium EC and other projects of that family have
 59  been made Coverity aware, that is, their build systems support building
 60  with a custom compiler configuration passed in “just right” to enable
 61  Coverity to add its hooks.
 62  
 63  The public coreboot CI system at
 64  [https://qa.coreboot.org/](https://qa.coreboot.org/) regularly does
 65  builds with Coverity and sends them off to Coverity Scan.
 66  
 67  Specifically, it covers:
 68  
 69  * Chromium EC: [Coverity Scan site][crECCoverity] ([build job][crECBuildJob])
 70  * coreboot: [Coverity Scan site][corebootCoverity] ([build job][corebootBuildJob]), [scan-build output][corebootScanBuild] ([build job][corebootScanBuildJob])
 71  * em100: [Coverity Scan site][em100Coverity] ([build job][em100BuildJob])
 72  * fcode-utils: [Coverity Scan site][fcodeUtilsCoverity] ([build job][fcodeUtilsBuildJob])
 73  * flashrom: [Coverity Scan site][flashromCoverity] ([build job][flashromBuildJob])
 74  * memtest86+: [Coverity Scan site][memtestCoverity] ([build job][memtestBuildJob])
 75  * vboot: [Coverity Scan site][vbootCoverity] ([build job][vbootBuildJob])
 76  
 77  [crECCoverity]: https://scan.coverity.com/projects/chromium-ec
 78  [corebootCoverity]: https://scan.coverity.com/projects/coreboot
 79  [em100Coverity]: https://scan.coverity.com/projects/em100
 80  [fcodeUtilsCoverity]: https://scan.coverity.com/projects/fcode-utils
 81  [flashromCoverity]: https://scan.coverity.com/projects/flashrom
 82  [memtestCoverity]: https://scan.coverity.com/projects/memtest86
 83  [vbootCoverity]: https://scan.coverity.com/projects/vboot
 84  
 85  [corebootScanBuild]: https://www.coreboot.org/scan-build/
 86  
 87  [crECBuildJob]: https://qa.coreboot.org/view/coverity/job/ChromeEC-Coverity/
 88  [corebootBuildJob]: https://qa.coreboot.org/view/coverity/job/coreboot-coverity/
 89  [corebootScanBuildJob]: https://qa.coreboot.org/view/coverity/job/coreboot_scanbuild/
 90  [em100BuildJob]: https://qa.coreboot.org/view/coverity/job/em100-coverity/
 91  [fcodeUtilsBuildJob]: https://qa.coreboot.org/view/coverity/job/fcode-utils-coverity/
 92  [flashromBuildJob]: https://qa.coreboot.org/view/coverity/job/flashrom-coverity/
 93  [memtestBuildJob]: https://qa.coreboot.org/view/coverity/job/memtest86plus-coverity/
 94  [vbootBuildJob]: https://qa.coreboot.org/view/coverity/job/vboot-coverity/
 95  
 96  Some projects (e.g. Chromium EC) build a different subset of boards on
 97  each run, ensuring that everything is analyzed eventually. The downside
 98  is that coverity issues pop up and disappear somewhat randomly as they
 99  are discovered and go unnoticed in a later build.
100  
101  More projects that are hosted on review.coreboot.org (potentially as a
102  mirror, like vboot and EC) could be served through that pipeline. Reach
103  out to {stepan,martin}@coreboot.org.