Coverage analysis

Coverage is the question: how much of the system have we checked, and how much of it should we check? It is asking a higher level question of how we think about systems- do we think in terms of paths and branches in source code, or states the system can be in or possible bugs that could be detected, or how much the system adheres to the specified/desired behavior. Are systems completely captured by their implementation details, or are there higher level insights that is greater than the “sum of its parts”? These are questions that can be asked of every system we build, from the desktop computer, to mobile, to AI based systems like autonomous vehicles.

At the practical level, coverage is a measure of how good is our machinery of tests, assertions, formal verification, monitors and random testing really is. It is a fundamentally important part of designing and building systems.

Our work on coverage has got a lot of recognition, including a best paper award at DAC 2014. DAC is the premier conference in design automation. Our assertion coverage  algorithms are used by multiple industrial collaborators like Qualcomm, Oracle and TI.

We have looked at coverage from different angles. We have addressed the question for assertions- how good are our properties? We have found methods for coverage closure, or converging on coverage using the iterative inductive power of machine learning. We have found input test stimulus based on coverage feedback, to target less traversed parts of the system.

  1. Lingyi Liu, David Sheridan, William Tuohy, Shobha Vasudevan Towards coverage closure: Using GoldMine assertions for generating design validation stimulus, Design Automation and Test in Europe (DATE) 2011: 173-178
  2. Viraj Athavale,Sam Hertz, Darshan Jetly, Vijay Ganesan, Jim Krysl and Shobha Vasudevan, Using static analysis for coverage extraction from emulation/prototyping platforms, IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis, (CODES+ISSS) 2012: 207-214
  3. Chen-Hsuan Lin, Lingyi Liu and Shobha Vasudevan, Generating concise assertions with complete coverage, ACM Great Lakes Symposium on VLSI (GLSVLSI) 2013, pp 185-190
  4. Viraj Athavale, Sai Ma, Samuel Hertz and Shobha Vasudevan, Code Coverage of Assertions Using RTL Source Code Analysis. Design Automation Conference (DAC) 2014. Best paper award.
  5. Shobha Vasudevan: Coverage closure in SoC verification: Are we chasing a mirage? VLSI Test Symposium (VTS )2011
  6. Samuel Hertz, Debjit Pal, Shobha Vasudevan. A Figure of Merit for Assertions in Verification. Asia South Pacific Design Automation Conference (ASPDAC) 2019. To appear.