| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Get control of your email attachments. Connect all your Gmail accounts and in less than 2 minutes, Dokkio will automatically organize your file attachments. You can also connect Dokkio to Drive, Dropbox, and Slack. Sign up for free.

View
 

Statistics To Gather

Page history last edited by David Wagner 11 years, 9 months ago

This page lists the statistics we want to gather as part of a catchconv run and upload to Metafuzz. Each statistic comes with a status indicating whether we currently record it or not.

 

* Number of bug buckets found. Status: recorded.

* Number of total bugs found. Status: recorded.

* CPU time spent on symbolic execution. Status: not recorded.

* CPU time spent on solving constraints. Status: recorded.

* CPU time spent on triage. Status: not recorded.

* CPU time spent total. Status: not recorded.

Add more here.

 

DaveW suggestions for statistics you'll want to report in the final paper (this is admittedly a different question than the statistics you want to gather for each run and upload to Metafuzz, so maybe it's not relevant here):

* Classify by the type of bug (InvalidRead, InvalidWrite, DoubleFree, etc.) and the number of bugs in each category.

* Count how many bugs were fixed (or acknowledged by developers as real bugs) vs number of bugs reported to developers.

* Number of bugs found by catchconv vs number of bugs found by zzuf

* By the way, I'm not convinced you should be doing bug buckets on the client.  I thought the plan was to do bug bucketing on the server, so that the bucketing could use the new duplicate detection methods (e.g., use function names and source code line numbers rather than PC values; allow some "fuzz" in the line numbers; take only the first few lines of the stack trace), to ensure that the count of bugs is not inappropriately inflated

Comments (0)

You don't have permission to comment on this page.