Skip to content

Added CHC benchmarks

Levente Bajczi requested to merge levente.bajczi/sv-benchmarks:main into main

This PR adds 2774 new benchmarks to the repository. These are transformed CHC benchmarks, originally sourced from the https://github.com/chc-comp/ organization to be used for CHC-COMP, which have been converted into CFAs and then to C files. See our (accepted, pending publication) paper at HCVS'23 on the transformation of CHC problems to CFAs: submitted.pdf, and our paper at SPIN'24 paper.pdf. The reproduction package for the latter can be found under 10.5281/zenodo.10529453.

I do not understand how to achieve the goal suggest to add it to benchmark definitions (.XML) in repositories for benchmark-definitions below.

I realize that the submission of this MR is technically before the deadline for SV-COMP24, but given its size, I wouldn't mind if merging this MR was postponed to after this year's SV-COMP.

I checked with several verification tools for signed integer overflow, and I found no problems with the benchmarks. Furthermore, I included comments in the yml files for the new tasks, stating which CHC solver verified the verdict.

  • programs added to new and appropriately named directory

  • license present and acceptable (in machine-readable comment at beginning of program as specified by the REUSE project)

  • contributed-by present (either in README file or as comment at beginning of program)

  • programs added to a .set file of an existing category, or new sub-category established (if justified)

  • suggest to add it to benchmark definitions (.XML) in repositories for benchmark-definitions

  • intended property matches the corresponding .prp file

  • programs and expected answer added to a .yml file according to task definitions

  • data model present in task-definition file
  • original (ideally not preprocessed) sources present
  • preprocessed files present Not necessary
  • preprocessed files generated with correct architecture Not necessary
  • Makefile added with correct content and without overly broad suppression of warnings
Edited by Levente Bajczi

Merge request reports