Skip to content

Add new options to specify a validation task

Dirk Beyer requested to merge expected-witness-type into main

Starting this year, we also evaluate validators. The precondition is that the competition runs the validators on all kinds of witnesses, even wrong ones.

Because we have a time limit of 900 s for correctness and of 90 s for violation validation, we need to have two benchmark definitions.

This leads to the problem that validators that have a benchmark definition for correctness validation now can verify violation witnesses using 900 s time limit. This must be disabled, according to the rules.

Therefore, I have to ask validators to implement the following two options (or similar) to your validator:

--expectCorrectnessWitness If anything other than a correctness witness is given as input witness, then abort (with unknown or error).

--expectViolationWitness If anything other than a violation witness is given as input witness, then abort (with unknown or error).

I have added options to the benchmark definitions in this MR already, suggesting names that I think follow your conventions for option names.

The suggested implementation of the options is to look into the XML witness file, and if the type does not match the option, then abort. The validator must not output TRUE or FALSE in such cases.

Status:

Edited by Dirk Beyer

Merge request reports