Draft: Add CI validation functionality

Add Validation support to existing CI. Focus on:

• PDF Value Validation
• Language-Agreement between Fortran/Python/C++
• Metadata Integrity checks

Design

I've added the lhapdf-validate script. I've used argparse nearly everywhere so that we have an easier time keeping track of things. The script can be used like ./lhapdf-validate CT10nlo 0 or just ./lhapdf-validate. Both do the same thing as the default arguments are CT10nlo for the PDF and 0 for the member (Use -h to view all arguments!)

In it's current form, calling ./lhapdf-validate performs two tasks:

• First up is the test against different LHAPDF versions. If the inputted version isn't present in the archival data (archival data referring to the values I generated manually for all versions post lhapdf-6.2.3 (there were about 8)), then the test aborts. The default version to test against, in case there is no explicit input, is to the latest git tag at the time of running. As of today, until we make a new git tag, it will compare the current build (i.e. lhapdf-6.5.5a) against the latest git tag (i.e. lhapdf-6.5.4).
• Second up is the compiler test, where we test value stability across the different compilers. We first generate the output files for all 3-languages, which takes around 2s. Then we perform the comparison.

Both tests compare values against each other. Floats are shifty and can't be compared directly so I employed the use of the math.isclose() function, which is governed by this formula:

abs(a - b) <= max(relative_tolerance * max(abs(a), abs(b)), absolute_tolerance)

I've tested both tests exhaustively for failures and different test-cases and I'm pretty confident about it's stability. Debugging is easy too because all the failure points have some kind of custom error message associated with them, which should make life easier.

Shortcomings

There are a few obvious shortcomings (for now):

• Firstly, the archival data was generated using CT10nlo with member 0. That means that even though you can input a different PDF into the test, it's most certainly going to fail because it's being compared against data that was generated using CT10nlo, so there's not much flexibility there.

• For now, you'd have to manually add values into the archive for each new build. This is easy to automate using the CI by coding in a job that activates every time there is a new tag, but I just haven't gotten to it yet. This means that as soon as we push lhapdf-6.5.5 to GitLab, the latest tag would be 6.5.5, which would not have any archival data.

  • Also, this means that whenever you push a tag to GitLab, you're in a situation where the latest tag is the one you just pushed so you're testing the exact same versions against each other (this would happened every time a new tag is pushed...)

• The messages that alert you of the disparity between xfxQ2 values don't actually tell you the pid/x/q arguments used to generate the xfxQ2 values that are in disagreement. This is also something that can be easily added. I should probably fix this...