Parameter-Criteria Comparison View
Compare jobs to evaluate the influence of different job parameters on the result.
Goal is to see a comparison view of two or more jobs, where one side shows the differences in input while the other side shows the differences in outcome. The input differences would be the parameters, while the outcome difference would be the criteria.
What are parameters?
Parameters can be any key-value metric that can be used to describe the model/job.
Parameters can be existing information about the job like the job id or the main name.
But, it should also be possible for the user to add specific properties to a job that describe that model/job.
These parameters could be any metric that the user can express in a key-value fashion (version identifiers, actual solver parameters, temperature scaling, etc.).
These parameters would be added to a job on the detail page manually by the user.
Automatic reading of parameters of the model can be added as a future feature.
What are criteria?
Similar to parameters, criteria are any key-value metric that can be used to describe the outcome/result of a job.
The criteria might be something as simple as the already existing result assessment.
But, it probably makes sense to also enable the user to create of specific result criteria.
This could be any result metric that the user can express in a key-value fashion (HIC, peak pressure, star rating, etc.).
The diary does not need to concern itself with the actual determination of the criteria and its value. This can be done by a CAE post-processor of the users choosing.
CAE Job Diary only offers a way to document the criteria (which is the same for parameters).
The user interface to add new parameters/criteria should recommend other already existing parameter/criterion names. This should help avoid duplication (e.g. spelling differences).
As a future step, the criteria determination could be automated. This does not have to be through the diary, but the diary could access the automatically created result criteria. E.g. once the job is finished, a criteria file in is expected in the job dir (something structured and machine readable/parsable, like json).
- One job can have zero or many parameters/criteria related with it.
- A parameter/criterion can be used by one or many jobs.
- A parameter/criterion has a name/label.
- The parameter/criterion will have different values for different jobs, but only one value per job.
- There probably need to be a connecting model that creates the connection from job to parameter/criterion to value. job and parameter/criterion will be foreign keys.
- It should be possible to select jobs that have a certain parameter/criterion set.
- It should be possible to select all parameters/criteria of a job.
This might be the killer feature. Got this idea from a users feedback. That is the feature he is looking for to make the diary really useful to him.