Organizers create evaluation plans, which are containers holding collections of artifacts to be reviewed. Organizers also must add people as evaluators. Paper authors are given invitations to submit for AE, which allows them to tag their objects, experiments, and any other item that belongs to them as artifacts.
These artifacts are then reviewed by assigned evaluators, as chosen by the organizers/chair.
AE artifacts are only a logical mapping to objects/worksets/etc. No duplicate objects are created. Artifacts should be revision specific, but easily updatable (so authors may have productive back and fourth with evaluators).
Once AE has concluded, organizers publish AE results. When this occurs, users may view their score and comments privately.
User permission protection
Traditionally in OCCAM, permissions are set per person. The evaluators assigned by the organizers will be able to override the ability to see and run these objects. This allows evaluators to anonymously carry out their reviews, and does not promiscuously grant elevated privileges to evaluators as a whole.
During AE, run logs generated by evaluators will exist in a neutral space (as to not pollute the original object), and will be visible to the authors for evidence of problems that arise.