cgo18ae-review-93
==+== ===================================================================== ==+== Begin Review ==+== Reviewer: Gurbinder Gill gill@cs.texas.edu
==+== Paper #93 ==-== Title: High Performance Stencil Code Generation with LIFT
==+== A. Artifact publicly available?
==-== Are all artifacts related to this paper are publicly available?
==-== Note that it is not obligatory to make artifacts publicly
==-== available!
==-==
==-== The author-created artifacts relevant to this paper will receive
==-== an ACM "artifact available" badge only if they have been placed
==-== on a publically accessible archival repository.
==-== Choices:
==-== 1. Publicly available
==-== 2. Not publicly available
==-== Enter the number of your choice:
1
==+== B. Artifact functional?
==-== * Package complete?
==-==
==-== * All components relevant to evaluation are included in the
==-== package?
==-==
==-==
==-== * Well documented?
==-==
==-== * Enough to understand, install and evaluate artifact?
==-==
==-==
==-== * Exercisable?
==-==
==-== * Includes scripts and/or software to perform appropriate
==-== experiments and generate results?
==-==
==-==
==-== * Consistent?
==-==
==-== * Artifacts are relevant to the associated paper and contribute in
==-== some inherent way to the generation of its main results?
==-==
==-==
==-==
==-== The artifacts associated with the paper will receive an "Artifacts
==-== Evaluated - Functional" badge only if they are found to be
==-== documented, consistent, complete, exercisable, and include
==-== appropriate evidence of verification and validation.
==-== Choices:
==-== 1. Exceeded expectations
==-== 2. Met expectations
==-== 3. Fell below expectations
==-== Enter the number of your choice:
1
==+== C. Artifact customizable and reusable?
==-== Can this artifact and experimental workflow be easily reused and
==-== customized? For example, can it be used on a different platform,
==-== with different benchmarks, data sets, compilers, tools, under
==-== different conditions and parameters, etc.?
==-==
==-== The artifacts associated with the paper will receive an "Artifact
==-== Evaluated - Reusable" badge only if they are of a quality that
==-== significantly exceeds minimal functionality. That is, they have
==-== all the qualities of the Artifacts Evaluated - Functional level,
==-== but, in addition, they are very carefully documented and
==-== well-structured to the extent that reuse and repurposing are
==-== facilitated. In particular, norms and standards of the research
==-== community for artifacts of this type are strictly adhered to.
==-== Choices:
==-== 1. Exceeded expectations
==-== 2. Met expectations
==-== 3. Fell below expectations
==-== Enter the number of your choice:
2
==+== D. Results validated?
==-== Can all main results from the paper be validated using provided
==-== artifacts?
==-== Report any unexpected artifact behavior (depends on the type of
==-== artifact such as unexpected output, scalability issues, crashes,
==-== performance variation, etc).
==-==
==-== The artifacts associated with the paper will receive a "Results
==-== replicated" badge only if the main results of the paper have
==-== been obtained in a subsequent study by a person or team other than
==-== the authors, using, in part, artifacts provided by the author.
==-==
==-== Note that variation of empirical and numerical results is
==-== tolerated. In fact it is often unavoidable in computer systems
==-== research - see "how to report and compare empirical results?" in
==-== AE FAQ!
==-== Choices:
==-== 1. Exceeded expectations
==-== 2. Met expectations
==-== 3. Fell below expectations
==-== Enter the number of your choice:
2
==+== E. Comments for author ==-== Markdown styling and LaTeX math supported.
Few comments:
- Some of the path variables were hardcoded in the scripts, for example, DATADIR and OpenCL paths in the benchmarks were hard coded.
==+== F. Comments for PC ==-== Hidden from authors. ==-== Markdown styling and LaTeX math supported.
I used the following configuration for the artifact evaluation:
-
Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
-
GPU: Tesla K80
-
gcc/6.1
-
cuda/8.0
-
The documentation provided with the artifact was thorough and made the evaluation process very easy.
-
Authors did a very good job specifying the workflow, downloading input data set, setting up the required 3rd party platforms and running experiments to produce the results and plots used in the paper.
-
Results were consistent with those reported in the paper.
==+== Scratchpad (for unsaved private notes)
==+== End Review