ISAV 2021 is participating in the SC21 Reproducibility Initiative. As part of that initiative, each SC20 tech paper submission must include an Artifact Description (AD) Appendix.
ISAV 2021 does not require authors of short-paper submissions to include Artifact Description (AD) Appendix. While we will not disqualify a paper based on information provided or not provided in this appendix, nor if the appendix is not available, the availability and quality of an appendix will be used in ranking a paper.
To find more information about and access to resources helpful in preparing your AD for your submission, visit the SC21 AD/AE Appendix FAQ and Author Kit page, which contains more information about the AD/AE appendices, and links to resources for authors.
Q. Is the Artifact Description appendix required in order to submit a paper to ISAV 2021?
A. No. These appendices are not required. If you do not submit any appendix, it will not disqualify your submission. At the same time, if two papers are otherwise comparable in quality, the existence and quality of appendices can be a factor in ranking one paper over another.
Q. Who will review my appendices?
A. The Artifact Description and Computational Results Analysis appendices will be submitted at the same time as your paper and will be reviewed as part of the standard review process by the same reviewers who handle the rest of your paper.
Q. What are my responsibilities when reviewing a paper that has an Artifacts Description appendix?
A. Reviewers are expected to read any appendix that is submitted with a paper. The information in these appendices is often included in any well-written paper. By collecting this information in a uniform way, we hope that the review process will be more effective without requiring a significant new burden on reviewers.
Q. Must I confirm that links in an appendix work?
A. Confirming links in an appendix are not required of every reviewer. We are asking for volunteers from the review committee who would be willing to validate links provided in appendices.
Q. Must I evaluate the code and results associated with a Computational Results Analysis appendix?
A. Code and results evaluations associated with a Computational Results Analysis appendix are not required of every reviewer. We are asking for volunteers from the review committee who would be willing to evaluate code and results associated with these appendices.
Q. How may I use appendices as part of the review process?
A. The absence or poor quality of an appendix cannot be used to categorically disqualify a paper. Furthermore, inaccessibility of artifacts such as source code cannot be used to categorically disqualify a paper. However, the presence and quality of an appendix can be used as part of the review process. In particular, if two papers have similar ranking based on the manuscripts, the availability and quality of appendices can provide additional evaluation criteria.