Topics of Interest

ACM REP ‘23 invites submissions in several categories as described below with focus on computing disciplines within computer science and across scientific computing disciplines of biology, chemistry, physics, astronomy, genomics, geosciences, etc. The conference encourages submissions in which experimental results are reproducible in of themselves and, if not, then sufficiently documents the reproducibility experience.

Topics of interest include, but are not limited to, the following as they relate to various aspects of reproducibility and replicability.

Reproducibility Concepts

  • Experiment dependency management.
  • Experiment portability for code, performance, and related metrics.
  • Software and artifact packaging and container-related reproducibility methods.
  • Approximate reproducibility.
  • Record and replay methods.
  • Data versioning and preservation.
  • Provenance of data-intensive experiments.
  • Automated experiment execution and validation.
  • Reproducibility-aware computational infrastructure.
  • Experiment discoverability for re-use.
  • Approaches for advancing reproducibility.

Reproducibility Experiences

  • Experience of sharing and consuming reproducible artifacts.
  • Conference-scale artifact evaluation experiences and practices.
  • Experiences as part of hackathons and summer programs.
  • Classroom and teaching experiences.
  • Usability and adaptability of reproducibility frameworks into already-established domain-specific tools.
  • Frameworks for sociological constructs to incentivize paradigm shifts.
  • Policies around publication of articles/software.
  • Experiences within computational science communities

Broader Reproducibility

  • Cost-benefit analysis frameworks for reproducibility.
  • Novel methods and techniques that impact reproducibility.
  • Reusability, repurposability, and replicability methods.
  • Long-term artifact archiving and verification/testing for future reproducibility.

Submission Guidelines

We solicit papers describing original work relevant to reproducibility and independent verification of scientific results. And not published or under review elsewhere. ACM REP is a single-blind reviewed conference. Therefore, authors must include their names and affiliations on the first page. ACM REP submissions can be either research papers, surveys, vision, and experience papers. Papers will be evaluated according to their significance, originality, technical content, style, clarity, relevance, and likelihood of generating discussion. Authors should note that changes to the author list after the submission deadline are not allowed without permission from the PC Chairs. At least one author of each accepted paper is required to register for, attend, and present the work at the conference.

Remote attendance and presentations are welcome even though in-person attendance and presentations are highly encouraged.

Research Papers (Long and Short)

We solicit both full length papers (10 pages) and short papers (4 pages). The former tend to be descriptions of complete technical work, while the latter tend to be descriptions of interesting, innovative ideas, which nevertheless require more work to mature. The program committee may decide to accept some full papers as short papers. Full papers will be given a presentation slot in the conference, while short papers will be presented in the form of posters. All papers regardless of size, will be given an entry in the conference proceedings. The requested page limit is without references. Authors may optionally include reproducibility information that allows for automated validation of experimental results (see artifact evaluation criteria) Accepted submissions passing automated validation will earn the ACM Reproducibility badges in accordance with the artifact review and validation policy.

Artifact Evaluation Criteria

The conference will also be soliciting code/data artifacts. For submitted papers, these artifacts will be optional supplemental material and solicited based on the PC criteria. The artifacts will be mandatory for accepted full papers with experimental results. The artifacts will be reviewed by an Artifact Evaluation committee, and those that pass will be awarded Reproducibility Badges per ACM policy.

Formatting

Papers must be submitted in PDF format according to the ACM template published in the ACM guidelines, selecting the generic “sigconf” sample. The PDF files must have all non-standard fonts embedded. Papers must be self-contained and in English. If submitting a short paper, authors must indicate “SHORT:” at the beginning of the title. The review process is single-blind.

Submission Site

The conference submission site is https://acm-rep22.hotcrp.com (yes, it is the right link for ACM REP ‘23 submissions)

Important Dates

Paper submission (Long and Short): February 6, 2023 February 20, 2023
Notification of acceptance: May 12, 2023
Camera-ready copy: June 1, 2023
Conference: June 27-29, 2023

Program Committee

Program Chairs

Tanu Malik (DePaul University)
Jay Lofstead (Sandia National Laboratories)

Program Committee

Vijay ChidambaramUniversity of Texas at Austin
Roberto Di CosmoINRIA / IRIF / Université Paris Cité
Ludovic CourtèsINRIA
Juliana FreireNYU Tandon
Fraida FundNYU Tandon
Todd GamblinLawrence Livermore
Ananth GramaPurdue University
Paul GrothUniversity of Amsterdam
Haryadi GunawiUniversity of Chicago
Daniel S. KatzUniversity of Illinois
Kate KeaheyArgonne National Laboratory / University of Chicago
Ignacio LagunaLawrence Livermore
Bertram LudaescherUniversity of Illinois, Urbana Champagne
Reed MilewiczSandia National Laboratories
Thomas PasquierUniversity of British Columbia
Limor PeerYale
Solal PirelliEPFL
Beth PlaleIndiana University
Line PouchardBrookhaven National Lab.
Birali RuneshaUniversity of Chicago
Vicky RampinNYU
Mahadev SatyanarayananCarnegie Mellon University
Stefanie ScherzingerUniversity of Passau
Salvatore SignorelloUniversity of Lisbon
Victoria StoddenUniversity of Southern California
Michela TauferUniversity of Tennessee, Knoxville
Divesh TewariNortheastern Univ.
Douglas ThainUniversity of Notre Dame
Anjo Vahldiek-OberwagnerIntel Labs
Stefano ZacchiroliPolytechnic Institute of Paris / Software Heritage
Theo ZimmermanTélécom Paris