Note: The following is valid for full paper submissions. Other submission types follow a similar, but typically simpler review process.
Summary: The HRI conference has a rigorous peer review process. All full papers are typically read by three external reviewers and two area chairs. Most of the papers are discussed at the program committee meeting, where final decisions on acceptance or rejection are made.
Program Committee Members and Roles
Program Committee Co-chairs (or simply Program Chairs, PCCs) organize and oversee the whole review process, select TCs and ACs, prepare the conference program, etc.
Track Chairs (TCs) participate in selecting ACs for their corresponding track, assign papers to ACs, and lead discussions at the PC meeting.
Area Chairs (ACs) get assigned several of the submitted papers (typically about 5) as 1AC (primary area chair), and a separate set of papers as 2AC (secondary area chair).
As 1AC they:
- Recruit reviewers for the papers.
- Upon receiving the reviews, write a meta-review.
- After the authors’ rebuttal is received, lead the discussion with external reviewers.
- Participate in the program committee meeting.
- Update the reviews with post PC meeting comments.
As 2AC they:
- based on reviews, rebuttal, and discussions, write a grand meta-review before the PC meeting.
Review Process Flow
An overview of the review process is shown in the figure.
Initial checks, desk rejections, track changes
Upon initial submission, the PCCs, TCs, ACs perform an initial check of the papers that they are in charge of. This includes desk rejections and track changes.
During checks it is determined whether the submission is appropriate for HRI or should be desk rejected. See the Full Paper page for more details about desk rejects.
The paper track that the authors have chosen is also checked at this time. If it is deemed necessary for an appropriate evaluation, the paper may be assigned to a different track.
External reviews
TCs assign each paper to one 1AC and one 2AC. 1ACs proceed with finding three reviewers for each paper. These external reviewers are chosen from experts in the topic of the paper (who are not in the program committee). External reviewers need to submit their review before the deadline, usually within a few weeks.
The external reviewers read the paper in detail and provide an assessment of the paper’s quality and the significance of its contributions, as well as clear feedback and suggestions for authors to improve their paper. They are expected to keep the submissions confidential, provide reviews in a timely manner, write reviews that are concrete, informative, constructive, and thorough, as well as polite, professional, and kind.
Meta-reviews and notification of results
When all three external reviews are in, 1AC writes their review of the paper (the 1AC meta-review), which is typically a summary of the main points of contention from external reviews. After that the review notifications including the external reviews and the 1AC meta-review are sent to the authors.
Rebuttals
During the rebuttal phase the authors have an opportunity to respond to the initial reviews of their submitted papers. In their rebuttal authors can clarify misunderstandings, correct inaccuracies, provide additional information, or address specific concerns raised by the reviewers. The goal is to ensure that reviewers and program committee members have a complete and accurate understanding of the work when making their final judgment. The rebuttals are an important part of the HRI full paper review process and often affect the scores and the final decision to accept or reject a paper. It is thus recommended that the authors prepare and submit a strong rebuttal.
Discussions and 2AC review
Once the rebuttal phase is over, the 1AC initializes an online discussion with the external reviewers. During the discussion the reviewers and 1AC may debate the paper issues and strengths and the author’s rebuttal, and possibly change their scores. Typically they also add a post rebuttal comment to their reviews.
Finally, the 2AC assigned to a paper checks the paper and the whole review process up to that point and writes the 2AC grand-review, which is a last check to assure that the paper was treated fairly and thoroughly.
Program Committee meeting (PC meeting)
All PCCs, TCs, and ACs convene at a PC meeting to discuss the submissions. At the PC meeting the members discuss within their own subcommittees. There is one subcommittee per track, except for the User Studies track, which typically receive the largest number of submissions and may therefore be split into multiple subcommittees. During the meeting, each subcommittee discusses one by one each paper that they are in charge of and a decision is made whether to accept or reject it. All papers for which the reviewer scores crossed a certain threshold are discussed. In addition, the 1AC and 2AC for a paper can choose to flag a paper for discussion, even if the scores were low.
Notification of results
After the PC meeting, the assigned 1AC adds a post PC meeting comment to their review for all the papers that were discussed, after which the final decision notifications are sent to the authors.