Guidlines

External Reviewer Guidelines

Guidance for external reviewers supporting the HRI 2027 peer-review process.

Peer-Review Process Overview

The primary purpose of the HRI peer-review system is to systematically and fairly evaluate submissions for publication at the HRI conference. As such, there are many checks and experts involved along the way. The process is as follows:

Throughout this process you, as the external reviewer, will be in contact mainly with your 1AC—that is the program committee member who invited you to review a specific paper. If you are reviewing more than one paper, the 1AC of each paper might be different. You might also be contacted by track chairs or the program committee chairs (Paul Robinette, Fumihide Tanaka, Silvia Rossi) with reminders or other instructions.

Review Deadline

Reviews are due on October 20, 2026 (Tuesday) AoE. This is a hard deadline. After receiving all reviews, the 1AC will need to read them in detail and write a meta-review that summarizes all the reviews, within only a few days. Hence, your 1AC will greatly appreciate your timely submissions of the review. Any delay in submitting the reviews can result in delaying the whole process.

Conflict of Interest

To ensure objectivity in the review process, we ask you to please disqualify yourself from submitting a review for papers authored by people with whom you are in any of the following relationships, as per ACM policies for conflict of interest:

As we are following a double-blind review process for HRI 2027, whether or not you have a conflict with a paper might not be clear. If you suspect that you have a conflict, please contact your 1AC as well as the program chairs at pc2027@humanrobotinteraction.org.

Reviewer Responsibilities

Your main responsibilities as an HRI 2027 reviewer include:

All reviewers should follow the ACM peer review policy. This policy includes information about the use of Generative AI during the review process:

ACM Peer Review Policy

“Reviewers may use generative AI or other third-party tools with the sole purpose of improving the quality and readability of reviewer reports for the author, provided any and all parts of the review that would potentially identify the submission, author identities, reviewer identity, or other confidential content is removed prior to uploading into third party tools.”

Please be reminded that the use of generative AI for generating reviews is restricted and misuse will be reported.

Review Calibration and Quality

HRI is a premiere, highly competitive venue for publishing top-quality HRI work. Papers are expected to meet a high standard of originality, soundness, and quality of presentation. The acceptance rate of the conference is typically around 25%.

Reviewers play a key role in selecting which papers get accepted and helping authors improve their papers. The review needs to be of high quality (1) to help the program committee make an informed decision and (2) to give authors a positive experience, whether their paper is accepted or rejected.

To support point 1, reviews need to be detailed, claims backed by evidence (e.g., citations), and consistent, with numerical scores reflected in the verbal review (e.g., avoid highly positive reviews with a score of 2).

To support point 2, reviews must be professional, polite and constructive, and whenever possible, positively framed. Reviews will be checked throughout the process to ensure these criteria are followed.

Review quality is particularly important given that about three-quarters of the papers are rejected. If we are to expect authors to continue to submit their best work to this venue, authors need to know that their papers are carefully and fairly considered. Authors should be respected for the amount of work put into their submission, even if the paper has flaws. Negative, condescending, or unsubstantiated reviews will fracture our community by pushing people away. While writing your review, always put yourself in the author’s position: what type of feedback would you like to see that could actually improve your work?

We count on you to make HRI a more inclusive and welcoming community! Imagine you are a first-year graduate student who has worked on this paper for a year – is your review sufficiently thorough to provide the respect their work deserves? Is your review encouraging? There are ways to give negative feedback in a way that is still constructive and encouraging for the author to try again in the future.

Review Criteria

HRI is the place for innovative work, and we welcome big-idea, provocative, and visionary papers, even if they may not be perfectly implemented. HRI also invites broad participation from a range of fields involving humans and robots including robotics, psychology, artificial intelligence, cognitive science, HCI, human factors, organizational behaviour, simulation, design, anthropology, and many other fields.

Submissions should not have been previously published or be under simultaneous review for any other conference or journal. Please consult with your 1AC and the program chairs if you have concerns about pre-published or simultaneous-submission work.

Remember that no paper or study is perfect. Please keep an eye on potential impact and novelty, even in the face of minor issues with the paper. Favor slightly flawed, impactful work over perfectly executed, low-impact work.

Review Form Overview

The online review form has the following fields that the reviewer will respond to.

Suggested Outline for Detailed Review

While no review format is enforced, the following suggested outline covers the required components:

Summary — Start with a brief summary of the work, the main findings, and potential contributions.

Strengths and Weaknesses — List the strengths and weaknesses of the paper in brief paragraphs or a bulleted list form.

Detailed Comments — Expand on the strengths and weaknesses with more details. Discuss the contributions, make recommendations on design, implementation, and evaluation, provide pointers to additional relevant work, and add information that might benefit the authors’ research program. Make sure to comment on the track-specific review criteria.

Suggestions for Improvement — Provide concrete and actionable suggestions for improving the work and the paper. For example, give clear explanations of why something is a problem, and provide steps that should be taken to improve it. If limited related work is a problem, explain why and provide example references. If the paper lacks clarity, the review can pose specific questions that authors can address.

Recommendation — Conclude with a brief but clear statement of whether you recommend the paper to be included in this year’s program, including nuances of your opinion not clearly reflected in the numeric score. For example,

“I believe that this paper makes a significant contribution to HRI and should be included in this year’s program,”

“While the paper has some weaknesses, it could be included in the program, as it makes the following contributions: …,”

“Although the paper has some weaknesses, they can be addressed by making the following improvements: …,”

“While the work presented in the paper is promising, it does not seem to be ready for publication due to major weaknesses in …,”

“The paper does not seem to be appropriate for the topics and goals of the conference and the authors should be encouraged to submit their work elsewhere such as ….”

The review should clearly support the reviewer’s overall recommendation. The review should consider which weaknesses can be addressed through the rebuttal and camera-ready timeline, and specify what is needed in the rebuttal. We want the best work at HRI: if minor edits and reframing can make a paper publishable, propose this to the authors. If addressing the critical weaknesses of the paper would require major work (e.g., data collection, new analysis, major restructuring) then the paper should be recommended for rejection.

The program committee member who recruited you will be checking your review for quality and might ask reviewers to clarify points in a review or rephrase certain points. If you have any questions about the paper or the process, please contact your 1AC.

Paper Format, Length, Accessibility, Anonymization

All papers should follow the paper preparation guidelines provided on the HRI 2027 website. These guidelines cover:

Reviewers should keep an eye on the papers they review to ensure they follow these guidelines. If you encounter a submission that does not follow these guidelines, please contact your 1AC and the program chairs.

The length of the paper should be proportional to its contributions, rather than trying to “fill up” an 8-page submission. You should not negatively rate a paper if it is shorter than the maximum allowed length. In addition, you can weigh the length of the paper in making your assessment of the contribution.

While we expect authors to follow the guidelines for anonymizing submissions, we recognize that there are cases in which complete anonymization becomes difficult (e.g. prior publication in public archives, a very unique robot, …). If you feel that knowing the full identity of the authors might be biasing your review, please contact your 1AC as soon as possible so another reviewer can be assigned. Remember that just because you can figure out who the authors are, does not mean that the authors failed to anonymize properly. As long as the authors followed the guidelines above, the paper is considered sufficiently anonymous for the review process.

Full Paper Tracks

Each paper is assigned to a primary track which determines the subcommittee that will discuss and make decisions about the paper. There are five different tracks at HRI 2027:

Reviewers should read the information about the different tracks provided for authors on the HRI 2027 webpage. Although 1ACs will ensure that the expertise of the reviewers matches the track for which a paper is being considered, reviewers would benefit from understanding the track and their differing expectations. If you are unsure about the track of the paper you are reviewing or the expectations for the track you are reviewing for, please contact your 1AC.

Post-Rebuttal Discussion and Review Updates

Authors will have the opportunity to respond to your review with a rebuttal after they receive the reviews and meta-reviews. Rebuttals will be submitted by November 3, 2026 AoE and the discussion phase will begin immediately after.

If the authors submit a rebuttal, you will be contacted by your program committee member to read the rebuttal, enter into a discussion, and possibly modify your review. Typically, you append a section titled “Post Rebuttal Comments” to your review, where you acknowledge the rebuttal and indicate whether it addressed any questions or concerns you had posed in your original review. You can also mention any changes to your score in that section.

Please note that rebuttals are supposed to be self-contained within the 5000-character text box. It should not contain links to external resources. Please disregard any links within a rebuttal, and just read the rebuttal text itself.

The rebuttal is intended solely to clarify potential misunderstandings or errors in the interpretation of the submission and should not be regarded as an opportunity to perform major revisions on its content.

Please carefully consider any rebuttal submitted and adjust your review accordingly. It can be easy to lock into your initial recommendation, so try to keep an open mind. This is the last chance you will have to affect the review process!

Contact

Program Chairs: Paul Robinette, Fumihide Tanaka, Silvia Rossi

Email: pc2027@humanrobotinteraction.org