The ACM/IEEE International Conference on Human-Robot Interaction is a premier, highly-selective venue presenting the latest advances in Human-Robot Interaction. The 16th Annual HRI conference theme is “Bolder HRI.” The conference seeks contributions from a broad set of perspectives, including technical, design, behavioural, theoretical, methodological, and metrological, that advance fundamental and applied knowledge and methods in human-robot interaction. Full papers will be archived in the ACM Digital Library.
General Notes and Covid-19
We acknowledge that many authors are concerned with how the pandemic has affected the feasibility of conducting in-person user studies. We want to stress that in-person user studies are not the only means of testing a research question and there are other ways to contribute to knowledge. Systematic reviews, online studies, and video-based assessments are examples of alternative methods. We encourage authors to consider other ways that contributions can be validated. We will stress in the reviewing guidelines that, for many of the submitted works, in-person user studies might not necessarily be the most appropriate, nor an indispensable method to demonstrate validity or provide valuable insights to the research community. We understand that Covid-19 has seriously impacted face-to-face research in many contexts. Covid-19 presents many challenges but it also offers opportunities for the HRI community to conduct research into ways robots can reduce the impact and spread of pandemics.
October 5, 2020 (23:59 AoE): Submission Deadline
November 20, 2020: Review Notification, Rebuttal Period Begins
November 24, 2020: Rebuttal Period Ends
December 7, 2020: Decision Notification
January 5, 2021: Camera-ready Papers Due
March 8-11, 2021: Conference
Format and Submission
Full papers are up to eight camera-ready pages, including figures, but excluding references. Submissions longer than eight pages of content excluding references will be desk rejected and not reviewed. Accepted full papers will be published in the conference proceedings and presented in an oral session. The HRI conference is highly selective with a rigorous, two-stage review model that includes an expert program committee meeting where papers are extensively discussed. As such, all submissions are expected to be mature, polished, and detailed accounts of cutting-edge research described and presented in camera-ready style. In cases of equally qualified papers, positive consideration will be given to submissions that address this year’s theme, “Bolder HRI.”
All papers for the conference must be submitted in PDF format and conform to the ACM SIG proceedings specifications. Please note that we are following the general ACM SIG format, not the SIGCHI format. Authors should use the sample-sigconf.tex or interim_layout.docx template files. In addition, the ACM has partnered with Overleaf, where authors can start writing using this link directly (note that this Overleaf document uses the new ACM workflow by default, which is not what HRI is using; to fix this, make sure the document uses the “sigconf” document class, rather than the “acmart” document class that is enabled in the Overleaf document by default).
Microsoft Word Template
For Microsoft Word users, use the template in the above link under “Word Authors.” The template linked is called “interim-layout.docx” (it is the only word template for proceedings on the page; alternatively, you can download this template directly here). As ACM instructs, please disregard any embedded instructions in the Interim layout. This document refers to processes that ACM may use in the future.
For LaTeX users, navigate to the section titled “LaTeX Authors” in the above link and download the ACM master template .zip titled “LaTeX (Version 1.71)” (alternatively, you can download the .zip directly here). From there, unzip the master template folder and use the “sigconf” template within it (i.e., see the sample-sigconf.tex within the “samples” folder). Make sure that you use “sigconf” for your document class.
For users wishing to use Overleaf, navigate to the section titled “LaTeX Collaborative Authoring Tool on Overleaf Platform” from the above link. From there, click on the link to the ACM Overleaf templates (alternatively, you can navigate to the templates directly by clicking here). You should then use the “ACM Conference Proceedings “Master” Template. However, note that this template is set up by default to use the “New Workflow” (i.e., single-column format), which you must fix. To fix this, simply comment out the “manuscript” document class and uncomment the “sigconf” document class, as in the image below:
In connection with SIGCHI’s efforts to improve conference accessibility, HRI is moving towards generating accessible PDFs for all accepted papers. This year, we encourage all authors to add accessibility features to their PDF submissions. Please see the CHI 2021 guidelines for information on how to prepare an accessible PDF: https://chi2021.acm.org/for-authors/presenting/papers/guide-to-an-accessible-submission
If you have questions or need assistance, please contact the conference accessibility chairs at firstname.lastname@example.org
Requesting accessible PDFs:
If you need an accessible PDF to review, please send an email to email@example.com at your earliest convenience and we will arrange to get you an accessible version of the PDF.
To enable accessible PDF creation, we request that authors provide alt text for their figures (uploaded as a supporting file as part of their submission). For a guide to writing proper alt text for figures, please see:
The HRI 2021 review process is double-blind; every aspect of all submissions must be properly anonymized (see the anonymization guidelines). Any submission that contains any element (e.g., full paper document, artifact, or supplementary materials) that violates the anonymization guidelines will be desk rejected.
Authors must carefully manage their writing if submitting multiple papers concurrently to HRI 2021 that would include citations to each other in order to not violate the anonymization guidelines. The best way of handling this type of situation is to refer to the submission numbers in the respective papers. Each submission will be assigned a submission number once the submission has been created in the precision conference system. Once a submission is created, authors can use this submission number in their paper text, e.g., “While our concurrent submission #1234 provides more detail on this aspect of our experiment, here we focus on…”
All submissions must use only “Type 1” (scalable) fonts (not bitmapped fonts), an ACM digital library requirement.
Posting articles publicly online (website / arXiv)
To maintain the double-blind review process, we are requesting authors refrain from posting their articles online until December 7, 2020 (the notification date). Posting before this date may lead to a violation of the anonymization guidelines, resulting in a paper being desk rejected.
Authors will be expected to review a paper if they’re asked by the Program Committee.
Supplementary Materials and Artifacts
This year, authors have the opportunity to upload up to three supplemental files in conjunction with their full paper submission. These materials may be submitted via the “Supporting File 1,” “Supporting File 2,” and “Supporting File 3” sections within the full papers submission form in the precision conference submission system. While authors are encouraged to upload all supplementary materials directly, it may be infeasible to upload certain items directly (e.g., large data sets or code repositories). In this case, authors may upload a document with a link to where these anonymized supplemental materials are hosted.
Supplementary materials are not required for a submission. If authors do choose to submit supplementary materials, such materials may not be used to get around the page limit for full papers. It is important that any supplemental materials that are uploaded are also properly anonymized. Any submission that contains any element (full paper or supplementary materials) that violates the anonymization guidelines will be desk rejected.
In general, there are three main types of supplemental materials that may be submitted: videos, appendices, and artifacts (e.g., software, hardware, data sets, etc.):
Authors may upload an appendix directly or provide a link to supporting appendix material hosted anonymously online. Appendices are only for supplementary materials that would interrupt the flow of the text if presented in the main document. Examples may be questionnaires used as measurements, tables of supplementary data, or figures of experimental apparatus. Additional experimental analysis, additional results, and lengthy text that further clarifies aspects of the full paper submission is not appropriate for an appendix (i.e., full paper submissions must stand on their own without requiring further explanation, analysis, or results provided through appendix information). Appendices may not be used to get around the 8-page limit for full paper submissions. Any submissions that attempt to use appendices or supplementary materials in general in a manner that violates the page limits will be desk rejected.
Authors may submit a 1-minute video (up to 100 MB) as a supplement to their full paper. Videos are not mandatory but may be helpful to visibly showcase a working system, experimental conditions, environment context, results, etc. Only MPG, MPEG or MP4 video formats can be used. Ensure that videos are properly anonymized prior to submission.
Across all themes, we encourage submissions that introduce a novel “artifact” as an enabler to reproducibility, replicability, and recreation of HRI research, and/or to support new lines of HRI research. An artifact could be software, hardware, data sets, protocols, new evaluation measures, etc. Submissions should contain a detailed description of the artifact introduced, proposed, or implemented, as well as information about how it is novel and different from other existing artifacts, and, if possible, a link to an anonymized, live version of the artifact at time of submission for review.
Authors submitting artifacts must provide relevant details regarding any aspects related to artifact clearance and release (e.g., obtaining Institutional Review Board (IRB) clearance for releasing data collected by human participants, organizational clearances for the release of software/hardware, etc.). Ensure that artifacts are properly anonymized prior to submission. Any submission that includes any element (full paper, artifact, or supplementary materials) that does not follow the anonymization guidelines will be desk rejected.
Examples of papers with artifacts:
Systems paper: Huang, et al. (2017). Code3: A system for end-to-end programming of mobile manipulator robots for novices and experts. HRI 2017.
Dataset paper: Celiktutan, et al. (2017). Multimodal Human-Human-Robot Interactions (MHHRI) Dataset for Studying Personality and Engagement. IEEE Transactions on Affective Computing 2017.
Benchmarking paper: Wisspeintner et al. (2010). RoboCup@Home: Results in Benchmarking Domestic Service Robots. RoboCup 2009.
Selecting a Theme
To facilitate quality interdisciplinary reviewing, and to inform reviewer selection, authors will be required to select one main theme for each submission. They may optionally also select a second theme for their full paper submissions. It is important for authors to carefully select the theme as it will have an impact on how the submission is evaluated and which reviewers are recruited. It is recognized that papers may not clearly fit within one theme. Consider the primary contribution to make the selection and be sure to select the appropriate sub-theme based on the theme descriptions below. While authors will suggest a primary theme, the program chairs may move the paper to a different theme to improve fit.
The HRI 2021 conference has four themes: User Studies, Technical Advances, Design, and Theory and Methods. Papers may have overlap between themes, but authors are encouraged to consider the main contribution of the work using this brief rule of thumb:
Human-Robot Interaction User Studies
The primary contribution is human-focused, e.g., how humans perceive, interact with, or otherwise engage with robots.
Technical Advances in Human-Robot Interaction
The primary contribution is robot-focused, e.g., systems, algorithms, or computational methods supporting HRI.
Human-Robot Interaction Design
The primary contribution is design-focused, e.g., new morphologies, behavior paradigms, and interaction capabilities for robots.
Theory and Methods in Human-Robot Interaction
The primary contribution is methodological, e.g., new ways of studying HRI, elucidating or connecting fundamental HRI principles beyond individual interfaces or projects, new theoretical concepts in HRI, literature reviews, work that focuses on reproducing, replicating, or recreating prior HRI work (or fails to), etc.
Across all themes, if a paper includes a study with human participants:
To support building a strong evidence base in HRI, and encourage future reproducibility of published work, all submissions involving studies with human participants should clearly outline their methodology, including:
- ethical aspects considered and clearance obtained where appropriate (c.f., Geiskkovitch et al. 2016, Sections 5.2, 5.4)
- participant demographics and sampling approach (c.f., de Graaf 2017, Section 2.3)
- data collection and analysis methods (c.f., Paepcke and Takayama 2010, Section V)
- study environment and context (c.f., Short et al 2018, Section 3.5)
- if a Wizard-of-Oz paradigm was used, a detailed description of the robot, wizard, user, etc. (c.f., Riek 2012, Table 2)
- if a robot was used, a detailed description of the platform, its level of autonomy, capabilities, etc. (c.f., Beer et al. 2014, Figure 5)
To help authors understand which theme might be the best fit for their work, more details on each of the four main themes for HRI 2021 is provided below.
1. HUMAN-ROBOT INTERACTION USER STUDIES
This theme is for research contributions that provide new knowledge of human-robot interactions derived from data and analysis of humans and robots in laboratory or in-the-wild settings. Work can include data and analysis that is quantitative, qualitative, or both. It may be formative or summative in nature, can be hypothesis-driven or exploratory, and take a positivist or interpretivist approach. Studies can employ robots across the autonomy spectrum. Video-based and/or virtual user study paradigms are acceptable with appropriate/sufficient motivation and clarifications regarding any limitations that such a methodology introduces, though authors are encouraged to use in-person robots wherever possible. Successful submissions should reflect rigorous empirical methodologies and analyses that yield novel insights into human-robot interaction and should discuss the limitations and generalizability of the methods used.
User Studies papers should include clear consideration of their methods’ reliability as well as internal, external, and ecological validity. For example, the measures used should be validated either in prior work or within the given paper. If the focus of the paper is the development of new measures themselves, rather than using measures to derive new insights into human-robot interactions, authors should instead consider submitting to the Theory and Methods theme.
Work that partially or fully reproduces, replicates, or recreates a prior study (or fails to) as part of the contribution is welcome within the User Studies track. However, if the primary focus of the paper is replication and the knowledge gained regarding HRI theories or methodologies based on successful or unsuccessful replication, it should instead belong in the Theory and Methods theme. Likewise, work that is primarily on methodological advancements or analysis may also belong in the Theory and Methods theme. Papers that provide novel interaction techniques or designs as a primary contribution, but include a detailed user study, may belong in the Technical Advances or Design themes.
Theme chairs: Takayuki Kanda and Friederike Eyssel
- Velner et al. (2020). Intonation in Robot Speech: Does it Work the Same as with People? HRI 2020.
- Wojciechowska, et al. (2019). Collocated Human-Drone Interaction: Methodology and Approach Strategy. HRI 2019.
- Fraune, et al. (2019). Is Human-Robot Interaction More Competitive Between Groups Than Between Individuals? HRI 2019.
- Bremner, et al. (2016). Personality Perception of Robot Avatar Tele-operators. HRI 2016.
- Leite, et al. (2012). Modelling empathic behaviour in a robotic game companion for children: an ethnographic study in real-world settings. HRI 2012.
2. TECHNICAL ADVANCES IN HUMAN-ROBOT INTERACTION
This theme is for research contributions that provide novel robot systems, algorithms, interface technologies, and computational methods supporting human-robot interaction. This includes contributions that enable robots to better understand, interact with, and collaborate with people, including both collocated and distal interaction. Submissions must present full details of the proposed technological advance to facilitate in-depth review and enable future reproducibility (e.g., formal descriptions, pseudocode, or open-sourced code). Successful papers will clearly demonstrate how the technology improves or enables human-robot interaction and will include evaluation appropriate to the work (e.g., comparisons to other methods, standard machine learning or computer vision metrics, usability studies, etc.).
The Technical Advances theme welcomes contributions that include artifacts as part of the technical contribution, such as new datasets, benchmarks, or open source software releases. The Technical Advances theme also welcomes contributions that present novel systems created through integrative efforts that combine elements of prior technologies in new ways. However, if the primary focus of the work is recreating or replicating prior technical advances, rather than integrating them into new systems, the paper should instead belong in the Theory and Methods theme. Likewise, if the primary contribution of the paper is the methodologies and theories underlying technical developments in HRI, it may instead belong in the Theory and Methods theme. If the primary focus of a paper is on the evaluation of an interaction or generalizable knowledge gained about human-robot interactions, rather than specific technologies supporting interactions, then it may instead belong in the User Studies theme. If the primary focus is on designing new systems and interactions with an emphasis on the design process, rather than the technologies behind it, it may instead belong in the Design theme.
Theme chair: Harold Soh
- Bobu et al. (2020). LESS is More: Rethinking Probabilistic Models of Human Behavior. HRI 2020.
- Petric, et al. (2019). Hierarchical POMDP Framework for a Robot-Assisted ASD Diagnostic Protocol. HRI 2019.
- Roesler, et al. (2019). Evaluation of Word Representations in Grounding Natural Language Instructions Through Computational Human-Robot Interaction. HRI 2019.
- Short, et al. (2019). SAIL: Simulation-Informed Active In-the-Wild Learning. HRI 2019.
- Clark-Turner, et al. (2018). Deep reinforcement learning of abstract reasoning from demonstrations. HRI 2018.
3. HUMAN-ROBOT INTERACTION DESIGN
This theme is for design-centric research contributions to human-robot interaction. This includes the design of new robot morphologies and appearances, behavior paradigms, interaction techniques and scenarios, and interfaces. The design research should support unique or improved interaction experiences or abilities for robots. Research on the design process itself or proposing new design strategies, frameworks, or models as relevant to human-robot interaction are also welcome. Submissions must fully describe their design outcomes or process to enable detailed review and replication of the work. Further, successful papers will have evaluation appropriate to the work, for example end-user evaluation or a critical reflection on the design process or methodology.
If a paper’s primary focus is on a technical system or novel algorithm, rather than knowledge and insights gained from the design of such a system or algorithm, it may instead belong in the Technical Advances theme. If the main contribution is an in-depth study that reflects on a broader interaction question, it may instead belong in the User Studies theme. The Design theme welcomes work that includes or integrates elements which recreate and/or replicate prior designs; however, if a paper’s primary focus is recreating or replicating an existing design concept or artifact, it may belong in the Theory and Methods theme.
Theme chair: Guy Hoffman
- Vilk and Fitter (2020). Comedians in Cafes Getting Data: Evaluating Timing and Adaptivity in Real-World Robot Comedy Performance. HRI 2020.
- Moharana, et al. (2019). Robots for Joy, Robots for Sorrow: Community Based Robot Design for Dementia Caregivers. HRI 2019.
- Azenkot, et al. (2016). Enabling Building Service Robots to Guide Blind People: A Participatory Design Approach. HRI 2016.
- Sirkin, et al. (2015). Mechanical Ottoman: How Robotic Furniture Offers and Withdraws Support. HRI 2015.
- Pantofaru, et al. (2012). Exploring the role of robots in home organization. HRI 2012.
4. THEORY AND METHODS IN HUMAN-ROBOT INTERACTION
This theme is for research contributions that improve our understanding and ability to study fundamental HRI principles that span beyond individual interfaces, systems, studies, or projects. Such contributions may include detailing underlying interaction paradigms, introducing theoretical concepts, providing new interpretations of previously known results, developing novel evaluation methodologies, or improving the base of evidence for the field of HRI through reproducing, replicating, or recreating prior HRI/HRI-relevant work (note: this refers to the entire field, not only papers published at the ACM/IEEE HRI conference).
While work that includes elements of recreation or replication are welcome in any theme, research whose central focus is recreation/replication and the resulting knowledge gained from successful or unsuccessful replication should be submitted to the Theory and Methods theme. Reproducibility efforts may focus on prior quantitative or qualitative HRI work, or both. Such work may focus on evaluating the reliability of prior observations through direct reproductions, where an author seeks to obtain the same results from an independently conducted study, using procedures and methods as closely matched to the original study as possible. Alternatively, reproducibility efforts may build upon prior evidence to understand under what conditions, and for whom, a prior HRI finding holds true through the use of conceptual reproductions, where an author seeks to obtain the same results from an independently conducted study while systematically varying procedures and methods. Authors seeking to reproduce, replicate, or repeat quantitative work are encouraged to follow guidelines developed by the US National Science Foundation and Dept. of Education on how to design, conduct, and report such studies (See: [NSF 2018], pages 4-5). Authors should also provide clear motivations for choosing the specific work they are reproducing. It is important to note that, although the text above is framed in terms of successful reproductions, this theme also highly encourages sharing negative results (e.g., a researcher fails to reproduce or replicate another study’s findings). In such cases, papers must interpret and discuss the results and analysis with care as absence of evidence is not evidence of absence (i.e., just because a study fails to replicate a prior experiment does not necessarily mean the original experiment was flawed).
Other types of submissions to the Theory and Methods theme may be derived from original or surveyed empirical research, analysis of existing research and methods, or may be purely theoretical or philosophical. Successful papers will clearly detail how they extend our current fundamental understanding of human-robot interaction and why the work is significant and has potential for impact. As appropriate, work must be defended by clear and sound arguments, a systematic data collection strategy, supporting data and appropriate evidence, and/or a thorough reflective analysis of the research with respect to the existing state of the art.
Theme chair: Holly Yanco
Sample papers on various aspects of Theory and Methods:
- Coyne et al. (2020). Using the Geneva Emotion Wheel to Measure Perceived Affect in Human-Robot Interaction. HRI 2020
- Tolmeijer et al. (2020). Taxonomy of Trust-Relevant Failures and Mitigation Strategies. HRI 2020.
- Carpinella, et al. (2017). The robotic social attributes scale (RoSAS): Development and validation. HRI 2017.
- Baxter, et al. (2016). From characterising three years of HRI to methodology and reporting recommendations. HRI 2016.
- Sequeira, et al. (2016). Discovering Social Interaction Strategies for Robots from Restricted-Perception Wizard-of-Oz Studies. HRI 2016.
- Fischer, et al. (2012). Levels of Embodiment: Linguistic Analyses of Factors Influencing HRI. HRI 2012.
Sample Reproducibility papers:
- Strait et al. (2020). A Three-Site Reproduction of the Joint Simon Effect with the NAO Robot. HRI 2020.
- Strait, et al. (2019). Children’s responding to humanlike agents reflects an uncanny valley. HRI 2019.
- Vogt, et al. (2019). Second Language Tutoring Using Social Robots: A Large-Scale Study. HRI 2019.
National Science Foundation and Institute of Education Sciences, U.S. Department of Education, “Companion Guidelines on Replication & Reproducibility in Education Research”. URL: https://ies.ed.gov/pdf/CompanionGuidelinesReplicationReproducibility.pdf Last updated: Nov 28, 2018.
Elizabeth Broadbent, University of Auckland, New Zealand
Dave Feil-Seifer, University of Nevada, Reno, USA
Daniel Szafir, University of Colorado Boulder, USA