Full Papers

The ACM/IEEE International Conference on Human-Robot Interaction is a premier, highly-selective venue presenting the latest advances in Human-Robot Interaction. The 18th Annual HRI conference theme is “HRI for all.” We encourage the community to consider ways to both make the field a more inclusive place for those who may not feel included, as well as to encourage inclusion within our research methods and practices. The conference seeks contributions from a broad set of perspectives, including technical, design, behavioural, theoretical, and methodological, that advance fundamental and applied research in human-robot interaction. Full papers will be archived in the ACM Digital Library.

PCS Submission Site

Important Dates

Full papers

  • October 3rd, 2022 (23:59 AoE): Full Paper Submission Deadline
  • November 8th, 2022: Reviews Due
  • November 14th, 2022: Review Notification, Rebuttal Period Begins
  • November 18th, 2022: Rebuttal Period Ends
  • December 2nd, 2022: Decision Notification
  • January 5th, 2023: Camera-ready Papers Due
  • March 13-16, 2023: Conference

HRI2023 Submission Themes

The HRI 2023 conference has five themes: Studies; Technical; Design; Systems; and Theory, Methods, and Reproducibility.

To facilitate quality interdisciplinary reviewing, and to inform reviewer selection, authors will be required to select one main theme for each submission. They may optionally also select a second theme for their full paper submissions. It is important for authors to carefully select the theme as it will have an impact on how the submission is evaluated and which reviewers are recruited. It is recognized that papers may not clearly fit within one theme. Consider the primary contribution to make the selection and be sure to select the appropriate sub-theme based on the theme descriptions below (e.g., a paper with a strong technical contribution and a user study should have “Technical” as the main theme). While authors will suggest a primary theme, the program chairs may move the paper to a different theme to improve fit.

STUDIES

The primary contribution is human-focused, e.g., how humans perceive, interact with, or otherwise engage with robots. This theme is for research contributions that provide new knowledge of human-robot interactions derived from data and analysis of humans and robots in laboratory or in-the-wild settings. Work can include data and analysis that is quantitative, qualitative, or both. It may be formative or summative in nature, can be hypothesis-driven or exploratory, and take a positivist or interpretivist approach. Studies can employ robots across the autonomy spectrum. Video-based and/or virtual user study paradigms are acceptable with appropriate/sufficient motivation and clarifications regarding any limitations that such a methodology introduces, though authors are encouraged to use in-person robots wherever possible. Successful submissions should reflect rigorous empirical methodologies and analyses that yield novel insights into human-robot interaction and should discuss the limitations and generalizability of the methods used.

Studies papers should include clear consideration of their methods’ reliability as well as internal, external, and ecological validity. For example, the measures used should be validated either in prior work or within the given paper. If the focus of the paper is methodological, the development of new measures themselves (rather than using measures to derive new insights into human-robot interactions), or work that partially or fully reproduces, replicates, or recreates a prior study (or fails to) as part of the contribution, authors should instead consider submitting to the Theory, Methods, and Reproducibility theme.

Papers that provide novel interaction techniques or designs as a primary contribution, but include a detailed user study, may belong in the Technical, Systems or Design themes.

Sample Papers:

TECHNICAL

The primary contribution is robot-focused, e.g., systems, algorithms, or computational methods supporting HRI. This theme is for research contributions that provide novel robot systems, algorithms, interface technologies, and computational methods supporting human-robot interaction. This includes contributions that enable robots to better understand, interact with, and collaborate with people, including both collocated and distal interaction. Submissions must present full details of the proposed technological advance to facilitate in-depth review and enable future reproducibility (e.g., formal descriptions, pseudocode, or open-sourced code). Successful papers will clearly demonstrate how the technology improves or enables human-robot interaction and will include evaluation appropriate to the work (e.g., comparisons to other methods, standard machine learning or computer vision metrics, human-robot interaction studies, etc.).

The Technical theme welcomes contributions that include artifacts as part of the technical contribution, such as new datasets, benchmarks, or open source software releases. This theme also welcomes contributions that present novel algorithms created through integrative efforts that combine elements of prior technologies in new ways, or existing algorithms applied to a new problem.

However, if the primary focus of the work is recreating or replicating prior technical advances, rather than integrating them into new systems, the paper should instead belong in the Theory, Methods, and Reproducibility theme. Likewise, if the primary contribution of the paper is integrating existing systems to achieve some novel functionality/behavior, it may instead belong in the Systems theme. If the primary focus of a paper is on the evaluation of an interaction or generalizable knowledge gained about human-robot interactions, rather than specific technologies supporting interactions, then it may instead belong in the Studies theme. If the primary focus is on designing new systems and interactions with an emphasis on the design process, rather than the technologies behind it, it may instead belong in the Design theme.

Sample Papers:

DESIGN

The primary contribution is design-focused, e.g., new morphologies, behavior paradigms, and interaction capabilities for robots. This theme is for design-centric research contributions to human-robot interaction. This includes the design of new robot morphologies and appearances, behavior paradigms, interaction techniques and scenarios, and interfaces. The design research should support unique or improved interaction experiences or abilities for robots. Research on the design process itself or proposing new design strategies, frameworks, or models as relevant to human-robot interaction are also welcome. Submissions must fully describe their design outcomes or process to enable detailed review and replication of the work. Further, successful papers will have evaluation appropriate to the work, for example end-user evaluation or a critical reflection on the design process or methodology.

If a paper’s primary focus is on a technical system or novel algorithm, rather than knowledge and insights gained from the design of such a system or algorithm, it may instead belong in the Technical or Systems theme. If the main contribution is an in-depth study that reflects on a broader interaction question, it may instead belong in the Studies theme. The Design theme welcomes work that includes or integrates elements which recreate and/or replicate prior designs; however, if a paper’s primary focus is recreating or replicating an existing design concept or artifact, it may belong in the Theory, Methods, and Reproducibility theme.

Sample Papers:

SYSTEMS

The primary contribution is investigating or describing how underlying techniques come together to achieve system-level HRI behavior. This can include achieving novel functionality from known techniques, known functionality from novel techniques, or another permutation of techniques and functionality. We are putting such papers into their own theme this year because, in our experience, such work is often disregarded as not a sufficient contribution.

This theme also welcomes submissions of artifacts such as software, hardware, and data sets. Submissions should contain a detailed description of the artifact introduced, proposed, or implemented, as well as information about how it is novel and different from other existing artifacts, and a link to an anonymized, live version of the artifact at time of submission for review.

The software and/or dataset submissions must be made available to reviewers at the time of submission (in an anonymized form), open-source/open-access at time of review, and will be assessed for documentation and accessibility as part of the review process. Reviewers will be asked to focus on priority on (1) significance for the community, (2) novelty, (3) ease of access & use (including documentation).

Before submission, authors submitting artifacts must have already obtained and report Institutional Review Board (IRB) clearances to release any data collected from human participants, received any relevant organizational clearances to release software/hardware, etc. In specific cases (e.g. a dataset with privacy-sensitive data), the code or dataset might not be directly downloadable. In this case, the authors must outline in their submission reasonable steps for other researchers to access the data (e.g. ask for specific ethical approval).

Note that software or datasets which cannot be shared due to intellectual property issues (including using proprietary licensing) cannot be submitted.

Sample Papers:

THEORY, METHODS, AND REPRODUCIBILITY

The primary contribution is to further the fundamental science of HRI. This includes theoretical, methodological, and reproducibility work.

Theory papers may include new ways of studying HRI, elucidating or connecting fundamental HRI principles beyond individual interfaces or projects, new theoretical concepts in HRI, literature reviews, etc. Such contributions may include detailing underlying interaction paradigms, introducing theoretical or philosophical concepts, or providing new interpretations of previously known results.

Methods papers focus on developing novel evaluation methodologies (e.g. new questionnaires), or in the analysis of existing research and methods derived from original or surveyed empirical research.

Reproducibility contributions target research that supports the science of HRI via reproducing, replicating, or re-creating prior HRI/HRI-relevant work, and artifacts for HRI research such as code and dataset sharing, to help our community foster Open Science practices, and build a strong and reliable evidence base. (Note: This refers to the entire field, not only papers published at the ACM/IEEE HRI conference.) Such papers may include:

  • Reproducibility of prior quantitative HRI work: Authors may conduct reproductions that span work across the spectrum of HRI — Studies, Technical, Methods, or Design. Here, there are two types of reproduction (as defined by (NSF 2018)): direct reproductions, where an author seeks to obtain the same results from an independently conducted study, using procedures and methods as closely matched to the original study as possible, or conceptual reproductions, where an author seeks to obtain the same results from an independently conducted study, where procedures and methods are systematically varied. Across either type of reproduction (direct or conceptual), if the work yields a completely new HRI finding, it may be submitted to this track or another depending on the author’s interpretation.

    Authors seeking to reproduce, replicate, or repeat quantitative work are encouraged to follow guidelines developed by the US National Science Foundation and Dept. of Education on how to design, conduct, and report such studies (see: NSF 2018, pages 4-5). Authors should also provide clear motivations for choosing the specific work they are reproducing.

    It is important to note that although the text above is framed in terms of successful reproductions of HRI science, this track also highly encourages sharing negative results. (e.g., a researcher fails to reproduce or replicate another study’s findings). In such cases, the expectation is that the results are analyzed and interpreted carefully, as absence of evidence is not evidence of absence.
  • Re-creation of prior HRI qualitative / design work: For qualitative or design-focused HRI work, authors may seek to explore an HRI paradigm within a new culture or context, or re-create or implement designs created by another. These papers may be framed as case studies, field reports, or updated design guidelines, and should clearly describe lessons learned and best practices.
  • Replication of artifacts for HRI science: We encourage submissions that introduce artifacts as an enabler to reproducibility, replicability, and re-creation of HRI research, and/or to support new lines of HRI research. For more details on how to appropriately report artifact submissions, please see the Systems theme.

Successful papers in this theme will clearly detail how they extend our current fundamental understanding of human-robot interaction and why the work is significant and has potential for impact. As appropriate, work must be defended by clear and sound arguments, a systematic data collection strategy, supporting data and appropriate evidence, and/or a thorough reflective analysis of the research with respect to the existing state of the art.

Sample Theory Papers:

Sample Methods Papers:

Sample Reproducibility Papers:

Additional Information for Authors

Studies with Human Participants

“As a published ACM author, you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects” (Note: ACM has instituted a new policy on research involving human participants and subjects as of August 15, 2021. Please check the above links if your studies involved human participants and subjects)

To support building a strong evidence base in HRI, and encourage future reproducibility of published work, all submissions involving studies with human participants should clearly outline their methodology regardless of the theme they are submitted to, including:

  • Ethical aspects considered and clearance obtained where appropriate (c.f., Geiskkovitch et al. 2016, Sections 5.2, 5.4)
  • Participant demographics and sampling approach, e.g. gender, ethnicity, etc. (c.f., de Graaf 2017, Section 2.3, recommendations in Schlesinger et al., 2017)
  • Data collection and analysis methods (c.f., Paepcke and Takayama 2010, Section V)
  • Study environment and context (c.f., Short et al. 2018, Section 3.5)
  • If a Wizard-of-Oz paradigm was used, a detailed description of the robot, wizard, user, etc. (c.f., Riek 2012, Table 2)
  • If a robot was used, a detailed description of the platform, its level of autonomy, capabilities, etc. (c.f., Beer et al. 2014, Figure 5)

Format and Submission

Full papers are up to eight camera-ready pages, including figures, but excluding references. Submissions longer than eight pages of content excluding references will be desk rejected and not reviewed. Accepted full papers will be published in the conference proceedings and presented in an oral session. The HRI conference is highly selective with a rigorous, two-stage review model that includes an expert program committee meeting where papers are extensively discussed. As such, all submissions are expected to be mature, polished, and detailed accounts of cutting-edge research described and presented in camera-ready style. In cases of equally qualified papers, positive consideration will be given to submissions that address this year’s theme, “HRI for all.”

All papers for the conference must be submitted in PDF format and conform to ACM Proceedings specifications. Please note that we are following the general ACM SIG format (“sigconf”, double column format), not the SIGCHI format. Templates are available at this link (US letter). In addition, ACM has partnered with Overleaf, where you can start writing using this link directly (note that this Overleaf document uses the new ACM workflow by default, which is not what HRI is using; to fix this, make sure the document uses the “sigconf” document class, rather than the “manuscript,screen,review” document class that is enabled in the Overleaf document by default).

Submission Accessibility

The PDF format can have major accessibility problems, especially for screen reader users. In order to support those among us who need accessible PDFs, HRI is working to improve the accessibility of our PDF proceedings and review process. We are asking all authors to make an effort to make their submissions more accessible at every point in the submission process.

If you are submitting a paper on accessibility or assistive technology, please refer to the SIGACCESS guidelines on writing about disability.

As you prepare your document, please follow these steps from the SIGCHI Guide to an Accessible Submission, then refer to the guide for information on how to prepare the final accessible PDF:

  1. Mark up content such as headings and lists using the correct Word template style or LaTeX markup.
  2. In figures, legends, and the text that refers to the figures, ferent shapes and patterns to provide a means other than color to visually distinguish elements.
  3. Provide a text description for all figures (see the SIGACCESS Guide to Describing Figures).
  4. Create every table as a real table, not an image, and indicate which cells are headers.
  5. Create every equation as a marked-up equation, not an image.
  6. Set the metadata of your document.

If you are a LaTeX user, please be aware that you may run into challenges with generating an accessible PDF; we ask that you do your best but understand that it may not be possible to generate a fully accessible PDF from LaTeX. You can do a rudimentary check of your PDF’s accessibility using Adobe Reader’s Read Out Loud tool and Apple’s built-in reading tool, but be aware that these are not full-featured screen readers and will not check all key accessibility features.

If you have questions or need assistance, please contact the conference accessibility chairs at accessibility2023@humanrobotinteraction.org.

Anonymization

The HRI 2023 review process is double-blind; every aspect of all submissions must be properly anonymized (see the anonymization guidelines). Any submission that contains any element (e.g., full paper document, artifact, or supplementary materials) that violates the anonymization guidelines will be desk rejected, including articles found on arXiv or an author’s website. If there are exceptional circumstances, please contact the program chairs as soon as possible.

Supplementary Materials

This year, authors have the opportunity to upload up to three supplemental files in conjunction with their full paper submission. These materials may be submitted via the “Supporting File 1,” “Supporting File 2,” and “Supporting File 3” sections within the full papers submission form in the precision conference submission system. While authors are encouraged to upload all supplementary materials directly, it may be infeasible to upload certain items directly (e.g., large data sets or code repositories). In this case, authors may upload a document with a link to where these anonymized supplemental materials are hosted.

Supplementary materials are not required for a submission. If authors do choose to submit supplementary materials, such materials may not be used to get around the page limit for full papers. It is important that any supplemental materials that are uploaded are also properly anonymized. Any submission that contains any element (full paper or supplementary materials) that violates the anonymization guidelines will be desk rejected.

In general, there are three main types of supplemental materials that may be submitted: videos, appendices, and artifacts (e.g., software, hardware, data sets, etc.):

Video Guidelines

Authors may submit a 1-minute video (up to 100 MB) as a supplement to their full paper. Videos are not mandatory but may be helpful to visibly showcase a working system, experimental conditions, environment context, results, etc. Only MPG, MPEG or MP4 video formats can be used. Ensure that videos are properly anonymized prior to submission.

Artifact Guidelines

Across all themes, we encourage submissions that introduce a novel “artifact” as an enabler to reproducibility, replicability, and recreation of HRI research, and/or to support new lines of HRI research. An artifact could be software, hardware, data sets, protocols, new evaluation measures, etc. Submissions should contain a detailed description of the artifact introduced, proposed, or implemented, as well as information about how it is novel and different from other existing artifacts, and, if possible, a link to an anonymized, live version of the artifact at time of submission for review.

Authors submitting artifacts must provide relevant details regarding any aspects related to artifact clearance and release (e.g., obtaining Institutional Review Board (IRB) clearance for releasing data collected by human participants, organizational clearances for the release of software/hardware, etc.). Ensure that artifacts are properly anonymized prior to submission. Any submission that includes any element (full paper, artifact, or supplementary materials) that does not follow the anonymization guidelines will be desk rejected.

Examples of papers with artifacts:

Appendix Guidelines

Authors may upload an appendix directly or provide a link to supporting appendix material hosted anonymously online. Appendices are only for supplementary materials that would interrupt the flow of the text if presented in the main document. Examples may be questionnaires used as measurements, tables of supplementary data, or figures of experimental apparatus. Additional experimental analysis, additional results, and lengthy text that further clarifies aspects of the full paper submission is not appropriate for an appendix (i.e., full paper submissions must stand on their own without requiring further explanation, analysis, or results provided through appendix information). Appendices may not be used to get around the 8-page limit for full paper submissions. Any submissions that attempt to use appendices or supplementary materials in general in a manner that violates the page limits will be desk rejected.

PROGRAM CHAIRS

Maya Cakmak (University of Washington, USA)

Iolanda Leite (KTH Royal Institute of Technology, Sweden)

Contact: pc2023@humanrobotinteraction.org