FULL PAPERS

The ACM/IEEE International Conference on Human-Robot Interaction is a premier, highly-selective venue presenting the latest advances in Human-Robot Interaction. The 15th Annual HRI conference theme is “Real World Human-Robot Interaction”. The conference seeks contributions from a broad set of perspectives, including technical, design, behavioural, theoretical, methodological, and metrological, that advance fundamental and applied knowledge and methods in human-robot interaction. Full papers will be archived in the ACM Digital Library and IEEE Xplore Digital Library.

Important Dates

1 October 2019 (23:59 PDT): Submission Deadline. (Link will be posted mid-summer)
4 November 2019: Review Notification, rebuttal period begins.
11 November 2019: Rebuttal period ends
29 November 2019: Decision notification
8 January 2020: Camera ready papers due
23-26 March 2020: Conference

Format and Submission

Full papers are up to eight camera-ready pages, including figures, but excluding references. Submissions longer than eight pages of content excluding references will be desk rejected and not reviewed. Accepted full papers will be published in the conference proceedings and presented in an oral session. The HRI conference is highly selective with a rigorous, two-stage review model that includes an in-person expert program committee meeting where papers are extensively discussed. As such, all submissions are expected to be mature, polished, and detailed accounts of cutting-edge research described and presented in camera-ready style. In cases of equally qualified papers, positive consideration will be given to submissions that address this year’s theme, “Real-World Human Robot Interaction”.

*UPDATED* This year we are following a new template workflow process to improve accessibility and automation. All authors must follow this new process (using the new Word or LaTeX template), and generate a PDF for submission: all papers must be in PDF format. Please note that this means we are not using the traditional SIGCHI or general ACM SIG templates, and that authors using Word will submit in single-column format.

The HRI 2020 review process is double blind; submissions must be properly anonymized (see the anonymization guidelines). Supplementary materials (such as videos) can be uploaded, or external links to videos and supplementary material are also allowed in your paper. All submissions must use only “Type 1” (scalable) fonts (not bitmapped fonts), an ACM digital library requirement.

To facilitate quality interdisciplinary reviewing, and to inform reviewer selection, authors will be required to select one main theme and one optional second theme for their full paper submissions. It is important for authors to carefully select the theme as it will have an impact on how the submission is judged, and which reviewers are recruited. It is recognized that papers may not clearly fit within one theme. Consider the primary contribution to make the selection, and be sure to select the appropriate sub-theme. While authors will suggest a primary theme; the program chairs may move the paper to a different theme to improve fit.

Authors are encouraged to consult the guide regarding submissions to HRI provided at: http://humanrobotinteraction.org/2020/guide-to-submission-types/

SELECTING A THEME

The HRI 2020 conference has five themes: User Studies, Technical Advances, Design, Theory and Methods, and Reproducibility (See details below). Papers may have overlap between themes, but authors are encouraged to consider the main contribution of the work using this brief rule of thumb:

  • Human Robot Interaction User Studies: The primary contribution is human-focused, e.g.,  how humans perceive, interact with, or otherwise engage with robots.

  • Technical Advances in Human Robot Interaction: The primary contribution is robot-focused, e.g., systems, algorithms, or computational methods supporting HRI.

  • Human-Robot Interaction Design: The primary contribution is design-focused, e.g., new morphologies, behavior paradigms, and interaction capabilities for robots.

  • Theory and Methods in Human-Robot Interaction: The primary contribution is methodology-focused, e.g., fundamental HRI principles beyond individual interfaces or projects, new theoretical concepts in HRI, etc.

  • Reproducibility of Human-Robot Interaction: The primary contribution is science-focused, e.g., Reproduces, Replicates, or Re-creates prior HRI work (or fails to), provides new HRI artifacts (e.g., datasets, software), etc.

Across all themes, if a paper includes a study with human participants:
To support building a strong evidence base in HRI, and encourage future reproducibility, all submissions involving studies with human participants must clearly outline:

  1. The methodology used, including: ethical aspects considered and clearance obtained, participant demographics and sampling approach, data collection and analysis methods, environment and context, etc.
  2. The technology used, for example, studies involving robots should describe in detail the platform, its software, its level of autonomy, etc.
  3. If a Wizard is used to control any functions of a robot during a study, authors should report wizard parameters in detail (See Riek 2012, JHRI, pg 130).

1. Human Robot Interaction User Studies

This theme targets research that provides data on and analysis of human-robot interaction, in laboratory or in-the-wild settings. Work can be quantitative, qualitative, or both. It may be formative or summative in nature, and can be hypothesis-driven or exploratory. Studies can employ robots across the autonomy spectrum. Video-based user study paradigms are acceptable with appropriate/sufficient motivation, though authors are encouraged to use in-person robots wherever possible. Successful submissions should reflect rigorous methodologies (quantitative or qualitative)  and mature analyses that yield novel insights into human-robot interaction, and should discuss the limitations and generalizability of the methods used.

Quantitative Studies papers should include clear consideration of their methods’ internal, external, and ecological validity. For example, measures used should be validated either in prior work or within the given paper.  For development of new measures, authors should instead consider submitting to the Theory and Methods theme (See details below).

Papers that provide novel interaction techniques or designs as a primary contribution, but include a detailed user study, may belong in the Design theme. Work that is primarily on methodological advancements or analysis may belong in the Theory and Methods theme. Work that reproduces, replicates, or re-creates a prior study (or fails to) as its main contribution may belong in the HRI Reproducibility theme.

Theme chairs: Ginevra Castellano and Maha Salem

2. Technical Advances in Human-Robot Interaction

This theme targets research providing novel robot system designs, algorithms, interface technologies, and computational methods supporting human-robot interaction. This includes contributions that enable robots to better understand, interact with, and collaborate with people, including co-located interaction or teleoperation. Submissions must present full details of the proposed technological advance to facilitate in-depth review and enable the future reproducibility, e.g., formal descriptions, pseudocode, or open-sourced code. Successful papers will clearly demonstrate how the technology improves or enables human-robot interaction, and will include evaluation appropriate to the work (e.g., comparisons to other methods, standard machine learning or computer vision metrics, usability studies, etc.).

If the primary focus of the paper is on the evaluation of interaction, and not the specific technology, then it may belong in the HRI User Studies theme. If the primary focus is on a novel interaction design – and not new technologies behind it – it may belong in the Design theme. If the primary focus is on new artifacts for HRI science, e.g, datasets, benchmarks, or open source software releases – it may belong in the HRI Reproducibility theme.

Theme chair: Maya Cakmak

3. Human-Robot Interaction Design

This theme targets research that makes a design-centric contribution to human-robot interaction. This includes the design of new robot morphologies and appearances, behavior paradigms, interaction techniques and scenarios, and telepresence interfaces. The design research should support unique or improved interaction experiences or abilities for robots. Research on the design process itself is welcome. Submissions must fully describe their design outcomes or process to enable detailed review and replication of the work. Further, successful papers will have evaluation appropriate to the work, for example end-user evaluation or a critical reflection on the design process or methodology.

If the paper’s primary focus is on a technical system description or novel algorithms it may be a Technical Advances paper. If the main contribution is an in-depth study that reflects on a broader interaction question it may be a User Studies paper. If a paper’s primary contribution is to re-create or replicate an existing design concept or artifact, it may belong in the HRI Reproducibility theme.

Theme chair: Jodi Forlizzi

4. Theory and Methods in Human-Robot Interaction

This theme targets research contributing to the understanding and study of fundamental HRI principles that span beyond individual interfaces or projects. This includes detailing underlying interaction paradigms, theoretical concepts, new interpretations of known results, or new evaluation methodologies. Submissions may be derived from original or surveyed empirical research, analysis of existing research and methods, or may also be purely theoretical or philosophical. Successful papers will clearly detail how they extend our current fundamental understanding of human-robot interaction and why the work is significant and has potential for impact. As appropriate, work must be defended by clear and sound arguments, a systematic data collection strategy, supporting data, and/or a thorough reflective analysis of the research with respect to the existing state of the art.

Theme chair: Kerstin Fischer

5. Reproducibility in Human-Robot Interaction

This theme targets research that makes a contribution supporting the science of HRI via reproducing, replicating or re-creating prior HRI/HRI-relevant work, and artifacts for HRI research, to help our community build a strong and reliable evidence base. (Note: This refers to the entire field, not only papers published at the ACM/IEEE HRI conference.)   To incentivize submission to this new track, this year accepted papers will receive ACM badges on their papers upon publication (see [ACM 2016]).

5.1.  Reproducibility of prior quantitative HRI work: Authors may conduct reproductions that span quantitative work across the spectrum of HRI – Studies, Technical, Methods, or Design. (e.g.,  the original findings were obtained through primarily quantitative methodologies.) Here, there are two types of reproduction (as defined by (NSF 2018)):

  1. Direct Reproductions, where an author seeks to obtain the same results from an independently conducted study, using procedures and methods as closely matched to the original study as possible. For example, Study S shows result X with methodology M and robot R. The reproduction, Study SR, uses M and R to confirm (or not confirm) X. SR is conducted independently of S (e.g., with a different team or independent participant population)].

    The goal of a direct reproduction is to evaluate the reliability of a previously observed HRI finding.

  2. Conceptual Reproductions, where an author seeks to obtain the same results from an independently conducted study, where procedures and methods are systematically varied. For example: Study S shows result X with methodology M and robot R. The reproduction, Study SR, systematically varies M1 and/or R1 to confirm (or not confirm) X. SR is conducted independently of S (e.g., with a different team or independent participant population)].

    The goal of a conceptual reproduction is to build upon prior evidence to understand under what conditions, and for whom, an HRI finding holds true. In a conceptual reproduction, the research questions will help the author determine which aspects of the prior study are systematically varied (NSF, 2018).
    Here are a few examples of conceptual replications per theme area:

    1. Studies / Design Conceptual Reproduction Example: If an author’s goal is to see whether behavior previously observed with robot R similarly manifests with other robots, they might vary the platforms but employ the same method. If they are also curious about how the methods used in the original study affected the results, they may vary the methods used in the original study

    2. Technical Conceptual Reproduction Example: If an author’s goal is to see if a teaming algorithm presented in a prior paper yields the same results on experiments conducted on other robot platforms, they would vary the robot platform but employ the same method

    3. Theory and Methods Conceptual Reproduction Example: If an author’s goal is to see whether a theory or method presented in prior work as being suitable for culture C also holds true in cultures C1 and C2, they would vary the cultural context but employ the same method and/or robot.

Across either type of reproduction (direct or conceptual), if the work yields a completely new HRI finding, if may be submitted to this track or another depending on the author’s interpretation.

Authors seeking to reproduce, replicate, or repeat quantitative work are encouraged to follow guidelines developed by the US National Science Foundation and Dept. of Education on how to design, conduct, and report such studies (See: [NSF 2018], pages 4-5). Authors should also  provide clear motivations for choosing the specific work they are reproducing.
It is important to note that although the text above is framed in terms of successful reproductions of HRI science, this track also highly encourages sharing negative results. (e.g., a researcher fails to reproduce or replicate another study’s findings). In such cases, the expectation is that the results are analysed and interpreted carefully, as absence of evidence is not evidence of absence.

5.2.  Re-creation of prior HRI qualitative / design work: For qualitative or design-focused HRI work, authors may seek to explore an HRI paradigm within a new culture or context, or re-create or implement designs created by another. These papers may be framed as case studies, field reports, or updated design guidelines, and should clearly describe lessons learned and best practices.

5.3.  Artifacts for HRI science: We encourage submissions that introduce a novel “artifact” as an enabler to reproducibility, replicability, and re-creation of HRI research. An artifact could be software, hardware, data sets, protocols, evaluation measures, etc. Submissions should contain a detailed description of the artifact introduced, proposed, or implemented, as well as information about how it is novel and different from other existing artifacts, and a link to an anonymized, live version of the artifact at time of submission for review.

Before submission, authors submitting artifacts must have obtained and report Institutional Review Board (IRB) clearances to release any data collected from human participants, received any relevant organizational clearances to release software/hardware, etc.  

Theme chair: Megan Strait

References

Association for Computing Machinery. “Artifact Review and Badging”. URL:  https://www.acm.org/publications/policies/artifact-review-badging. Last updated: Apr 2018

National Science Foundation and Institute of Education Sciences, U.S. Department of Education, “Companion Guidelines on Replication & Reproducibility in Education Research”. URL: https://ies.ed.gov/pdf/CompanionGuidelinesReplicationReproducibility.pdf Last updated: Nov 28, 2018.

Riek, L.D. (2012) “Wizard of Oz Studies in HRI: A Systematic Review and New Reporting Guidelines.” Journal of Human Robot Interaction. 1:(1), p. 119-136. URL: https://dl.acm.org/citation.cfm?id=3109687