Important Dates
Submission Deadline: October 7, 2024, 11:59pm AoEReviews Notification: November 7, 2024Rebuttals Due: November 14, 2024, 11:59pm AoEDecision Notification: December 2, 2024- Camera Ready Deadline: January 5, 2025, 11:59 pm AoE
The ACM/IEEE International Conference on Human-Robot Interaction is a premier, highly-selective venue presenting the latest advances in Human-Robot Interaction. The conference theme for HRI 2025 is “Robots for a sustainable world” and will focus on contributions that aim to bring HRI out of the lab and into everyday life. This year’s HRI conference will feature three types of short contributions (4 pages, excluding references) focused on datasets, code, and replication studies. These short contributions aim t
- Advance the sharing and reuse of scientific data and code.
- Promote wider sharing, reuse, and replication of existing scientific work.
- Provide additional credit and recognition to authors involved in the production, sharing, and reuse of datasets, code, open science, and replication.
Similar to the HRI Full papers, HRI Short Contributions will be fully and rigorously reviewed, and will be archived in the ACM Digital Library and IEEE Xplore.
Submission Categories
To facilitate quality reviewing, and to inform reviewer selection, authors will be required to select a submission category for their short contribution.
These categories are:
Review Process
Submissions should be in a double-blind format. Reviewers will assess submissions based on the criteria associated with each type of submission (accessibility, completeness, utility, and relevance).
When making your submission, please note that all authors will be required to review at least one submission as well as to provide a list of three recommended potential reviewers.
For code and data submissions, please also attach your anonymized code / data as a zip file and submit as supplementary material. The deadlines for the supplementary material is the same as the paper deadlines. Accepted short contributions papers with a video in supplementary material will be considered for Demo Fast Track. More details here.
Common Anonymization Guidelines
When anonymizing your code or data, make sure to carefully check that your licenses, documentation, readme, code / data comments etc. do not contain any identifiable information. Papers will be desk-rejected if anonymization rules are violated. Check here for more in-detail information regarding general anonymization guidelines.
Some codebases may already exist in repositories such as Github etc. For those cases, here are some helpful tools to easily anonymize your existing code or data:
- Anonymous Github (view-only link): https://anonymous.4open.science/.
- OSF (view-only link): https://help.osf.io/article/201-create-a-view-only-link-for-a-project.
- Dataverse (private URL): https://guides.dataverse.org/en/latest/user/dataset-management.html#private-url-to-review-unpublished-dataset.
- Zenodo and figshare: https://github.com/dgraziotin/disclose-data-dbr-first-then-opendata.
However, note that we will not accept any digital links of anonymized code primarily because some links are view-only. Please download your anonymized code and submit as supplementary zip files because our reviewers may need to ensure that they can run the code by following the instructions in the README file. The same applies for data submissions. If your dataset is too big to upload, please zip a representative sample for reviewers to check.
Data Submissions
Data submissions are short papers that provide access to and detailed descriptions of research data that is of value to the HRI community. These submissions may include both qualitative and quantitative data collected in HRI settings. Data that has already been published as a part of a previous publication may not be included.
During initial submission, data should be submitted as a supplementary zip file. If the dataset is too large, a representative sample should be submitted.
During final camera-ready submission, regardless of the type of data published, all datasets should be freely and publicly available via a digital repository. Appropriate repositories must be:
- Supported by and recognized within the HRI community.
- Provide permanent identifies for submitted databases (e.g., accession numbers, DOIs).
- Provide metadata and other documentation to support the use and reuse of data in a given archive.
- Ensure Implement or be working toward implementing the FAIR principles (see recommended databases below) of datasets and associated documentation.
- Allow access to data without unnecessary restrictions or, when necessary restrictions are in place, provide clear criteria for access and timely mechanisms of access.
Recommended databases that meet this criteria include: Zenodo (preferred), Dataverse, Dryad, Figshare, Mendeley Data, and Science Data Bank.
⚠️ Note ⚠️: For datasets containing human-subjects data authors should specify how subjects were selected, what consent processes were followed, how data sharing was communicated, and specification of any additional ethical considerations. It is also important to clearly indicate ethical board (e.g. IRB) approval for the data collected and approval for data sharing or, where no board was involved, clearly to state this fact. Furthermore anonymity of subjects should be ensured with any sensitive or identifying information removed or embargoed pending regulatory approval. In such cases the presence of this information should be reflected but the actual information/data/transcript segments themselves should be blacked out or removed.
Submissions will be assessed by reviewers on the basis of completeness, utility, and relevance.
- Completeness: Successful submissions should include a clear licensing, and maintenance plan. Recommended licenses are visible here. Sufficient detail must be provided on how the data was collected and organized, what kind of information it contains, how it should be used ethically and responsibly, as well as how it will be made available and maintained. Where applicable established best practices on reproducibility should be cited and conformed to. For cases where questionnaires were used, individual questionnaire items and scoring should be provided.
- Utility and relevance: Data submissions should highlight how the dataset in question has the potential to influence future work or practice in the HRI domain.
We encourage datasets submissions to outline the steps required to reproduce their data both in the collection, post-processing, and usage stages of the data creation process. This should include references code where possible.
For eligible data submissions it is recommended that you provide some form of data visualization illustrating aspects of importance or note related to the submitted dataset.
Given the variety of possible dataset submissions, a single format for submissions that encompasses all possible submissions is not feasible but authors should use the recommended template for HRI submissions as outlined on the conference website and visible here.
For authors new to data submissions or seeking additional guidance the sections and structure below is advised but not required:
- Title (recommended length 110 characters or fewer).
- Abstract (maximum 170 words).
- Study Overview: This section provides an overview of the study generating the data and its potential use. It should also include a brief summaries of previous publications utilizing the data where applicable.
- Methods: The Methods section describes the steps taken to produce the data, including the experimental design used, data acquisition, and computational processing where implemented. It may also cover questionnaire details and their deployment, citing existing works and providing items where applicable.
- Dataset: This section explains the dataset associated with the work, including data maintenance, and a detailed overview of file formats. It includes links to supplemental documentation.
- Usage Notes: This section provides instructions for researchers using the data, addressing ethical considerations and responsible use guidelines where applicable. It highlights the data’s utility and relevance to the HRI community, along with a summary of more detailed documentation.
- References (IEEE Format).
In specific cases (e.g. dataset with privacy-sensitive data), a complete dataset might not be directly downloadable. In this case, the authors must outline in their submission reasonable steps for other researchers to access any aspects of the data that cannot be made openly available (e.g. ask for specific ethical approval).
⚠️ Note ⚠️: Dataset which cannot be shared due to intellectual property issues (including using proprietary licensing) cannot be submitted.
Relation to existing HRI themes
Data submissions can be submitted to the HRI Technical Advances track already. However, this track has a narrower focus, offering an avenue to present and publish datasets that would be useful/important to the community as standalone contributions.
Code Submissions
Code submissions are short papers that provide access to and detailed descriptions of different code, tools, and software that is of value to the HRI community. These submissions may include both software, analysis code, and other digital tools of use to the HRI community. Code that has already been published as a part of a previous publication may not be included.
During initial submission, code should be submitted as a supplementary zip file.
During final camera-ready submission, regardless of the type of code published, all code submissions should be made freely and publicly available via a digital repository at the time of submission. Appropriate repositories must be:
- Supported by and recognized within the HRI community.
- Provide metadata and other documentation to support the use, reuse, and potential modification of code in a given repository.
- Ensure long term preservation of code and associated documentation.
- Allow access to the code without unnecessary restrictions or, when necessary restrictions are in place, provide clear criteria for access and timely mechanisms of access.
Recommended databases that meet this criteria include: Github (preferred), Zenodo (preferred), Dataverse, Dryad, Figshare, Mendeley Data, and Science Data Bank.
Within these repositories the following must be present:
- Compiled standalone software and/or source code.
- A clear open-source license for the code or software submitted (see here).
- A small – simulated or real – dataset to demo the software/code (where applicable).
- A readme file that contains:
- System requirements:
- All software dependencies and operating systems (including version numbers).
- Versions the software has been tested on.
- Any required non-standard hardware [including specific makes and models for robotic platforms].
- Installation guide:
- Instructions on how to install the software or run the code.
- Demo:
- Instructions for the demo or demo video of the code.
- Expected output (where appropriate).
- Instructions for use:
- How to run the software or code.
- System requirements:
We strongly recommend that you ask colleagues that are not familiar with the code or software to test it prior to submission.
Compatibility with at least one common and accessible hardware-software platform is required. The platform must be in common use by the HRI community (e.g.,Software platforms such as Ubuntu and ROS versions, Windows 10/11, macOS versions, etc., Languages such as Python, C++, MATLAB, R etc.). If reviewers cannot run your software/code (either due to lack of clear instructions or using outdated software) then it cannot be assessed along the required criteria and may be rejected.
Software manuscripts will not be considered under the following circumstances:
- They require access to databases or other resources whose persistence is not guaranteed (e.g., individual laboratory databases without funding support).
- Running the software depends on proprietary or otherwise unobtainable ancillary software or hardware.
Submissions will be assessed by reviewers on the basis of completeness, utility, and relevance.
- Completeness: Successful submissions should include a clear licensing, and maintenance plan. Recommended licenses are visible here. Sufficient detail must be provided on how the submitted code can be used both ethically and responsibly to further HRI research. Furthermore, submissions should aso highlight how the submitted code will be made available and maintained. Where applicable established best practices regarding documentation and origination of the code and repositories should be cited and conformed to.
- Utility and relevance: Code submissions should highlight how the code in question has the potential to influence future work or practice in the HRI domain.
Given the variety of possible code submissions, a single format for submissions that encompasses all possible submissions is not feasible but authors should use the recommended template for HRI submissions as outlined on the conference website and visible here.
For authors new to code submissions or seeking additional guidance the sections and structure below is advised but not required:
- Title (recommended length 110 characters or fewer).
- Abstract (maximum 170 words).
- Background & Summary: This section provides an overview of the code or software and its potential use in the HRI community. It cites previous publications that utilized the code or software where applicable and summarizes this use where applicable.
- Purpose: This section describes the key operations performed by the software or code. It explains the fundamental task(s) it accomplishes and the general approach used to address a given problem or objective.
- Characteristics: This section describes the key characteristics of the software or code, including details of algorithms used, installation and usage instructions, and software dependencies where applicable. It also comments on whether the source code has been submitted for community input and provides links to the commenting site if so.
- Code/Software: This section explains the code or software associated with the work, including the code maintenance, and a detailed overview of files and formats. It includes links to supplemental documentation.
- Usage Notes: This section provides instructions and guidance for researchers using or modifying the software/code. It addresses ethical considerations and responsible use guidelines. It includes a summary of more detailed documentation associated with the code/software and highlights its utility and relevance to the HRI community.
- References (IEEE Format).
Replication Studies
Replication studies report on reproducing previous published results, positive or negative. The authors should provide a clear and detailed description of the methods for another researcher to closely replicate the experimental procedures and results. The authors should use the HRI paper template but adapt the sections to clearly outline the following:
- What is the scope of reproducibility of the paper?
- What were the methods?
- What were the results?
- Were you successful in reproducing the results? If not, what could be the plausible reasons?
- What was easy to replicate?
- What was difficult / unclear where the authors may have made some assumptions?
- Was there any attempt to contact the authors of the original submission that is being replicated?
Reviewers of these submissions will be asked to focus on the level of clarity and details on the above mentioned criteria and prioritize scientific rigor and importance of the replicated study/results to HRI as a field.
Relation to Full paper HRI themes
Replication studies which significantly expand onto the original research (e.g., add a new condition) and warrant a full-length paper should instead be submitted as a regular full paper.
Sample Reproducibility Papers
- Strait et al., A Three-Site Reproduction of the Joint Simon Effect with the NAO Robot, HRI 2020.
- Li et al., On-Road and Online Studies to Investigate Beliefs and Behaviors of Netherlands, US and Mexico Pedestrians Encountering Hidden-Driver Vehicles, HRI 2020.
- Pereira et al., Effects of Different Interaction Contexts when Evaluating Gaze Models in HRI, HRI 2020.
Contacts
Short Contributions Chairs: Anara Sandygulova (Nazarbayev University), Tapomayukh Bhattacharjee (Cornell University).