Data Submission

Assesement criteria

Submissions will be assessed by reviewers on the basis of availability, completeness, utility, and relevance.

• Availability:

Successful submissions should be easily findable and accessible and include a clear licensing, and maintenance plan. Recommended licenses are visible at LINK.

• Completeness: 

Sufficient detail must be provided on how the data was collected and organized, what kind of information it contains, how it should be used ethically and responsibly, as well as how it will be made available and maintained. Where applicable established best practices on reproducibility should be cited and conformed to. For cases where questionnaires were used, individual questionnaire items and scoring should be provided. 

• Utility & Relevance: 

Data submissions should highlight how the dataset in question has the potential to influence future work or practice in the HRI domain. 

We encourage datasets submissions to outline the steps required to reproduce their data both in the collection, post-processing, and usage stages of the data creation process. This should include references code where possible.

For eligible data submissions it is recommended that you provide some form of data visualization illustrating aspects of importance or note related to the submitted dataset. 

Data Base requirements

Regardless of the type of data published all datasets should be freely and publicly available via a digital repository at the time of submission. Appropriate repositories must be:

• Supported by and recognized within the HRI community.
• Provide permanent identifies for submitted databases (e.g., accession numbers, DOIs).
• Provide metadata and other documentation to support the use and reuse of data in a given archive. 
• Ensure long term preservation of datasets and associated documentation. 
• Allow peer reviewers to access the data for the purposes of evaluation. 
• Implement or be working toward implementing the FAIR principles (see recommended databases below).
• Allow access to data without unnecessary restrictions or, when necessary restrictions are in place, provide clear criteria for access and timely mechanisms of access.

Recommended databases that meet this criteria include: [Prefered: Github, Zenodo], Dataverse, Dryad, Figshare, Mendeley Data, and Science Data Bank.

Formatting Guidelines

Given the variety of possible dataset submissions, a single format for submissions that encompasses all possible submissions is not feasible but authors should use the recommended template for HRI submissions as outlined on the conference website and visible HERE. 

For authors new to data submissions or seeking additional guidance the sections and structure below is advised but not required:

– Title: (recommended length 110 characters or fewer)

– Abstract: (maximum 170 words)

– Study Overview: This section provides an overview of the study generating the data and its potential use. It should also include a brief summaries of previous publications (anonymized where possible) utilizing the data where applicable.

– Methods: The Methods section describes the steps taken to produce the data, including the experimental design used, data acquisition, and computational processing where implemented. It may also cover questionnaire details and their deployment, citing existing works and providing items where applicable.

– Dataset: This section explains the dataset associated with the work, including storage repository, data maintenance, and a detailed overview of file formats. It includes links to supplemental documentation.

– Usage Notes: This section provides instructions for researchers using the data, addressing ethical considerations and responsible use guidelines where applicable. It highlights the data’s utility and relevance to the HRI community, along with a summary of more detailed documentation.

References (ACM Format)

In specific cases (e.g. dataset with privacy-sensitive data), a complete dataset might not be directly downloadable. In this case, the authors must outline in their submission reasonable steps for other researchers to access any aspects of the data that cannot be made openly available (e.g. ask for specific ethical approval).

– Title: (recommended length 110 characters or fewer)

– Abstract: (maximum 170 words)

– Study Overview:This section provides an overview of the study generating the data and its potential use. It should also include a brief summaries of previous publications (anonymized where possible) utilizing the data where applicable.

– Methods: The Methods section describes the steps taken to produce the data, including the experimental design used, data acquisition, and computational processing where implemented. It may also cover questionnaire details and their deployment, citing existing works and providing items where applicable.

– Dataset: This section explains the dataset associated with the work, including storage repository, data maintenance, and a detailed overview of file formats. It includes links to supplemental documentation.

– Usage Notes: This section provides instructions for researchers using the data, addressing ethical considerations and responsible use guidelines where applicable. It highlights the data’s utility and relevance to the HRI community, along with a summary of more detailed documentation.

References (ACM Format)

In specific cases (e.g. dataset with privacy-sensitive data), a complete dataset might not be directly downloadable. In this case, the authors must outline in their submission reasonable steps for other researchers to access any aspects of the data that cannot be made openly available (e.g. ask for specific ethical approval).

Note: Dataset which cannot be shared due to intellectual property issues (including using proprietary licensing) cannot be submitted.  

Relation to existing HRI themes:

Data submissions can be submitted to the HRI Technical Advances track already. However,this track has a narrower focus, offering an avenue to present and publish datasets that would be useful/important to the community as standalone contributions.

**NOTE:** For datasets containing human-subjects data authors should specify how subjects were selected, what consent processes were followed, how data sharing was communicated, and specification of any additional ethical considerations. It is also important to clearly indicate ethical board (e.g. IRB) approval for the data collected and approval for data sharing or, where no board was involved, clearly to state this fact. Furthermore anonymity of subjects should be ensured with any sensitive or identifying information removed or embargoed pending regulatory approval. In such cases the presence of this information should be reflected but the actual information/data/transcript segments themselves should be blacked out or removed.

We strongly recommend that you ask colleagues that are not familiar with the code or software to test it prior to submission. 

Code Compatibility

Compatibility with at least one common and accessible hardware-software platform is required. The platform must be in common use by the HRI community (e.g.,Software platforms such as Ubuntu and ROS versions, Windows 10/11, macOS versions etc., Languages such as Python, C++, MATLAB, R etc.). If reviewers cannot run your software/code (either due to lack of clear instructions or using outdated software) then it cannot be assessed along the required criteria and may be rejected.

Software manuscripts will not be considered under the following circumstances:

• They require access to databases or other resources whose persistence is not guaranteed (e.g., individual laboratory databases without funding support).
• Running the software depends on proprietary or otherwise unobtainable ancillary software or hardware.

Formatting Guidelines

Given the variety of possible dataset submissions, a single format for submissions that encompasses all possible submissions is not feasible but authors should use the recommended template for HRI submissions as outlined on the conference website and visible HERE. 

For authors new to data submissions or seeking additional guidance the sections and structure below is advised but not required:

– Title (recommended length 110 characters or fewer)

– Abstract (maximum 170 words)

– Background & Summary: This section provides an overview of the code or software and its potential use in the HRI community. It cites previous publications that utilized the code or software where applicable and summarizes this use where applicable. If the cited work is the author’s own anonymize the work in text and in your bibliographies to ensure double blind review.

– Purpose: This section describes the key operations performed by the software or code. It explains the fundamental task(s) it accomplishes and the general approach used to address a given problem or objective.

– Characteristics: This section describes the key characteristics of the software or code, including details of algorithms used, installation and usage instructions, and software dependencies where applicable. It also comments on whether the source code has been submitted for community input and provides links to the commenting site if so.

– Code/Software: This section explains the code or software associated with the work, including the storage repository, data maintenance, and a detailed overview of files and formats. It includes links to supplemental documentation.

– Usage Notes: This section provides instructions and guidance for researchers using or modifying the software/code. It addresses ethical considerations and responsible use guidelines. It includes a summary of more detailed documentation associated with the code/software and highlights its utility and relevance to the HRI community.

References: (ACM Format)