Live Document

5: Problem Domains in HRI

We have already mentioned several of the problem domains and application areas in modern HRI. In this section, we elaborate on many of these problem domains to present a survey of the kinds of problems encountered in HRI. Importantly, many of these problems have broad social impact; thus, much work in HRI uses science and engineering to respond to needs in society.

Scholtz provided a taxonomy of roles that robots can assume in HRI [164]:

  • Supervisor,
  • Operator,
  • Mechanic,
  • Peer, and
  • Bystander.

To this list, we add the following

  • Mentor: the robot is in a teaching or leadership role for the human, and
  • Information Consumer: the human does not control the robot, but the human uses information coming from the robot in, for example, a reconnaissance task.

Julie A. Adams: abstract supervisor

Similar taxonomies are certainly possible, but identifying how people perceive a robot’s role has important ramifications for how they interact with the robot [228]. Using Scholtz’s taxonomy provides insight into the current and future interactions in these applications. Table 1 classifies the most frequent types of interactions for the application areas discussed in the remainder of this section. For many of these areas, current research patterns exhibit a trend away from remote interactions towards proximate interactions, and away from operator roles towards peer or mentor roles.

Table 1. Examples of roles and proximity patterns that arise in several application areas.

A. Search and Rescue
The highest profile HRI research area in the United States is a strong example of work with important social impact. This area is urban search and rescue (USAR), and is exemplified by the use of robots in rescue and recovery efforts after the collapse of the World Trade Center buildings [229]. Lead by pioneering government and academic efforts [230, 231], USAR has grown into one of the most important areas of HRI.

Because of its importance, USAR has become an HRI challenge problem. As such, there have been efforts to provide standardized USAR test areas and performance measures, and to standardize robot-assisted USAR efforts [53, 232, 233]. There are regular USAR competitions at robot-related conferences [53-55]. A wide variety of interface concepts, autonomy designs, sensor-processing algorithms, robot morphologies, field studies, and human factors analyses and experiments have been created in the name of robot-assisted USAR (see, for example, [144, 145, 192, 234, 235]). Recently, these efforts are being extended from ground robots to include aerial robots used in natural disaster and wilderness search [236, 237].

B. Assistive and Educational Robotics
In the spirit of socially relevant research, robots are being developed to serve in assistive and educational capacities. Assistive robotics is perhaps one of the highest profile areas of HRI in the world. This application domain often places the robot in a peer-like or mentoring role with the human in practice, even though the intention of the robot is designed to provide service to the human. For example, robots being developed to assist the visually challenged work in close proximity to humans and must merit an appropriate level of trust. The goal of work in this area is to increase the set of tasks that a blind person can independently perform. These tasks include providing navigation assistance in unstructured domains [238-240], and providing information about locations (and prices) in grocery stores and transportation facilities [241].

Another important area of research, especially in countries with a burgeoning population of the elderly, is providing support for those who have age-related challenges [242-246]. Researchers in this area focus on both physical needs such as mobility assistance [247] and emotional [242, 248] and cognitive needs [249]. Many working in this area are fully aware of the ethical considerations that arise by delegating a companionship role to a robot and the trust issues that arise by having an artificial agent working with someone with mental challenges such a senility [250]. Having the ability to remotely manage a personal assistant robot over the internet may be a step in the direction of addressing these ethical concerns, though substantial technological limitations exist for such interaction [251].

For some people with physical and mental challenges, robots may provide an opportunity for interaction and therapy. Such work is being explored with autistic children [30, 113]. Many of these children respond weakly or not at all to social cues, but well to mechanical devices. Robots provide a possible therapeutic role for using a mechanical device to improve social interactions [183]. Robots are also being considered for other domains where children are benefited, such as those who have experienced trauma. Importantly, the social dimension of HRI is considered necessary not only in assistive roles but also in many areas of proximate interaction [29, 58, 168, 176, 252, 253]. Indeed, the area of social robotics is so large that it is already time to update the highly cited 2003 survey [142].

Recently, researchers have focused on several aspects within robot-assisted autism therapy, for example, appearance of the robot [334, 335, 336], ethical considerations, and enabling clinicians to program robots. Ethical considerations involve whether robot-assisted therapy provides long-term benefits and does not cause emotional or cognitive harm [337, 338]. The impetus for enabling clinicians to program robots stems from clinicians wanting to avoid preservation of sameness, that is, repetitive robot behavior on which children with autism tend to fixate. By enabling clinicians to customize robot behavior, preservation of sameness can be avoided, in addition to allowing clinicans to tailor the robot behavior to fulfill therapeutic needs for each child. Approaches to enabling clinicians to program include visual programming [339] and programming by demonstration [340, 341]. [J. Alan Atherton; 10 May 2012; BYU].

For some people with physical challenges, the embodiment of a robot provides unique opportunities not available in other forms of technology. For example, researchers are working on designing robots that provide support for physical therapy. Efforts include providing prescribed force and movement trajectories to help rebuild flexibility and strength [254]. Other work includes detecting motivational state and adjusting therapy to maximize benefits [255]. Intelligent wheelchairs are a type of robot that uses external sensors to support path planning, collision avoidance, etc. for a person that requires a wheelchair [247, 256].

Physical interaction with robots is not limited to providing assistance to those with some form of disability. The cobot architecture is designed to assist humans in industrial settings [257]. Indeed, developing telerobots for use in hazardous materials handling has been ongoing for several decades [258]. However, cooperative systems are not limited to large-scale industrial settings, but may be most useful in small-scale tasks “such as microassembly and microsurgery” [259]. The technologies developed for such assistive support appear to apply not only to proximate physical interaction, but also to the general telemanipulation problem [260].

In the spirit of providing assistance to those without a disability, some researchers are also exploring how robots can be used to promote education for typical children, both in the home and in schools [261, 262]. The personal service robot is an extension to the theme of providing service to the general population. Indeed, South Korea has ambitious goals about the advancement of such robots [263].

C. Entertainment
Although there have been many examples of entertainment robotics, from an HRI perspective, not as much has been published in the literature. Early entertainment robotics centered on animatronics, where the robot generally plays prerecorded sounds that are synchronized with the robots motion. These types of robots can often be found in old movies and theme parks; however, the interaction is mostly in one direction, that of the robot presenting information, although the robot’s performance may be triggered by the presence of the human. However, the 2005 AICHI Expo demonstrated several robots designed to entertain, including of the use of robots as actors and dance partners [50]; similar work on the relationship between acting, drama, and artificial agents is presented in [264]. Here again, the role of the human is as an observer, and the interaction is minimal and more implicit [264].

The Valerie and Tank robots at Carnegie Mellon University were designed as robot receptionists as a joint project between CMU’s Robotics Institute and the School of Drama [253]. The robots have a rich back story and strive to increase users’ interaction by encouraging the user to ask about the robots’ lives outside of their role as receptionists. Although robots that act as tour guides might be considered a form of assistive robotics, we classify such robots under the category of entertainment because their primary role is to engage participants [130, 169].

Although not strictly entertainment, the Insect Telepresence project [265] placed Madagascar Hissing Cockroaches in a terrarium, and allowed users, through mediated telepresence, to drive a miniature camera within the terrarium, at eye level to the cockroaches. The image from the camera was projected much larger than life on a wall. Of interest was the mediated teleoperation, which limited the accelerations of the camera motions to below that which would cause the cockroaches to react to the camera, offering the humans a look into the cockroach’s world.

Other HRI-related research in the use of robots for entertainment include robotic story tellers [266], Robotic dance partners [267], robotic plants that give users information such as incoming email [268], and robotic pets [119, 142].

D. Military and Police
Many have called for robots to be used in tasks that are “dull, dirty, or dangerous.” Military and police applications can satisfy all three criteria for robot use. Applications include gathering information to support a dangerous task such as a SWAT team take-down, or using remote vehicles in combat to minimize risk exposure to soldiers. Current work emphasizes robots as servants to soldiers and officers [51], typically in remote operations, but efforts to have robots work in a peer-like role are underway [269, 270].

A typical use of robots in both military and police applications is in bomb disposal (called “improvised explosive devices” in many military situations) [233]. Remotely controlled robots are frequently used to approach and evaluate suspicious packages [271]. Controlling these robots is demanding on the operators, especially since many current interfaces and autonomy levels require operators to integrate information from multiple sources of data such as multiple cameras. In addition to controlling mobile platforms, telemanipulation is an important part of such work. Robot arms are often mounted on the platforms providing operators some ability to manipulate the object. Because of the limited situation awareness for controlling these arms, some researchers are exploring sensor technologies and information presentation techniques that improve awareness [272].

E. Space Exploration
Robots have long been part of space exploration. In fact, according to some definitions, a satellite can be considered a type of robot, albeit one that has a high degree of autonomy and typically requires minimal intervention.

As has been previously noted, remarkable success in space robotics include the Soviet Lunokhods which were used to explore the surface of the moon [12] followed by more recent NASA success in exploring the surface of Mars [24, 25]. It is anticipated the robots will continue to have a strong role in envisioned and future explorations of the lunar and Martian surfaces, in construction tasks on these surfaces as well as in the international space station, and in remote science and maintenance tasks[62, 273-275].

Many precursor and early human missions will depend heavily on remotely managed robots, but will also likely include extravehicular activities. To prepare for these missions, NASA and other international space agencies engage is frequent field work designed to evaluate both the robotic and HRI technologies. Examples of this include development of the Dante robots [26, 276], a series of Extra-Vehicular Activities (EVA)-based field tests [277-279], and field tests involving substantial communications delays and remote science in harsh environments [280, 281]. Research that extends these efforts includes the aforementioned Robonaut development and work on developing intelligent rovers and robot “mules” to pack astronaut equipment on long duration EVAs.

Complementing the focus on remote and EVA-based interaction are efforts to support astronauts at the international space station and on long duration space missions. These efforts focus on astronaut assistants and small satellites [282, 283]. Many of the NASA programs include strong planning components and seek to integrate software, robotic, and human agents [68]. Other work calls for standardization of robot parts, procedures, communications, and interfaces [284, 285].

F. UAV Reconnaissance and UUV Applications
Unmanned Air Vehicles are rapidly attracting attention as an application area for HRI and for aviation technology. UAVs have been around for a long time and have a large body of related literature, but the label for such vehicles varies over time. Among other labels, UAV work has been published under names like remotely piloted vehicles, uninhabited air vehicles, autonomous micro air vehicles and autonomous aerial systems. The key defining aspect of UAV interaction is the fact that the vehicles are, by definition, remotely operated, move in three dimensions, and typically have six degrees of freedom.
Interestingly, remote interactions and a high degree of freedom are also defining attributes for problems that use Underwater Unmanned Vehicles (UUVs), often refereed to as autonomous underwater vehicles. Because of this similarity, we unite our discussion of these two important areas of HRI. This unification may be particularly relevant because the literature from these two communities rarely overlaps even though the interaction dynamics have marked similarities.

In addition to the previously mentioned reconnaissance-based military and search-and-rescue uses of UAVs, there are a number of other envisioned uses of these vehicles. These include atmospheric science [286], landing site surveillance of a Mars rover [287], border patrol [288], pollution monitoring [289], forest fire monitoring [290], infrastructure inspection [289], and munitions-based military applications [291]. UUVs have similarly broad areas of use. They include undersea science, treasure hunting, undersea and surf-zone de-mining, and underwater construction and maintenance [292].

For remote operations, there are many interface designs that apply to both UAV and UUV operations. These include integrating multiple perspectives, developing a sense of telepresence, synchronizing frames of reference through tethers, and building mosaics [293, 294].

G. Other Applications
There are a host of other application domains where robots are used. These domains include home use, manufacturing, inventory management, mining, and precision agriculture. Importantly, some of these application domains include strong HRI research, but others attempt the creation of “fully autonomous” robots that do not require human interaction and, ironically, are not “autonomous enough” to allow interaction with humans. Although it is possible that some of these domains might allow “fire-up and forget” robots, it is likely that the usefulness and safety of robots in many of the domains will increase if HRI considerations are included in their design.