Objective:

Astronauts and mission specialists may have to teleoperate vehicles and construction equipment, either in space or on planetary surfaces. Where bandwidth is limited, slow update rates and end-to-end signal latency can make control difficult -- especially in cooperative tasks requiring multiple operators. This research task will study feedback interface tradeoffs for specific tasks, to permit optimum design (and perhaps dynamic adaptation) of multimodal virtual environments. Auditory feedback, for instance, might be useful where haptic feedback is limited. Investigators will develop a "multisensory transducer-integrator" theory and a testbed for exploring multimodal capability and architecture concepts for multi-agent team interfaces.

Applications:

Teleoperation with limited-bandwidth feedback; EVA support; space construction; space suit design; operations monitoring and situation awareness; virtual environments.

NASA Benefit:

Astronauts perform tasks within high-stress, high-workload environments. Optimum performance may require feedback systems that are carefully tuned to make best use of available sensory channels. This task will characterize acceptable bandwidth tradeoffs for teleoperation, including update rates and latency tolerance for tasks involving multiple operators (such as assembly of structures in space). Results should enhance both human operator performance and safety.

Keywords:

multimodal interface design, human perceptual modeling, modality substitution, teleoperation

Prior Technology:

Separate visual and auditory channels, with no coordination or tradeoff; no haptic (tactile) or nonverbal auditory feedback; lack of design knowledge for latency tolerance and update rate in cooperative tasks.

FY04 Milestone:

Scenario and testbed for flexible use of multimodal capability.

FY04 Quadchart Slide:

HCC_NRA_Begault_MmdIntf.ppt.

Accomplishments:

Theory of modality substitution; auditory-haptic and auditory-visual interaction experiments; identified parameters for latency tolerance and update rate.

Related Web Pages:

Research group page.

Contacts:

Durand R. Begault (PI), Ames Research Center (Code THH).