Objective:

Many NASA missions involve extreme workloads and environmental conditions. Conventional input/output devices such as keyboards, joysticks, and video monitors are difficult to use in the challenging environments of space exploration and global air transport. Spoken dialogue systems will help humans communicate with machines much as they would talk with other humans. They offer unique advantages in situations where hands are impeded or needed for other tasks, eyes are busy attending to the environment, or tasks are sufficiently complex that interface ease of use becomes critical. This research task will develop robust, contextual speech recognition within the RIALIST dialogue system. Challenges include contextual interpretation; asynchronous dialogue management; and natural dialogue designs. Investigators will compile speech recognition grammars; develop new algorithms; accommodate foreign accented English; and develop an open microphone speech recognition system. The driving application will be a spoken dialogue system to assist astronauts in performing procedures on the International Space Station (ISS). Potential applications include hands-free information capture; high-level control of autonomous agents and systems; collaborative dialogues; and just-in-time tutoring.

Applications:

Speech commanding of autonomous systems; conversational astronaut assistants; interaction with ISS systems, robots, or rovers; checklist navigation; just-in-time tutoring; hands-free data entry and note-taking; science data management; mission operations; automated transcription.

NASA Benefit:

Spoken dialogue systems can improve efficiency, robustness, and safety in NASA missions -- and may offer new capabilities -- in mission operations, communication with rovers and other semi-autonomous agents, and science data management. Applications include collaboration with semi-autonomous agents during extravehicular activities (EVAs); enabling computer interaction in challenging environments or when hands are impeded by space suits; capturing information from conversations between astronauts or pilots and operations personnel; and completing complex, high workload tasks where one voice command can accomplish the equivalent of many keystrokes or mouse moves.

Keywords:

spoken-language interface, natural-language dialogue system, voice commanding, speech recognition

Prior Technology:

Keyboard input; limited-vocabulary speech recognition; poor noise tolerance; difficulty with accents; rigid procedure sequences; two astronauts required for checklist procedures.

FY04 Milestone:

NRA final report; continuing development under directed funding.

FY04 Quadchart Slide:

HCC_NRA_Hieronymus_DlgIntf.ppt.

Accomplishments:

Requirements analysis for NASA natural language and speech interface research; core dialogue management system; demonstration systems; incorporated feedback from astronauts.

Related Web Pages:

RIALIST group page.

Contacts:

Beth A. Hockey (PI), USRA RIACS.
Frankie James, USRA RIACS.
John T. Dowding, USRA RIACS.
James L. Hieronymus (Co-I), USRA RIACS.