printer friendly version of this page

Talking to Computers: from Speech Sounds to Human Computer Interaction

Tutorial at INTERSPEECH 2007, Antwerp, Belgium

Building dialogue systems is a complex task that requires one to address both theoretical issues and a multitude of practical ones. This tutorial is targeted at students and experienced researchers who want to understand the building blocks of spoken/multimodal dialogue systems (speech recognition, Natural Language Understanding, dialogue management, task execution, response generation and speech synthesis) and start their own dialogue systems research.

We will provide an introduction to dialogue systems by addressing general design issues (application scenarios, HCI, devices and multi-modality) and outlining the available choices for the different modules of typical dialogue systems. We will systematically describe both commercially deployed systems and research prototypes along various dimensions, including architectural choices, motivating theoretical ideas, application domains, use of statistical versus rule-based methods, training material used in corpusbased components and evaluation methods and results. This will include aspects that are not frequently emphasized, for example the response generation component. We will provide information about available software toolkits that can give the participants starting points to conduct experimental research and implement their own dialogue systems. In addition, we will describe current research issues and challenges, for example the use of 'thick pipelines' and corpus-based methods.

The tutorial will be structured as follows:

1. Introduction: historical remarks on first and second generation dialogue systems. Motivation for using spoken/multimodal dialogue systems; discussion of application scenarios in brief.
2. Theoretical framework: detailed description of individual modules (Speech Recognition, Natural Language Understanding, Dialogue Management, Task Execution, Response Generation and Speech synthesis), underlying theoretical ideas, dialogue system architectures and interaction between modules.
3. Toolkits and standards (e.g. VoiceXML, SALT), annotation and data collection, evaluation methodologies. HCI: general principles and typical HCI design errors.

At the end of this tutorial, the participants will have a firm grasp of the various processing stages employed in dialogue systems, understand the choices available to them and the trade-offs these involve, and will be able to put current research papers into a wider context.

Presenters

Giuseppe Riccardi, Sebastian Varges,
University of Trento
,

Short Bios

Prof. Riccardi received his Laurea degree in Electrical Engineering and Master in Information Technology, in 1991, from the University of Padua and CEFRIEL Research Center, respectively. From 1990-1993 he collaborated with Alcatel-Telettra Research Laboratories (Milan, Italy). In 1995 he received his Ph.D. in Electrical Engineering from the Department of Electrical Engineering at the University of Padua, Italy. From 1993-2005, he worked first at AT&T Bell Laboratories and then AT&T Labs-Research where he worked in the Speech and Language Processing Lab. In 2005 joined the faculty of Engineering at University of Trento (Italy) and is affiliated with the interdisciplinary Department of Information and Communication Technology and Center for Mind/Brain Sciences. Prof. Riccardi's research on stochastic finite state machines for speech and language processing has been applied to a wide range of domains for task automation. He participated in the creation of the state-of-the-art AT&T spoken language system used in the 1994 DARPA ATIS evaluation. He and his colleagues have been pioneering the speech and language research in spontaneous speech for the well-known "How May I Help You?" research program which led to a speech service breakthrough. His research on learning finite state automata and transducers has lead to the creation of the first large scale finite state chain decoding for machine translation (“Anuvaadi”).

Sebastian Varges is Senior Marie Curie Fellow in the ADAMACH project (ADAptive and meaning MACHines) at the DIT department of the University Trento. Before Trento, he conducted research on an in-car dialogue system at Stanford University and on natural language generation at the Universities of Brighton and Edinburgh (where he obtained his PhD on "Instance-based Natural Language Generation"). His research interests focus on dialogue systems and natural language generation, in particular in combining statistical and rule-based approaches.