September 2005: Speech Communications Best Paper Award 2003-2004 was given
by the European Association for Signal Processing (Eurasip) for
"Prosodic and other cues to speech recognition failures" by Hirschberg,
Litman and Swerts
Spoken Dialogue and Affect for Intelligent Tutoring Systems:
Spoken dialogue is a natural and highly desirable form of
student-computer interaction, which provides both opportunities and
challenges to both dialogue-based tutoring systems, and to spoken language systems.
The goal of my research is to wed spoken language
technology with instructional technology, in order to promote learning
gains by enhancing communication richness.
For further details, see the ITSPOKE webpage,
which contains information on the ITSPOKE system and corpora, as
well as information on the
back-end (Why2,
a text-based tutoring system in the domain of qualitative physics).
Exceptionality and Natural Language Learning:
Previous work has shown that
when machine learning is applied to many natural language processing
tasks, exceptional training examples play an important role in
improving generalization accuracy. We are exploring whether such
results generalize to spoken dialogue, and how different
formalizations of "exceptionality" impact the performance of memory-based
and rule-based learning algorithms.
Click for further details and online publications.
Question Answering:
The development of resources for extending the current automatic question-answering paradigm to hande opinion-oriented, rather than fact-oriented, questions.
Also, the use of ensemble methods to combine the output of multiple QA
systems to improve performance, in the reading comprehension domain.
Click for further details and online publications.
Spoken Dialogue for CHAT: The design,
implementation, and empirical experiences of CobotDS, a
spoken dialogue system for accessing the LambdaMoo
text-based chat environment.
CobotDS allows phone users to talk with LambdaMoo users
via Cobot, a
software agent residing in LambdaMOO.
Click for further details
and online publications.
Reinforcement Learning for Optimizing Spoken Dialogue Agents:
The use of reinforcement learning to analyze and optimize
dialogue strategy design in spoken dialogue systems.
An empirical evaluation of an automatically optimized dialogue manager.
Click for further details
and online publications.
Prosodic Analysis of Misrecognitions and Corrections in Spoken Dialogue:
Analytic and machine learning results indicating how
prosodic differences can predict
misrecognized vs. correctly recognized turns, and correction vs.
other types of utterances.
Click for further details
and online publications.
Learning how to Predict Problematic Dialogue Situations, and an Application
to Adaptive Spoken Dialogue:
The use of rule induction to predict problematic dialogue situations
(e.g. poor speech recognition, "bail out" situations where
a caller should be transferred to a human operator).
The design and evaluation of a spoken dialogue system that adapts its behavior when problematic situations
are detected.
Click for further details
and online publications.
Evaluating Spoken Dialogue Agents:
The PARADISE (PARAdigm for DIalogue System Evaluation) framework for empirically
deriving an objective performance function, and PARADISE evaluations of
cooperative responses, the use of tutorial dialogues,
and adaptable dialogue behavior.
Click for further details
and online publications.
Device Representation and Reasoning with Affective Relations, and an Implementation in R++:
An approach to monitoring and diagnosis of complex systems that
integrates classical model-based diagnosis and heuristic expert systems.
Implemented using R++, an extension of C++ that incorporates rules into the object-oriented paradigm.
Click for further details, and
online publications.
Terminological Reasoning with Plans and an Application to Plan Recognition:
A plan-based knowledge representation system that integrates
description logics
with metric and qualitative temporal constraint languages, and
a companion plan recognition system (T-REX) that uses the plan-based description logic.
CLASP is an alternative plan-based knowledge representation system that extends description logics to handle temporal information by representing plans as regular expressions.
Click for further details
and online publications.
A Corpus-Based Approach to Classifying Discourse Segment Boundaries and Cue Phrases in Text and Speech:
Empirical discourse analysis, with an emphasis on coding of data,
the use of machine learning for hypothesis formation,
and quantitative evaluation of results.
Click for further details
and online publications.
Plan Recognition:
The use of plan recognition in both natural language dialogue
systems and in intelligent graphical interfaces.
Click for further details
and online publications.
