Multimodal Corpora: From Models of Natural Interaction to Systems and Applications

ISBN-13: 9783642047923 / Angielski / Miękka / 2009 / 223 str.

Michael Kipp;Jean-Claude Martin;Patrizia Paggio

Opis

Thebookpresentsa cross-sectionofstate-of-the-artresearchonmultimodalc- pora, a highly interdisciplinary area that is a prerequisite for various specialized disciplines. A number of the papers included are revised and expanded versions ofpapersacceptedtotheInternationalWorkshoponMultimodal Corpora: From Models of Natural Interaction to Systems and Applications, held in conjunction th with the 6 International Conference for Language Resources and Evaluation (LREC) on May 27, 2008, in Marrakech, Morocco. This international workshop series started in 2000 and has since then grown into a regular satellite event of the bi-annual LREC conference, attracting researchers from ?elds as diverse as psychology, arti?cial intelligence, robotics, signal processing, computational linguisticsandhuman computerinteraction. Tocomplement theselected papers from the 2008 workshop, we invited well-known researchers from corpus coll- tioninitiativestocontributetothisvolume. Wewereabletoobtainseveninvited research articles, including contributions from major international multimodal corpus projects like AMI and SmartWeb, which complement the six selected workshop contributions. All papers underwent a special review process for this volume, resulting in signi?cant revisions and extensions based on the experts advice. While we were pleased that the 2006 edition of the workshop resulted in a special issue of the Journal of Language Resources and Evaluation, published in 2007, we felt that this was the time for another major publication, given not onlytherapidprogressandincreasedinterestin this researchareabut especially in order to acknowledge the di?culty of disseminating results across discipline borders. The Springer LNAI series is the perfect platform for doing so. We also created the website www. multimodal-corpora."

Spis treści

Collecting and Distributing Multimodal Corpora.- The SmartWeb Corpora: Multimodal Access to the Web in Natural Environments.- Promoting free Dialog Video Corpora: The IFADV Corpus Example.- Coding and Analyzing Multimodal Corpora.- Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project.- Multi-track Annotation of Child Language and Gestures.- Gesture, Gaze and Persuasive Strategies in Political Discourse.- Unsupervised Clustering in Multimodal Multiparty Meeting Analysis.- Validating Multimodal Corpora.- Multimodal Corpora Annotation: Validation Methods to Assess Coding Scheme Reliability.- On the Contextual Analysis of Agreement Scores.- The Intercultural Dimension of Multimodal Corpora.- Creating Standardized Video Recordings of Multimodal Interactions across Cultures.- Multimodal Intercultural Information and Communication Technology – A Framework for Designing and Evaluating Multimodal Intercultural Communicators.- Tools for Browsing, Coding and Exchanging Data in Multimodal Corpora.- Tools and Resources for Visualising Conversational-Speech Interaction.- Accessing a Large Multimodal Corpus Using an Automatic Content Linking Device.- An Exchange Format for Multimodal Annotations.

Blurb

Empirically based research in multimodality starts with the collection of video data relevant for a specific research question. This data must then be enriched by human coders, who annotate the corpus and who also explore the potential interconnections between modalities (e.g., face, gesture, speech, posture, etc.). The data must then be validated for consistency using quantitative measures -- systematic analysis of the observed multimodal behaviors is the central characteristic of multimodal corpora. For analysis and modeling tasks, exploratory qualitative and quantitative analysis tools are needed for browsing, viewing, extraction, and modeling. At the same time, the intercultural dimension must be taken into account, since multimodal interaction is always embedded in a social setting.

This state-of-the-art survey documents the scientific outcome of the International Workshop on "Multimodal Corpora: From Models of Natural Interaction to Systems and Applications", held in conjunction with the 6th International Conference for Language Resources and Evaluation (LREC), on May 27, 2008, in Marrakech, Morocco. It presents current research from fields as diverse as psychology, artificial intelligence, robotics, signal processing, computational linguistics, and human-computer interaction.

The 6 selected workshop contributions are complemented by 7 invited research articles, including contributions from major international multimodal corpus projects like AMI and SmartWeb. All papers underwent a special review process for this volume, resulting in significant revisions and extensions based on the experts' advice.