Feedback, Communicative Gesturing, and Gazing

This PhD school focuses on multimodal and multilingual feedback in natural language interactions.

Feedback enables building of shared context and helps to maintain rapport and social bonds and is not only restricted to certain utterances and back-channelling behaviour. It can be understood in a large sense referring to the interlocutors' reaction when they take part in the conversation, especially how they use their hand gestures, body posture, gazing and facial expressions to express their contact, perception and understanding of the communicative situation.

The progress made in speech and language technology has greatly improved the study of communication and especially multimodal feedback giving processes through the collection of high-quality video recordings. It provides automatic analysis tools that enable accurate time-alignment and visualisation of the phenomena under study.

The school fosters the study of social communication and the use of video corpora across the Nordic countries. It aims at deepening the students' understanding of how gesturing and gazing are used to give feedback in different languages and providing material for comparison and categorization of various communicative strategies.

The focus is especially on Finnish, Swedish, Danish, and Estonian, but comparison is also extended to Japanese conversations which provide an opportunity to compare communicative functions beyond the Nordic sphere.

Read more details about the course here

Facts about the project
Project leader

Kristiina Jokinen, University of Helsinki, Finland