KEYNOTE SPEAKERS
Professor João Magalhães
Full Professor at the Computer Science Dep. at Universidade NOVA de Lisboa and national co-Director of the CMU Portugal partnership.
João Magalhães holds a Ph.D. degree (2008) from Imperial College London, UK. His research aims to move vision and language AI closer to the way humans understand it and communicate. He has made scientific contributions to the fields of multimedia search and summarization, multimodal conversational AI, data mining and multimodal information representation. He is currently coordinating the creation of the sovereign LLM AMALIA, and, in the past, has coordinated and participated in several research projects (national, EU-FP7 and H2020) where he pursues robust and generalizable methods in different domains. He is regularly involved in review panels, organization of international conferences and program committees. His work and the work of his group has been awarded, or nominated for, several honours and distinctions, most notably the 1st prize in the Amazon Alexa Taskbot Challenge 2022. He was the General Chair of ECIR 2020 and ACM Multimedia 2022, Honorary Chair for ACM Multimedia Asia 2021 and will be the PC chair of ACM Multimedia 2026.
Title of the talk: "Multimodal Conversational Assistance of Complex Manual Tasks"
Abstract
Conversational agents have become an integral part of our daily routines, aiding humans in various tasks. Helping users in real-world manual tasks is a complex and challenging paradigm, where it is necessary to leverage multiple information sources, provide several multimodal stimuli, and be able to correctly ground the conversation in a helpful and robust manner. In this talk I will describe TWIZ, a conversational AI assistant that is helpful, multimodal, knowledgeable, and engaging, and designed to guide users towards the successful completion of complex manual tasks. To achieve this, we focused our efforts on three main research questions: (1) Humanly-Shaped Conversations, by providing information in a knowledgeable way; (2) Multimodal Stimulus, making use of various modalities including voice, images, and videos; and (3) Zero-shot Conversational Flows, to improve the robustness of the interaction to unseen scenarios. TWIZ is an assistant capable of supporting a wide range of unseen tasks — it leverages Generative AI methods to deliver several innovative features such as creative cooking, video navigation through voice, and the robust PlanLLM, a Large Language Model trained for dialoguing about complex manual tasks.
Prof. Dr. Elisabeth André
Full Professor of Computer Science, Chair of Human-Centered Artificial Intelligence, Faculty of Applied Informatics, Augsburg University
Elisabeth André is a Full Professor of Computer Science and the Founding Chair of Human-Centered Artificial Intelligence at Augsburg University, Germany. A global leader in multimodal human-machine interaction and social signal processing, her work has been foundational in enabling machines to perceive and respond to human speech, gestures, and emotions in a natural, socially intelligent manner. Her research is dedicated to the development of “believable” virtual agents and social robots capable of sophisticated, human-like dialogue. For her pioneering contributions to artificial emotional intelligence, she was awarded the Gottfried Wilhelm Leibniz Prize by the German Research Foundation (DFG). As Germany’s most prestigious research honor, the prize recognizes her trailblazing role in bridging the gap between Artificial Intelligence and Human-Computer Interaction to create technology that is more intuitive and empathetic. Beyond her technical research, Professor André is recognized as one of the most influential voices in the field. In 2024, Manager Magazin named her one of the 15 most important women in AI in Germany, and in 2019, she was honored by the National Society for Informatics (GI) as one of the ten most influential figures in the history of German AI. Her long-standing impact on the community was further recognized with the ICMI Sustained Accomplishment Award and the 2025 AI Visionary TIGER Award. Professor André is a member of the German Academy of Sciences (Leopoldina), the CHI Academy, and a EurAI Fellow. She currently co-leads the “Work/Qualification and Human-Machine Interaction” group for Germany’s National Platform for Artificial Intelligence.
Title of the talk: "From Speech to Multimodal Interaction: Guardrails for Socially-Interactive AI"
Abstract
When speech moves from transcription to multimodal interaction, its requirements change fundamentally. In such settings, systems must operate under ambiguity and uncertainty while accounting for social context and application-specific constraints. This talk focuses on the design of guardrails that go beyond generic content filtering. I will present approaches for risk identification, evaluation, and mitigation, including mechanisms for uncertainty handling, policy compliance, and improving the reliability of LLM-based systems in multimodal interaction. These challenges and solutions are illustrated through three complementary perspectives from interdisciplinary projects: social coaching through role-play with robots (CAIDA) and virtual agents for children (CONFIDENCE), where particular care must be taken to avoid stigmatization of marginalized groups and psychologically harmful interaction scenarios; language-enabled robotics in healthcare (REGINA), where systems must ensure confidentiality, correctness, and certifiability while remaining accessible through natural language and low-code/no-code interfaces; and CAR-bench, a benchmark for evaluating consistency, uncertainty awareness, and capability awareness in multi-turn, tool-using LLM agents for in-car assistant scenarios.