[et_pb_section fb_built=”1″ custom_padding_last_edited=”off|desktop” admin_label=”Hero” _builder_version=”4.24.2″ background_enable_color=”off” custom_margin=”|||” custom_padding=”50px||0px|||” custom_padding_tablet=”130px||130px|” global_colors_info=”{}”][et_pb_row _builder_version=”4.27.4″ _module_preset=”default” width=”100%” max_width=”1280px” custom_padding=”|15px|3px|15px|false|false” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” header_font=”|||on|||||” custom_margin=”||1px|||” global_colors_info=”{}”]<\/p>\n
[\/et_pb_text][et_pb_divider divider_weight=”3px” _builder_version=”4.24.2″ _module_preset=”default” width=”25%” module_alignment=”left” global_colors_info=”{}”][\/et_pb_divider][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_team_member name=”Professor Jo\u00e3o Magalh\u00e3es” position=”Full Professor at the Computer Science Dep. at Universidade NOVA de Lisboa and national co-Director of the CMU Portugal partnership.” image_url=”https:\/\/iberspeech.tech\/2026\/wp-content\/uploads\/2026\/04\/Joao-Magalhaes-profile.jpeg” _builder_version=”4.27.4″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” header_font_size=”28px” sticky_enabled=”0″ position_font_size=”18px” custom_margin=”0px||||false|false”]<\/p>\n
<\/span><\/p>\n Jo\u00e3o Magalh\u00e3es holds a Ph.D. degree (2008) from Imperial College London, UK. His research aims to move vision and language AI closer to the way humans understand it and communicate. He has made scientific contributions to the fields of multimedia search and summarization, multimodal conversational AI, data mining and multimodal information representation. He is currently coordinating the creation of the sovereign LLM AMALIA, and, in the past, has coordinated and participated in several research projects (national, EU-FP7 and H2020) where he pursues robust and generalizable methods in different domains. He is regularly involved in review panels, organization of international conferences and program committees. His work and the work of his group has been awarded, or nominated for, several honours and distinctions, most notably the 1st prize in the Amazon Alexa Taskbot Challenge 2022. He was the General Chair of ECIR 2020 and ACM Multimedia 2022, Honorary Chair for ACM Multimedia Asia 2021 and will be the PC chair of ACM Multimedia 2026.<\/span><\/p>\n [\/et_pb_team_member][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”][et_pb_toggle title=”Title of the talk: Multimodal Conversational Assistance of Complex Manual Tasks” open=”on” toggle_icon=”C||divi||400″ _builder_version=”4.27.4″ _module_preset=”default” title_level=”h2″ title_font=”|||||on|||” body_font_size=”15px” custom_margin=”||30px|||” hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n Conversational agents have become an integral part of our daily routines, aiding humans in various tasks. Helping users in real-world manual tasks is a complex and challenging paradigm, where it is necessary to leverage multiple information sources, provide several multimodal stimuli, and be able to correctly ground the conversation in a helpful and robust manner. In this talk I will describe TWIZ, a conversational AI assistant that is helpful, multimodal, knowledgeable, and engaging, and designed to guide users towards the successful completion of complex manual tasks. To achieve this, we focused our efforts on three main research questions: (1) Humanly-Shaped Conversations, by providing information in a knowledgeable way; (2) Multimodal Stimulus, making use of various modalities including voice, images, and videos; and (3) Zero-shot Conversational Flows, to improve the robustness of the interaction to unseen scenarios. TWIZ is an assistant capable of supporting a wide range of unseen tasks — it leverages Generative AI methods to deliver several innovative features such as creative cooking, video navigation through voice, and the robust PlanLLM, a Large Language Model trained for dialoguing about complex manual tasks.<\/span><\/p>\n [\/et_pb_toggle][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.16″ _module_preset=”%22default%22″ custom_padding=”%220px||2px|||%22″ global_colors_info=”%22{}%22″][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.16″ _module_preset=”%22default%22″ custom_padding=”%220px||2px|||%22″ global_colors_info=”%22{}%22″][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":" KEYNOTE SPEAKERS Jo\u00e3o Magalh\u00e3es holds a Ph.D. degree (2008) from Imperial College London, UK. His research aims to move vision and language AI closer to the way humans understand it and communicate. He has made scientific contributions to the fields of multimedia search and summarization, multimodal conversational AI, data mining and multimodal information representation. He […]<\/p>\n","protected":false},"author":4,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":"","_links_to":"","_links_to_target":""},"class_list":["post-778","page","type-page","status-publish","hentry"],"yoast_head":"\nAbstract<\/span><\/h4>\n