Table of Contents

• A human-centered approach to design multimodal conversational systemsHeloisa Candello. 1 [doi]
• 3D Calling with Codec AvatarsYaser Sheikh. 2 [doi]
• Greta, what else? Our research towards building socially interactive agentsCatherine Pelachaud. 3 [doi]
• LLM-powered Multimodal Insight Summarization for UX TestingKelsey Turbeville, Jennarong Muengtaweepongsa, Samuel Stevens, Jason Moss, Amy Pon, Kyra Lee, Charu Mehra, Jenny Gutierrez Villalobos, Ranjitha Kumar. 4-11 [doi]
• On Multimodal Emotion Recognition for Human-Chatbot Interaction in the WildNikola Kovacevic, Christian Holz 0001, Markus Gross 0001, Rafael Wampfler. 12-21 [doi]
• Integrating Multimodal Affective Signals for Stress Detection from Audio-Visual DataDebasmita Ghose, Oz Gitelson, Brian Scassellati. 22-32 [doi]
• Feeling Textiles through AI: An exploration into Multimodal Language Models and Human Perception AlignmentShu Zhong, Elia Gatti, Youngjun Cho, Marianna Obrist. 33-37 [doi]
• Decoding Contact: Automatic Estimation of Contact Signatures in Parent-Infant Free Play InteractionsMetehan Doyran, Albert Ali Salah, Ronald Poppe. 38-46 [doi]
• ScentHaptics: Augmenting the Haptic Experiences of Digital Mid-Air Textiles with ScentChristopher Dawes, Jing Xue, Giada Brianza, Patricia Ivette Cornelio Martinez, Roberto A. Montano Murillo, Emanuela Maggioni, Marianna Obrist. 47-56 [doi]
• Online Multimodal End-of-Turn Prediction for Three-party ConversationsMeng Chen Lee, Zhigang Deng. 57-65 [doi]
• Detecting Deception in Natural Environments Using Incremental Transfer LearningMuneeb Imtiaz Ahmad, Abdullah Alzahrani, Sunbul M. Ahmad. 66-75 [doi]
• MR-Driven Near-Future Realities: Previewing Everyday Life Real-World Experiences Using Mixed RealityFlorian Mathis, Brad A. Myers, Ben Lafreniere, Michael Glueck, David P. S. Marques. 76-85 [doi]
• Exploring Interlocutor Gaze Interactions in Conversations based on Functional Spectrum AnalysisAyane Tashiro, Mai Imamura, Shiro Kumano, Kazuhiro Otsuka. 86-94 [doi]
• Leveraging Prosody as an Informative Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic ImplicationsMatilda Knierim, Sahil Jain, Murat Han Aydogan, Kenneth Mitra, Kush Desai, Akanksha Saran, Kim Baraka. 95-123 [doi]
• Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing?TianTian Feng, Daniel Yang, Digbalay Bose, Shrikanth Narayanan. 124-133 [doi]
• The Impact of Auditory Warning Types and Emergency Obstacle Avoidance Takeover Scenarios on Takeover BehaviorXuenan Li, Zhaoyang Xu. 134-143 [doi]
• Juicy Text: Onomatopoeia and Semantic Text Effects for Juicy Player ExperiencesÉmilie Fabre, Katie Seaborn, Adrien Verhulst, Yuta Itoh 0001, Jun Rekimoto. 144-153 [doi]
• Exploring the Alteration and Masking of Everyday Noise Sounds using Auditory Augmented RealityIsna Alfi Bustoni, Mark McGill, Stephen Anthony Brewster. 154-163 [doi]
• Understanding Non-Verbal Irony Markers: Machine Learning Insights Versus Human JudgmentMicol Spitale, Fabio Catania, Francesca Panzeri. 164-172 [doi]
• Generalization Boost in Bimodal Classification via Data Fusion Trained on Sparse DatasetsWentao Yu, Dorothea Kolossa, Robert M. Nickel. 173-181 [doi]
• Envisioning Futures: How the Modality of AI Recommendations Impacts Conversation Flow in AR-enhanced DialogueSteeven Villa, Yannick Weiss, Mei-Yi Lu, Moritz Ziarko, Albrecht Schmidt 0001, Jasmin Niess. 182-193 [doi]
• "Uh, This One?": Leveraging Behavioral Signals for Detecting Confusion during Physical TasksMaia Stiber, Dan Bohus, Sean Andrist. 194-203 [doi]
• NapTune: Efficient Model Tuning for Mood Classification using Previous Night's Sleep Measures along with Wearable Time-seriesDebaditya Shome, Nasim Montazeri Ghahjaverestan, Ali Etemad. 204-213 [doi]
• Whispering Wearables: Multimodal Approach to Silent Speech Recognition with Head-Worn DevicesTanmay Srivastava, R. Michael Winters, Thomas M. Gable, Yu-Te Wang, Teresa LaScala, Ivan J. Tashev. 214-223 [doi]
• Multilingual Dyadic Interaction Corpus NoXi+J: Toward Understanding Asian-European Non-verbal Cultural Characteristics and their Influences on EngagementMarius Funk, Shogo Okada, Elisabeth André. 224-233 [doi]
• NearFetch: Enhancing Touch-Based Mobile Interaction on Public Displays with an Embedded Programmable NFC ArrayQijun Cao, Junqi Zhang, Shengtao Fan, Jiaqi Rong, Menghao Qi, Zhuowen Duan, Peikun Zhao, Ling Liu, Zihao Zhou, Wenjie Chen. 234-243 [doi]
• Detecting Aware and Unaware Mind Wandering During Lecture Viewing: A Multimodal Machine Learning Approach Using Eye Tracking, Facial Videos and Physiological DataBabette Bühler, Efe Bozkir, Hannah Deininger, Patricia Goldberg, Peter Gerjets, Ulrich Trautwein, Enkelejda Kasneci. 244-253 [doi]
• M2RL: A Multimodal Multi-Interface Dataset for Robot Learning from Human DemonstrationsShaid Hasan, Mohammad Samin Yasar, Tariq Iqbal. 254-263 [doi]
• SemanticTap: A Haptic Toolkit for Vibration Semantic Design of SmartphoneRui Zhang, Yixuan Li, Zihuang Wu, Yong Zhang, Jie Zhao, Yang Jiao. 264-273 [doi]
• Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic EvaluationEsam Ghaleb, Bulat Khaertdinov, Wim T. J. L. Pouw, Marlou Rasenberg, Judith Holler, Asli Özyürek, Raquel Fernández. 274-283 [doi]
• Mitigation of gender bias in automatic facial non-verbal behaviors generationAlice Delbosc, Magalie Ochs, Nicolas Sabouret, Brian Ravenet, Stéphane Ayache. 284-292 [doi]
• Poke Typing: Effects of Hand-Tracking Input and Key Representation on Mid-Air Text Entry Performance in Virtual RealityMehmet Akhoroz, Caglar Yildirim. 293-301 [doi]
• A multimodal analysis of environmental stress experienced by older adults during outdoor walking trips: Implications for designing new intelligent technologies to enhance walkability in low-income Latino communitiesRaquel Yupanqui, John Sohn, Yoojun Kim, Raquel Flores, Hanwool Lee, Jinwoo Kim, Sanghyun Lee, Youngjib Ham, Chanam Lee, Theodora Chaspari. 302-311 [doi]
• Emotion Recognition for Multimodal Recognition of Attachment in School-Age ChildrenAreej Buker, Alessandro Vinciarelli. 312-320 [doi]
• Is Distance a Modality? Multi-Label Learning for Speech-Based Joint Prediction of Attributed Traits and Perceived Distances in 3D Audio Immersive EnvironmentsEva Fringi, Nesreen Alshubaily, Lorenzo Picinali, Stephen Anthony Brewster, Tanaya Guha, Alessandro Vinciarelli. 321-330 [doi]
• The Plausibility Paradox on Interactions with Complex Virtual Objects in Virtual EnvironmentsDaniel Alvarado-Chou, Yuen C. Law. 331-338 [doi]
• SMURF: Statistical Modality Uniqueness and Redundancy FactorizationTorsten Wörtwein, Nicholas B. Allen, Jeffrey F. Cohn, Louis-Philippe Morency. 339-349 [doi]
• Detecting Autism from Head Movements using KinesicsMuhittin Gokmen, Evangelos Sariyanidi, Lisa Yankowitz, Casey J. Zampella, Robert T. Schultz, Birkan Tunç. 350-354 [doi]
• Automatic mild cognitive impairment estimation from the group conversation of coimagination methodSixia Li, Kazumi Kumagai, Mihoko Otake-Matsuura, Shogo Okada. 355-360 [doi]
• Generating Facial Expression Sequences of Complex Emotions with Generative Adversarial NetworksZakariae Belmekki, David Antonio Gómez Jáuregui, Patrick Reuter, Jun Li, Jean-Claude Martin, Karl Jenkins, Nadine Couture. 361-372 [doi]
• A Model of Factors Contributing to the Success of Dialogical ExplanationsMeisam Booshehri, Hendrik Buschmeier, Philipp Cimiano. 373-381 [doi]
• Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality ForecastingDivij Gupta, Anubhav Bhatti, Suraj Parmar, Chen Dan 0005, Yuwei Liu, Bingjie Shen, San Lee. 382-386 [doi]
• Nonverbal Dynamics in Dyadic Videoconferencing Interaction: The Role of Video Resolution and Conversational QualityChenyao Diao, Stephanie Arévalo Arboleda, Alexander Raake. 387-396 [doi]
• Perception of Stress: A Comparative Multimodal Analysis of Time-Continuous Stress Ratings from Self and ObserversEhsanul Haque Nirjhar, Winfred Arthur, Theodora Chaspari. 397-406 [doi]
• Putting the "Brain" Back in the Eye-Mind Link: Aligning Eye Movements and Brain Activations During Naturalistic ReadingMegan Caruso, Rosy Southwell, Leanne M. Hirshfield, Sidney D'Mello. 407-417 [doi]
• Perceived Text Relevance Estimation Using Scanpaths and GNNsAbdulrahman Mohamed Selim, Omair Shahzad Bhatti, Michael Barz, Daniel Sonntag. 418-427 [doi]
• Towards Automated Annotation of Infant-Caregiver Engagement Phases with Multimodal Foundation ModelsDaksitha Senel Withanage Don, Dominik Schiller, Tobias Hallmen, Silvan Mertes, Tobias Baur 0001, Florian Lingenfelser, Mitho Müller, Lea Kaubisch, Corinna Reck, Elisabeth André. 428-438 [doi]
• First-Person Perspective Induces Stronger Feelings of Awe and Presence Compared to Third-Person Perspective in Virtual RealityHiromu Otsubo, Alexander Marquardt, Melissa Steininger, Marvin Lehnort, Felix Dollack, Yutaro Hirao, Monica Perusquía-Hernández, Hideaki Uchiyama, Ernst Kruijff, Bernhard E. Riecke, Kiyoshi Kiyokawa. 439-448 [doi]
• Predictability of Understanding in Explanatory Interactions Based on Multimodal CuesOlcay Türk, Stefan Lazarov, Yu Wang, Hendrik Buschmeier, Angela Grimminger, Petra Wagner. 449-458 [doi]
• Distinguishing Target and Non-Target Fixations with EEG and Eye Tracking in Realistic Visual ScenesMansi Sharma, Camilo Andrés Martínez Martínez, Benedikt Emanuel Wirth, Antonio Krüger, Philipp Müller 0001. 459-468 [doi]
• Multimodal User Enjoyment Detection in Human-Robot Conversation: The Power of Large Language ModelsAndré Pereira 0001, Lubos Marcinek, Jura Miniota, Sofia Thunberg, Erik Lagerstedt, Joakim Gustafson, Gabriel Skantze, Bahar Irfan. 469-478 [doi]
• Lip Abnormality Detection for Patients with Repaired Cleft Lip and Palate: A Lip Normalization ApproachKaren Rosero, Ali N. Salman, Rami R. Hallac, Carlos Busso. 479-487 [doi]
• MSP-GEO Corpus: A Multimodal Database for Understanding Video-Learning ExperienceAli N. Salman, Ning Wang, Luz Martinez-Lucas, Andrea Vidal, Carlos Busso. 488-497 [doi]
• Improving Usability of Data Charts in Multimodal Documents for Low Vision UsersYash Prakash, Akshay Kolgar Nayak, Shoaib Mohammed Alyaan, Pathan Aseef Khan, Hae-na Lee, Vikas Ashok. 498-507 [doi]
• Stressor Type Matters! - Exploring Factors Influencing Cross-Dataset Generalizability of Physiological Stress DetectionPooja Prajod, Bhargavi Mahesh, Elisabeth André. 508-517 [doi]
• Across Trials vs Subjects vs Contexts: A Multi-Reservoir Computing Approach for EEG Variations in Emotion RecognitionAnubhav, Kantaro Fujiwara. 518-525 [doi]
• DoubleDistillation: Enhancing LLMs for Informal Text Analysis using Multistage Knowledge Distillation from Speech and TextFatema Hasan, Yulong Li, James R. Foulds, Shimei Pan, Bishwaranjan Bhattacharjee. 526-535 [doi]
• Predicting Human Intent to Interact with a Public Robot: The People Approaching Robots Database (PAR-D)Sydney Thompson, Alexander Lew, Yifan Li, Elizabeth Stanish, Alex Huang, Rohan Phanse, Marynel Vázquez. 536-545 [doi]
• SEMPI: A Database for Understanding Social Engagement in Video-Mediated Multiparty InteractionMaksim Siniukov, Yufeng Yin 0002, Eli Fast, Yingshan Qi, Aarav Monga, Audrey Kim, Mohammad Soleymani 0001. 546-555 [doi]
• Participation Role-Driven Engagement Estimation of ASD Individuals in Neurodiverse Group DiscussionsKalin Stefanov, Yukiko I. Nakano, Chisa Kobayashi, Ibuki Hoshina, Tatsuya Sakato, Fumio Nihei, Chihiro Takayama, Ryo Ishii, Masatsugu Tsujii. 556-564 [doi]
• Do We Need To Watch It All? Efficient Job Interview Video Processing with Differentiable MaskingHung Le, Sixia Li, Candy Olivia Mawalim, Hung-Hsuan Huang, Chee Wee Leong, Shogo Okada. 565-574 [doi]
• Relating Students Cognitive Processes and Learner-Centered Emotions: An Advanced Deep Learning ApproachAshwin T. S, Gautam Biswas. 575-584 [doi]
• RealSeal: Revolutionizing Media Authentication with Real-Time Realism ScoringBhaktipriya Radharapu, Harish Krishna. 585-590 [doi]
• AI as Modality in Human Augmentation: Toward New Forms of Multimodal Interaction with AI-Embodied ModalitiesRadu-Daniel Vatavu. 591-595 [doi]
• Everything We Hear: Towards Tackling Misinformation in PodcastsSachin Pathiyan Cherumanal, Ujwal Gadiraju, Damiano Spina. 596-601 [doi]
• A musical Robot for People with DementiaPaul Raingeard de la Bletiere. 602-606 [doi]
• Enhancing Collaboration and Performance among EMS Students through Multimodal Learning AnalyticsVasundhara Joshi. 607-611 [doi]
• Towards Automatic Social Involvement EstimationZonghuan Li. 612-616 [doi]
• Video Game Technologies Applied for Teaching Assembly Language ProgrammingErnesto Rivera-Alvarado. 617-621 [doi]
• Modelling Social Intentions in Complex Conversational SettingsIvan Kondyurin. 622-626 [doi]
• Real-Time Trust Measurement in Human-Robot Interaction: Insights from Physiological BehavioursAbdullah Alzahrani, Muneeb Imtiaz Ahmad. 627-631 [doi]
• A Multimodal Understanding of the Eye-Mind LinkMegan Caruso. 632-636 [doi]
• Investigating Multi-Reservoir Computing for EEG-based Emotion RecognitionAnubhav. 637-641 [doi]
• Design Digital Multisensory Textile ExperiencesShu Zhong. 642-646 [doi]
• Towards Trustworthy and Efficient Diffusion ModelsJayneel Vora. 647-651 [doi]
• ERR@HRI 2024 Challenge: Multimodal Detection of Errors and Failures in Human-Robot InteractionsMicol Spitale, Maria Teresa Parreira, Maia Stiber, Minja Axelsson, Neval Kara, Garima Kankariya, Chien-Ming Huang 0001, Malte F. Jung, Wendy Ju, Hatice Gunes. 652-656 [doi]
• A Time Series Classification Pipeline for Detecting Interaction Ruptures in HRI Based on User ReactionsLennart Wachowiak, Peter Tisnikar, Andrew Coles, Gerard Canal, Oya Çeliktutan. 657-665 [doi]
• PRISCA at ERR@HRI 2024: Multimodal Representation Learning for Detecting Interaction Ruptures in HRIPradip Pramanick, Silvia Rossi 0002. 666-670 [doi]
• Predicting Errors and Failures in Human-Robot Interaction from Multi-Modal Temporal DataRuben Janssens, Eva Verhelst, Mathieu De Coster. 671-676 [doi]
• EVAC 2024 - Empathic Virtual Agent Challenge: Appraisal-based Recognition of Affective StatesFabien Ringeval, Björn W. Schuller, Gérard Bailly, Safaa Azzakhnini, Hippolyte Fournier. 677-683 [doi]
• Multimodal Emotion Recognition Harnessing the Complementarity of Speech, Language, and VisionThomas Thebaud, Anna Favaro, Yaohan Guan, Yuchen Yang, Prabhav Singh, Jesús Villalba 0001, Laureano Mono-Velazquez, Najim Dehak. 684-689 [doi]
• First Multimodal Banquet: Exploring Innovative Technology for Commensality and Human-Food Interaction (CoFI2024)Radoslaw Niewiadomski, Ferran Altarriba Bertran, Christopher Dawes, Marianna Obrist, Maurizio Mancini. 690-693 [doi]
• GENEA Workshop 2024: The 5th Workshop on Generation and Evaluation of Non-verbal Behaviour for Embodied AgentsYoungwoo Yoon, Taras Kucherenko, Alice Delbosc, Rajmund Nagy, Teodor Nikolov, Gustav Eje Henter. 694-695 [doi]
• HumanEYEze 2024: Workshop on Eye Tracking for Multimodal Human-Centric ComputingMichael Barz, Roman Bednarik, Andreas Bulling, Cristina Conati, Daniel Sonntag. 696-697 [doi]
• Multimodal Co-Construction of Explanations with XAI WorkshopHendrik Buschmeier, Teena Hassan, Stefan Kopp. 698-699 [doi]