TY - CPAPER TI - Behavioral and Perceptual Responses to 3D Character and Volumetric Avatars Under Conflicting Guidance in Mixed Reality AU - de Souza Cardoso, Luis Fernando AU - Schwandt, Tobias AU - Schleising, Andy AU - Jörg, Sophie AU - Broll, Wolfgang PY - 2026 SP - 26-34 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/115657 LA - eng AB - In Mixed Reality (MR), in-the-moment guidance can support physical tasks under time and attention constraints. In contexts where such guidance is embodied, users must rapidly evaluate whether to trust and follow an advisor’s recommendations. Prior work has examined how avatar representation shapes trust and social perception, but less is known about user responses when guidance conflicts and multiple embodied agents compete for attention in real time. We study trust-relevant perception and behavior in an MR paradigm where two co-present agents represent the same advisor and deliver step-by-step instructions with predefined disagreements. In a within-subjects study, we compared a volumetric avatar (VA) and a high-fidelity 3D character avatar (CA), holding identity and expressive behavior constant, and measured trust, social presence, and behavioral adherence during contradictory steps. Rendering style produced no detectable differences in trust, social presence, or co-presence. Instead, participants managed disagreement through mixed adherence and switching between agents, suggesting that instruction competition and time pressure strongly shape decision-making. These findings position conflicting guidance as a distinct context for studying trust in embodied agents, where compliance may reflect rapid prioritization rather than stable commitment to a representation. We discuss implications for designing and evaluating MR agents when timing and attention constraints can overshadow representation cues. Figure 1: KW - Embodied Interaction KW - Virtual/Augmented Reality KW - Avatar Realism KW - Quantitative Methods ER - TY - CPAPER TI - Behavioral and Perceptual Responses to 3D Character and Volumetric Avatars Under Conflicting Guidance in Mixed Reality AU - de Souza Cardoso, Luis Fernando AU - Schwandt, Tobias AU - Schleising, Andy AU - Jörg, Sophie AU - Broll, Wolfgang PY - 2026 T2 - IMX '26: Proceedings of the 2026 ACM International Conference on Interactive Media Experiences SP - 26-34 PB - ACM CY - New York, NY SN - 979-8-4007-2448-0 DO - 10.1145/3788851.3805007 UR - https://fis.uni-bamberg.de/handle/uniba/115536 LA - eng AB - In Mixed Reality (MR), in-the-moment guidance can support physical tasks under time and attention constraints. In contexts where such guidance is embodied, users must rapidly evaluate whether to trust and follow an advisor’s recommendations. Prior work has examined how avatar representation shapes trust and social perception, but less is known about user responses when guidance conflicts and multiple embodied agents compete for attention in real time. We study trust-relevant perception and behavior in an MR paradigm where two co-present agents represent the same advisor and deliver step-by-step instructions with predefined disagreements. In a within-subjects study, we compared a volumetric avatar (VA) and a high-fidelity 3D character avatar (CA), holding identity and expressive behavior constant, and measured trust, social presence, and behavioral adherence during contradictory steps. Rendering style produced no detectable differences in trust, social presence, or co-presence. Instead, participants managed disagreement through mixed adherence and switching between agents, suggesting that instruction competition and time pressure strongly shape decision-making. These findings position conflicting guidance as a distinct context for studying trust in embodied agents, where compliance may reflect rapid prioritization rather than stable commitment to a representation. We discuss implications for designing and evaluating MR agents when timing and attention constraints can overshadow representation cues. Figure 1: KW - Embodied Interaction KW - Virtual/Augmented Reality KW - Avatar Realism KW - Quantitative Methods ER - TY - JOUR TI - Modeling and Animation Virtual Humans for Real-time Applications AU - Hülsken, Frank AU - Eckes, Christian AU - Kuck, Roland AU - Unterberg, Jörg AU - Jörg, Sophie PY - 2026 SP - 11-20 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/114956 LA - eng AB - We report on the workflow for the creation of realistic virtual anthropomorphic characters. 3D-models of human heads have been reconstructed from real people by following a structured light approach to 3D-reconstruction. We describe how these high-resolution models have been simplified and articulated with blend shape and mesh skinning techniques to ensure real-time animation. The full-body models have been created manually based on photographs. We present a system for capturing whole body motions, including the fingers, based on an optical motion capture system with 6 DOF rigid bodies and cybergloves. The motion capture data was processed in one system, mapped to a virtual character and visualized in real-time. We developed tools and methods for quick post processing. To demonstrate the viability of our system, we captured a library consisting of more than 90 gestures. KW - Real-time animation KW - 3D-reconstruction KW - optical motion capture KW - facial animation ER - TY - CPAPER TI - Real-time Hand Motion Synthesis for Playing a Virtual Guitar AU - Canales, Ryan AU - Jörg, Sophie PY - 2026 SP - 1-11 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/112832 LA - eng AB - Concerts and performances in virtual reality are becoming increasingly popular. Accurately visualizing the detailed hand motions of musicians playing instruments is challenging. In this work, we present a real-time motion synthesis method that generates the detailed fretting hand motion for playing guitar. Our approach first involves capturing and post-processing hand motion data from guitar performances to create a data set for training. The post-processing ensures that the fingertip positions are placed as accurately as possible regarding distance from the fretboard and location between the frets. We then train a neural network that learns to predict hand and finger poses based on guitar tabs and previous poses. We found that our method produces reasonably stable motion and evaluate our results using accuracy measures and visual evaluation. KW - character animation KW - motion synthesis KW - virtual humans KW - hand motions ER - TY - CPAPER TI - User identification based on conversational gestures AU - Normoyle, Aline AU - Jörg, Sophie PY - 2025 T2 - 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) : proceedings SP - 291-294 PB - IEEE DO - 10.1109/vrw66409.2025.00068 UR - https://fis.uni-bamberg.de/handle/uniba/116762 LA - eng ER - TY - CPAPER TI - Real-time Hand Motion Synthesis for Playing a Virtual Guitar AU - Canales, Ryan AU - Jörg, Sophie PY - 2025 T2 - Proceedings of the 2025 18th ACM SIGGRAPH Conference on Motion, Interaction, and Games IS - 12 SP - 1-11 PB - ACM CY - New York SN - 979-8-4007-2236-3 DO - 10.1145/3769047.3769060 UR - https://fis.uni-bamberg.de/handle/uniba/112831 LA - eng AB - Concerts and performances in virtual reality are becoming increasingly popular. Accurately visualizing the detailed hand motions of musicians playing instruments is challenging. In this work, we present a real-time motion synthesis method that generates the detailed fretting hand motion for playing guitar. Our approach first involves capturing and post-processing hand motion data from guitar performances to create a data set for training. The post-processing ensures that the fingertip positions are placed as accurately as possible regarding distance from the fretboard and location between the frets. We then train a neural network that learns to predict hand and finger poses based on guitar tabs and previous poses. We found that our method produces reasonably stable motion and evaluate our results using accuracy measures and visual evaluation. KW - character animation KW - motion synthesis KW - virtual humans KW - hand motions ER - TY - CPAPER TI - Hands or Controllers? : How Input Devices and Audio Impact Collaborative Virtual Reality AU - Adkins, Alex AU - Canales, Ryan AU - Jörg, Sophie PY - 2025 SP - 1-12 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/112309 LA - eng AB - Advancing virtual reality technologies are enabling real-time virtual-face to virtual-face communication. Hand tracking systems that are integrated into Head-Mounted Displays (HMD) enable users to directly interact with their environments and with each other using their hands as opposed to using controllers. Due to the novelties of these technologies our understanding of how they impact our interactions is limited. In this paper, we investigate the consequences of using different interaction control systems, hand tracking or controllers, when interacting with others in a virtual environment. We design and implement NASA’s Survival on the Moon teamwork evaluation exercise in virtual reality (VR) and test for effects with and without allowing verbal communication. We evaluate social presence, perceived comprehension, team cohesion, group synergy, task workload, as well as task performance and duration. Our findings reveal that audio communication significantly enhances social presence, perceived comprehension, and team cohesion, but it also increases effort workload and negatively impacts group synergy. The choice of interaction control systems has limited impact on various aspects of virtual collaboration in this scenario, although participants using hand tracking reported lower effort workload, while participants using controllers reported lower mental workload in the absence of audio. KW - Communication KW - collaboration KW - gestures KW - avatars ER - TY - JOUR TI - Does Hand Size Matter? : The Effect of Avatar Hand Size on Non-verbal Communication in Virtual Reality AU - Henry, Jackson AU - Canales, Ryan AU - Stolarski, Catherine AU - Adkins, Alex AU - Venkatakrishnan, Rohith AU - Venkatakrishnan, Roshan AU - Jörg, Sophie PY - 2025 JO - IEEE transactions on visualization and computer graphics : TVCG VL - 31 IS - 5 SP - 3161-3171 PB - IEEE CY - New York, NY SN - 1077-2626 DO - 10.1109/TVCG.2025.3549894 UR - https://fis.uni-bamberg.de/handle/uniba/112290 LA - eng ER - TY - CPAPER TI - Real-Time Conversational Gaze Synthesis for Avatars AU - Canales, Ryan AU - Jain, Eakta AU - Jörg, Sophie PY - 2025 SP - 1-7 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/110683 LA - eng AB - Eye movement plays an important role in face-to-face communication. In this work, we present a deep learning approach for synthesizing the eye movements of avatars for two-party conversations and evaluate viewer perception of different types of eye motions. We aim to synthesize believable gaze behavior based on head motions and audio features as they would typically be available in virtual reality applications. To this end, we captured the head motion, eye motion, and audio of several two-party conversations and trained an RNN-based model to predict where an avatar looks in a two-person conversational scenario. We evaluated our approach with a user study on the perceived quality of the eye animation and compared our method with other eye animation methods. While our model was not rated highest, our model and our user study lead to a series of insights on model features, viewer perception, and study design that we present. KW - Virtual reality KW - Perception KW - Animation ER - TY - CPAPER TI - The Curation Tree : A Lightweight Behavior Tree Framework for Implementing Puzzle and Narrative Games AU - Normoyle, Aline AU - Jörg, Sophie AU - Hill, Jennifer PY - 2024 T2 - FDG 2024: Proceedings of the 19th International Conference on the Foundations of Digital Games ; Worcester, Massachusetts, USA May 21–24, 2024 IS - 63 PB - Association for Computing Machinery CY - New York, USA DO - 10.1145/3649921.3659840 UR - https://fis.uni-bamberg.de/handle/uniba/112207 LA - eng ER - TY - CPAPER TI - Hands or Controllers? : How Input Devices and Audio Impact Collaborative Virtual Reality AU - Adkins, Alex AU - Canales, Ryan AU - Jörg, Sophie PY - 2024 T2 - VRST '24 : Proceedings of the 30th ACM Symposium on Virtual Reality Software and Technology IS - 31 SP - 1-12 PB - Association for Computing Machinery CY - New York SN - 979-8-4007-0535-9 DO - 10.1145/3641825.3687718 UR - https://fis.uni-bamberg.de/handle/uniba/106267 LA - eng AB - Advancing virtual reality technologies are enabling real-time virtual-face to virtual-face communication. Hand tracking systems that are integrated into Head-Mounted Displays (HMD) enable users to directly interact with their environments and with each other using their hands as opposed to using controllers. Due to the novelties of these technologies our understanding of how they impact our interactions is limited. In this paper, we investigate the consequences of using different interaction control systems, hand tracking or controllers, when interacting with others in a virtual environment. We design and implement NASA’s Survival on the Moon teamwork evaluation exercise in virtual reality (VR) and test for effects with and without allowing verbal communication. We evaluate social presence, perceived comprehension, team cohesion, group synergy, task workload, as well as task performance and duration. Our findings reveal that audio communication significantly enhances social presence, perceived comprehension, and team cohesion, but it also increases effort workload and negatively impacts group synergy. The choice of interaction control systems has limited impact on various aspects of virtual collaboration in this scenario, although participants using hand tracking reported lower effort workload, while participants using controllers reported lower mental workload in the absence of audio. KW - Communication KW - collaboration KW - gestures KW - avatars ER - TY - JOUR TI - Towards mitigating uncann(eye)ness in face swaps via gaze-centric loss terms AU - Wilson, Ethan AU - Shic, Frederick AU - Jörg, Sophie AU - Jain, Eakta PY - 2024 JO - Computers & Graphics VL - 119 IS - April 2024, 103888 PB - Elsevier Science CY - Amsterdam [u.a.] SN - 0097-8493 DO - 10.1016/j.cag.2024.103888 UR - https://fis.uni-bamberg.de/handle/uniba/93647 LA - eng ER - TY - THES TI - Bewegte 3D-Objekte im urbanen Raum als Herausforderung für Augmented-Reality-Systeme AU - Öder, Bernhard PY - 2024 PB - Otto-Friedrich-Universität CY - Bamberg DO - 10.20378/irb-92165 UR - https://fis.uni-bamberg.de/handle/uniba/92165 LA - deu AB - Die Arbeit gibt einen Überblick über verschiedene Technologien, welche im Zusammenspiel die Grundlage für Augmented-Reality-Anwendungen darstellen. Vor diesem Hintergrund wird am Beispiel der ehemaligen Straßenbahn Bambergs gezeigt, dass der komplexe Fall von bewegten 3D-Objekten im urbanen Raum für Augmented-Reality-Systeme eingeschränkt realisierbar ist. Das Modell der Straßenbahn wird dazu mittels Bilderkennung in den Stadtraum projiziert und bewegt sich über vier Streckenabschnitte entlang einer Straße. Die Anwendung läuft auf herkömmlichen Smartphones. Als Hauptherausforderungen wurden die Bilderkennung im urbanen Raum, die präzise Objektplatzierung, die Visualisierungsstabilität der Straßenbahn gegenüber der realen Umwelt sowie das Verdeckungsverhalten erkannt. Als Faktoren, welche die Ausführung und die Visualisierungsqualität beeinflussen, wurden Belichtung, Auswahl des verwendeten Smartphones, das Nutzerverhalten und Verdeckungen erkannt. Aus den genannten Faktoren ergeben sich Einschränkungen bei der Erstellung einer solchen Anwendung, welche berücksichtigt werden müssen, um den Herausforderungen gerecht zu werden. Die Arbeit zeigt, wie die AR-Anwendung im Detail entstanden ist. Für eine einfache und präzise Objektplatzierung kommt Bilderkennung zum Einsatz. Umgesetzt wurde sie unter der Verwendung von Unity und Vuforia. Das Modell der Straßenbahn wurde in Blender erstellt. KW - Augmented Reality KW - Cultural Heritage KW - Welterbe KW - AR KW - Smart City ER - TY - CPAPER TI - Give me some room please! : Personal space bubbles for safety and performance AU - LaRubbio, Karina AU - Wilson, Ethan AU - Koppal, Sanjeev AU - Jörg, Sophie AU - Jain, Eakta PY - 2023 T2 - Proceedings : 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops ; 25 – 29 March 2023, Shanghai, China SP - 897-898 PB - IEEE Computer Society, Conference Publishing Services (CPS) CY - Los Alamitos, California ; Washington ; Tokyo DO - 10.1109/vrw58643.2023.00291 UR - https://fis.uni-bamberg.de/handle/uniba/112559 LA - eng ER - TY - JOUR TI - How important are Detailed Hand Motions for Communication for a Virtual Character Through the Lens of Charades? AU - Adkins, Alex AU - Lin, Lorraine AU - Sun, Yu AU - Ye, Yuting AU - Di Luca, Massimiliano AU - Jörg, Sophie AU - Normoyle, Aline PY - 2023 JO - ACM Transactions on Graphics VL - 42 IS - 3 SP - 1-16 PB - ACM CY - New York, NY [u.a.] SN - 0730-0301 DO - 10.1145/3578575 UR - https://fis.uni-bamberg.de/handle/uniba/106340 LA - eng ER - TY - CPAPER TI - Real-Time Conversational Gaze Synthesis for Avatars AU - Canales, Ryan AU - Jain, Eakta AU - Jörg, Sophie PY - 2023 T2 - MIG '23 : Proceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games IS - 17 SP - 1-7 PB - Association for Computing Machinery CY - New York SN - 979-8-4007-0393-5 DO - 10.1145/3623264.3624446 UR - https://fis.uni-bamberg.de/handle/uniba/106268 LA - eng AB - Eye movement plays an important role in face-to-face communication. In this work, we present a deep learning approach for synthesizing the eye movements of avatars for two-party conversations and evaluate viewer perception of different types of eye motions. We aim to synthesize believable gaze behavior based on head motions and audio features as they would typically be available in virtual reality applications. To this end, we captured the head motion, eye motion, and audio of several two-party conversations and trained an RNN-based model to predict where an avatar looks in a two-person conversational scenario. We evaluated our approach with a user study on the perceived quality of the eye animation and compared our method with other eye animation methods. While our model was not rated highest, our model and our user study lead to a series of insights on model features, viewer perception, and study design that we present. KW - Virtual reality KW - Perception KW - Animation ER - TY - CPAPER TI - Virtual Hands in VR : Motion Capture, Synthesis, and Perception AU - Jörg, Sophie AU - Mueller, Franziska AU - Neff, Michael AU - Ye, Yuting AU - Zordan, Victor PY - 2020 T2 - SIGGRAPH 2020: ACM SIGGRAPH 2020 Courses SP - 1-145 PB - ACM CY - New York SN - 978-1-4503-7972-4 DO - 10.1145/3388769.3407494 UR - https://fis.uni-bamberg.de/handle/uniba/106343 LA - eng ER - TY - CPAPER TI - Virtual Grasping Feedback and Virtual Hand Ownership AU - Canales, Ryan AU - Sun, Yu AU - Ye, Yuting AU - Jörg, Sophie AU - Di Luca, Massimiliano AU - Normoyle, Aline PY - 2019 T2 - SAP '19: ACM Symposium on Applied Perception 2019 SN - 978-1-4503-6890-2 DO - 10.1145/3343036.3343132 UR - https://fis.uni-bamberg.de/handle/uniba/106345 LA - eng ER - TY - CPAPER TI - The Effect of Hand Size and Interaction Modality on the Virtual Hand Illusion AU - Normoyle, Aline AU - Adkins, Alex AU - Sun, Yun AU - Robb, Andrew AU - Ye, Yuting AU - Di Luca, Massimiliano AU - Jörg, Sophie AU - Lin, Lorraine PY - 2019 T2 - 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) PB - IEEE DO - 10.1109/VR.2019.8797787 UR - https://fis.uni-bamberg.de/handle/uniba/106346 LA - eng ER - TY - CPAPER TI - Eye Movement Synthesis with 1/f Pink Noise AU - Duchowski, Andrew AU - Jörg, Sophie AU - Lawson, Aubrey AU - Bolte, Takumi AU - Swirski, Lech AU - Krejtz, Krzysztof PY - 2015 T2 - MIG '15: Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games SP - 47 - 56 PB - ACM CY - New York SN - 978-1-4503-3991-9 DO - 10.1145/2822013.2822014 UR - https://fis.uni-bamberg.de/handle/uniba/106353 LA - eng ER - TY - CPAPER TI - Dancing Alice : Exploring Embodied Pedagogical Strategies for Learning Computational Thinking AU - Daily, Shaundra Bryant AU - Jörg, Sophie AU - Gundersen, Kara AU - Babu, Sabarish AU - Leonard, Alison E PY - 2014 T2 - SIGCSE '14: Proceedings of the 45th ACM technical symposium on Computer science education SP - 91 - 96 PB - ACM CY - New York SN - 978-1-4503-2605-6 DO - 10.1145/2538862.2538917 UR - https://fis.uni-bamberg.de/handle/uniba/106358 LA - eng ER -