Skip to content

Read-only version: For security reasons, the Research Information System (FIS) has been temporarily replaced by this read-only version. All public content and full texts remain available; login, editing, submission and statistics are not available. Questions: fis@uni-bamberg.de

Home › Publications › Integrating Domain Knowledge for Robust and Interpretable Deep Neural Networks in Facial Affect Recognition

Publication

Integrating Domain Knowledge for Robust and Interpretable Deep Neural Networks in Facial Affect Recognition

Rieger, Ines Sophia (2026): Integrating Domain Knowledge for Robust and Interpretable Deep Neural Networks in Facial Affect Recognition, Bamberg: Otto-Friedrich-Universität, doi: 10.20378/irb-116940.

Details

Faculty/Chair
Author
Publisher Information
Otto-Friedrich-Universität : Bamberg
Year of publication
2026
Pages
XXII, 109, Tabellen, Diagramme
Supervisor
Language
English
Remark

Kumulative Dissertation, Otto-Friedrich-Universität Bamberg, 2026

DOI
Licence
URN
Abstract

Facial affect recognition is a key component of human-centered AI, enabling systems to respond appropriately to human nonverbal signals. The deployment of deep learning for facial affect recognition is critically hindered by vulnerability to data-driven biases and a lack of transparency. This thesis addresses these challenges through a multi-faceted framework, structured along the CRISP-ML(Q) machine learning lifecycle, integrating domain knowledge to build more robust and interpretable systems.

First, it quantifies demographic bias through extensive subject-level metadata annotation, uncovering performance gaps across demographic groups in leading facial expression recognition (FER) classifiers. To mitigate demographic and multi-label class imbalance, an optimization-based data balancing technique, MLOPT, is introduced to improve performance, fairness, and generalization. Second, the training of deep learning models is guided by injecting anatomical knowledge about facial Action Units (AUs), that is, localized facial muscle activations defined by the Facial Action Coding System (FACS). This is achieved through the CorrLoss regularization term and a Facial Movement Constraint Network (FMC-Net), which together improve model robustness and predictive performance. Third, interpretability is enhanced through a fuzzy-logic ensemble that combines expert rules with model predictions. This dual approach enables self-validation of the models by flagging uncertain outputs while providing inherently interpretable behavior through logical rules. Finally, a new evaluation framework, RelEx, is proposed to assess the physiological plausibility of trained models by quantifying the alignment between feature attributions and expert-defined facial regions.

By addressing these challenges with targeted solutions, this research demonstrates that systematically integrating domain knowledge moves FER beyond performance optimization. The resulting framework provides a foundation for developing AI systems that are not only high-performing but also robust and interpretable.

GND Keywords
Mustererkennung
Künstliche Intelligenz
Neuronales Netz
Interoperabilität
Keywords
Facial Affect Recognition
Artificial Intelligence
Deep Neural Networks
Robustness
Interpretability
Domain Knowledge
DDC Classification
004 Computer science
RVK Classification
ST 302
Type
Doctoralthesis
URI
Activation date
2026-09-22

Permalink: https://fis.uni-bamberg.de/handle/uniba/116940 · Data as JSON · Cite: BibTeX · RIS

Data as of: 2026-11-16

Restricted read-only access: login, editing and submission are temporarily disabled.