TY - JOUR TI - Toward trustworthy AI with integrative explainable AI frameworks AU - Finzel, Bettina PY - 2026 SP - 20-45 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/116862 LA - eng AB - As artificial intelligence (AI) increasingly permeates high-stakes domains such as healthcare, transportation, and law enforcement, ensuring its trustworthiness has become a critical challenge. This article proposes an integrative Explainable AI (XAI) framework to address the challenges of interpretability, explainability, interactivity, and robustness. By combining XAI methods, incorporating human-AI interaction and using suitable evaluation techniques, the implementation of this framework serves as a holistic XAI approach. The article discusses the framework’s contribution to trustworthy AI and gives an outlook on open challenges related to interdisciplinary collaboration, AI generalization and AI evaluation. KW - trustworthy AI KW - explainable AI KW - EU AI act KW - integrative XAI frameworks KW - XAI in medicine ER - TY - JOUR TI - Toward human-level concept learning : Pattern benchmarking for AI algorithms AU - Holzinger, Andreas AU - Saranti, Anna AU - Angerschmid, Alessa AU - Finzel, Bettina AU - Schmid, Ute AU - Mueller, Heimo PY - 2026 SP - 1-21 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/114970 LA - eng AB - Artificial intelligence (AI) today is very successful at standard pattern-recognition tasks due to the availability of large amounts of data and advances in statistical data-driven machine learning. However, there is still a large gap between AI pattern recognition and human-level concept learning. Humans can learn amazingly well even under uncertainty from just a few examples and are capable of generalizing these concepts to solve new conceptual problems. The growing interest in explainable machine intelligence requires experimental environments and diagnostic/benchmark datasets to analyze existing approaches and drive progress in pattern analysis and machine intelligence. In this paper, we provide an overview of current AI solutions for benchmarking concept learning, reasoning, and generalization; discuss the state-of-the-art of existing diagnostic/ benchmark datasets (such as CLEVR, CLEVRER, CLOSURE, CURI, Bongard-LOGO, V-PROM, RAVEN, Kandinsky Patterns, CLEVR-Humans, CLEVRER-Humans, and their extension containing human language); and provide an outlook of some future research directions in this exciting research domain. ER - TY - CPAPER TI - A Framework for Context-aware XAI in Human-AI Collaboration AU - Knoblach, Judith AU - Finzel, Bettina AU - Schmid, Ute PY - 2026 T2 - Contextualizing Explanations : Proceedings of the 3rd TRR 318 Conference SP - 24 PB - Bielefeld University Press CY - Bielefeld DO - 10.64136/deef2831 UR - https://fis.uni-bamberg.de/handle/uniba/114752 LA - eng ER - TY - JOUR TI - Toward trustworthy AI with integrative explainable AI frameworks AU - Finzel, Bettina PY - 2025 JO - Information technology : it VL - 67 IS - 1 SP - 20-45 PB - De Gruyter CY - Berlin SN - 1611-2776 DO - 10.1515/itit-2025-0007 UR - https://fis.uni-bamberg.de/handle/uniba/116722 LA - eng AB - As artificial intelligence (AI) increasingly permeates high-stakes domains such as healthcare, transportation, and law enforcement, ensuring its trustworthiness has become a critical challenge. This article proposes an integrative Explainable AI (XAI) framework to address the challenges of interpretability, explainability, interactivity, and robustness. By combining XAI methods, incorporating human-AI interaction and using suitable evaluation techniques, the implementation of this framework serves as a holistic XAI approach. The article discusses the framework’s contribution to trustworthy AI and gives an outlook on open challenges related to interdisciplinary collaboration, AI generalization and AI evaluation. KW - trustworthy AI KW - explainable AI KW - EU AI act KW - integrative XAI frameworks KW - XAI in medicine ER - TY - CPAPER TI - Toward Simple and Robust Contrastive Explanations for Image Classification by Leveraging Instance Similarity and Concept Relevance AU - Kaidashova, Yuliia AU - Finzel, Bettina AU - Schmid, Ute Maria ED - Springer Nature Switzerland PY - 2025 T2 - KI 2025: advances in artificial intelligence : 48th German Conference on AI, Potsdam, Germany, September 16-19, 2025, proceedings SP - 32-46 PB - Springer Nature Switzerland CY - Cham SN - 978-3-032-02812-9 DO - 10.1007/978-3-032-02813-6_3 UR - https://fis.uni-bamberg.de/handle/uniba/113752 LA - eng ER - TY - JOUR TI - Current methods in explainable artificial intelligence and future prospects for integrative physiology AU - Finzel, Bettina PY - 2025 SP - 513-529 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/107762 LA - eng AB - Explainable artificial intelligence (XAI) is gaining importance in physiological research, where artificial intelligence is now used as an analytical and predictive tool for many medical research questions. The primary goal of XAI is to make AI models understandable for human decision-makers. This can be achieved in particular through providing inherently interpretable AI methods or by making opaque models and their outputs transparent using post hoc explanations. This review introduces XAI core topics and provides a selective overview of current XAI methods in physiology. It further illustrates solved and discusses open challenges in XAI research using existing practical examples from the medical field. The article gives an outlook on two possible future prospects: (1) using XAI methods to provide trustworthy AI for integrative physiological research and (2) integrating physiological expertise about human explanation into XAI method development for useful and beneficial human-AI partnerships. KW - Explainable Artificial Intelligence (XAI) KW - Physiology KW - Explainability KW - Interpretability KW - Survey ER - TY - JOUR TI - Current methods in explainable artificial intelligence and future prospects for integrative physiology AU - Finzel, Bettina PY - 2025 JO - Pflügers Archiv : European journal of physiology VL - 477 IS - 4 SP - 513-529 PB - Springer CY - Berlin ; Heidelberg SN - 1432-2013 DO - 10.1007/s00424-025-03067-7 UR - https://fis.uni-bamberg.de/handle/uniba/107758 LA - eng AB - Explainable artificial intelligence (XAI) is gaining importance in physiological research, where artificial intelligence is now used as an analytical and predictive tool for many medical research questions. The primary goal of XAI is to make AI models understandable for human decision-makers. This can be achieved in particular through providing inherently interpretable AI methods or by making opaque models and their outputs transparent using post hoc explanations. This review introduces XAI core topics and provides a selective overview of current XAI methods in physiology. It further illustrates solved and discusses open challenges in XAI research using existing practical examples from the medical field. The article gives an outlook on two possible future prospects: (1) using XAI methods to provide trustworthy AI for integrative physiological research and (2) integrating physiological expertise about human explanation into XAI method development for useful and beneficial human-AI partnerships. KW - Explainable Artificial Intelligence (XAI) KW - Physiology KW - Explainability KW - Interpretability KW - Survey ER - TY - JOUR TI - Human-Centered Explanations : Lessons Learned from Image Classification for Medical and Clinical Decision Making AU - Finzel, Bettina PY - 2025 SP - 157-167 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/106099 LA - eng AB - To date, there is no universal explanatory method for making decisions of an AI-based system transparent to human decision makers. This is because, depending on the application domain, data modality, and classification model, the requirements for the expressiveness of explanations vary. Explainees, whether experts or novices (e.g., in medical and clinical diagnosis) or developers, have different information needs. To address the explanation gap, we motivate human-centered explanations and demonstrate the need for combined and expressive approaches based on two image classification use cases: digital pathology and clinical pain detection using facial expressions. Various explanatory approaches that have emerged or been applied in the three-year research project “Transparent Medical Expert Companion” are shortly reviewed and categorized in expressiveness according to their modality and scope. Their suitability for different contexts of explanation is assessed with regard to the explainees’ need for information. The article highlights open challenges and suggests future directions for integrative explanation frameworks. KW - Explainable Artificial Intelligence KW - Digital healthcare KW - Human-centered explanations KW - Explanation expressiveness KW - Explanation frameworks ER - TY - CPAPER TI - Near Hit and Near Miss Example Explanations for Model Revision in Binary Image Classification AU - Finzel, Bettina AU - Knoblach, Judith AU - Thaler, Anna Magdalena AU - Schmid, Ute PY - 2024 T2 - Intelligent Data Engineering and Automated Learning – IDEAL 2024 : 25th International Conference, Valencia, Spain, November 20–22, 2024, Proceedings, Part II SP - 260-271 PB - Springer Nature Switzerland CY - Cham SN - 9783031777370 DO - 10.1007/978-3-031-77738-7_22 UR - https://fis.uni-bamberg.de/handle/uniba/106515 LA - eng ER - TY - GEN TI - When a Relation Tells More Than a Concept : Exploring and Evaluating Classifier Decisions with CoReX AU - Finzel, Bettina AU - Hilme, Patrick AU - Rabold, Johannes AU - Schmid, Ute PY - 2024 SP - 1-56 PB - arXiv DO - 10.48550/arxiv.2405.01661 UR - https://fis.uni-bamberg.de/handle/uniba/106516 LA - eng ER - TY - CPAPER TI - Explaining with Attribute-Based and Relational Near Misses : An Interpretable Approach to Distinguishing Facial Expressions of Pain and Disgust AU - Finzel, Bettina AU - Kuhn, Simon P. AU - Tafler, David E. AU - Schmid, Ute Maria PY - 2024 T2 - Inductive Logic Programming : 31st International Conference, ILP 2022, Windsor Great Park, UK, September 28–30, 2022, Proceedings SP - 40-51 PB - Springer Nature Switzerland CY - Cham SN - 9783031556296 DO - 10.1007/978-3-031-55630-2_4 UR - https://fis.uni-bamberg.de/handle/uniba/106518 LA - eng ER - TY - JOUR TI - Human-Centered Explanations : Lessons Learned from Image Classification for Medical and Clinical Decision Making AU - Finzel, Bettina PY - 2024 JO - Künstliche Intelligenz : KI ; Forschung, Entwicklung, Erfahrungen VL - 38 IS - 3 SP - 157-167 PB - Springer CY - Berlin SN - 1610-1987 DO - 10.1007/s13218-024-00835-y UR - https://fis.uni-bamberg.de/handle/uniba/106097 LA - eng AB - To date, there is no universal explanatory method for making decisions of an AI-based system transparent to human decision makers. This is because, depending on the application domain, data modality, and classification model, the requirements for the expressiveness of explanations vary. Explainees, whether experts or novices (e.g., in medical and clinical diagnosis) or developers, have different information needs. To address the explanation gap, we motivate human-centered explanations and demonstrate the need for combined and expressive approaches based on two image classification use cases: digital pathology and clinical pain detection using facial expressions. Various explanatory approaches that have emerged or been applied in the three-year research project “Transparent Medical Expert Companion” are shortly reviewed and categorized in expressiveness according to their modality and scope. Their suitability for different contexts of explanation is assessed with regard to the explainees’ need for information. The article highlights open challenges and suggests future directions for integrative explanation frameworks. KW - Explainable Artificial Intelligence KW - Digital healthcare KW - Human-centered explanations KW - Explanation expressiveness KW - Explanation frameworks ER - TY - JOUR TI - A comprehensive taxonomy for explainable artificial intelligence : a systematic survey of surveys on methods and concepts AU - Schwalbe, Gesina AU - Finzel, Bettina PY - 2024 SP - 3043-3101 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/98420 LA - eng AB - In the meantime, a wide variety of terminologies, motivations, approaches, and evaluation criteria have been developed within the research field of explainable artificial intelligence (XAI). With the amount of XAI methods vastly growing, a taxonomy of methods is needed by researchers as well as practitioners: To grasp the breadth of the topic, compare methods, and to select the right XAI method based on traits required by a specific use-case context. Many taxonomies for XAI methods of varying level of detail and depth can be found in the literature. While they often have a different focus, they also exhibit many points of overlap. This paper unifies these efforts and provides a complete taxonomy of XAI methods with respect to notions present in the current state of research. In a structured literature analysis and meta-study, we identified and reviewed more than 50 of the most cited and current surveys on XAI methods, metrics, and method traits. After summarizing them in a survey of surveys, we merge terminologies and concepts of the articles into a unified structured taxonomy. Single concepts therein are illustrated by more than 50 diverse selected example methods in total, which we categorize accordingly. The taxonomy may serve both beginners, researchers, and practitioners as a reference and wide-ranging overview of XAI method traits and aspects. Hence, it provides foundations for targeted, use-case-oriented, and context-sensitive future research. KW - Explainable artificial intelligence KW - Interpretability KW - Taxonomy KW - Meta-analysis KW - Survey-of-surveys KW - Review ER - TY - CPAPER TI - Domain-Specific Evaluation of Visual Explanations for Application-Grounded Facial Expression Recognition AU - Finzel, Bettina AU - Rieger, Ines AU - Kuhn, Simon AU - Schmid, Ute PY - 2024 SP - 31-44 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/95294 LA - eng AB - Research in the field of explainable artificial intelligence has produced a vast amount of visual explanation methods for deep learning-based image classification in various domains of application. However, there is still a lack of domain-specific evaluation methods to assess an explanation’s quality and a classifier’s performance with respect to domain-specific requirements. In particular, evaluation methods could benefit from integrating human expertise into quality criteria and metrics. Such domain-specific evaluation methods can help to assess the robustness of deep learning models more precisely. In this paper, we present an approach for domain-specific evaluation of visual explanation methods in order to enhance the transparency of deep learning models and estimate their robustness accordingly. As an example use case, we apply our framework to facial expression recognition. We can show that the domain-specific evaluation is especially beneficial for challenging use cases such as facial expression recognition and provides application-grounded quality criteria that are not covered by standard evaluation methods. Our comparison of the domain-specific evaluation method with standard approaches thus shows that the quality of the expert knowledge is of great importance for assessing a model’s performance precisely. KW - Application-Grounded Facial Expression Recognition ER - TY - JOUR TI - A comprehensive taxonomy for explainable artificial intelligence : a systematic survey of surveys on methods and concepts AU - Schwalbe, Gesina AU - Finzel, Bettina PY - 2024 JO - Data mining and knowledge discovery VL - 38 IS - 5 SP - 3043-3101 PB - Springer Science + Business Media B.V CY - Dordrecht [u.a.] SN - 1384-5810 DO - 10.1007/s10618-022-00867-8 UR - https://fis.uni-bamberg.de/handle/uniba/94996 LA - eng AB - In the meantime, a wide variety of terminologies, motivations, approaches, and evaluation criteria have been developed within the research field of explainable artificial intelligence (XAI). With the amount of XAI methods vastly growing, a taxonomy of methods is needed by researchers as well as practitioners: To grasp the breadth of the topic, compare methods, and to select the right XAI method based on traits required by a specific use-case context. Many taxonomies for XAI methods of varying level of detail and depth can be found in the literature. While they often have a different focus, they also exhibit many points of overlap. This paper unifies these efforts and provides a complete taxonomy of XAI methods with respect to notions present in the current state of research. In a structured literature analysis and meta-study, we identified and reviewed more than 50 of the most cited and current surveys on XAI methods, metrics, and method traits. After summarizing them in a survey of surveys, we merge terminologies and concepts of the articles into a unified structured taxonomy. Single concepts therein are illustrated by more than 50 diverse selected example methods in total, which we categorize accordingly. The taxonomy may serve both beginners, researchers, and practitioners as a reference and wide-ranging overview of XAI method traits and aspects. Hence, it provides foundations for targeted, use-case-oriented, and context-sensitive future research. KW - Explainable artificial intelligence KW - Interpretability KW - Taxonomy KW - Meta-analysis KW - Survey-of-surveys KW - Review ER - TY - JOUR TI - Explaining and Evaluating Deep Tissue Classification by Visualizing Activations of Most Relevant Intermediate Layers AU - Mohammed, Aliya AU - Geppert, Carol AU - Hartmann, Arnd AU - Kuritcyn, Petr AU - Bruns, Volker AU - Schmid, Ute AU - Wittenberg, Thomas AU - Benz, Michaela AU - Finzel, Bettina PY - 2024 SP - 229-232 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/93007 LA - eng AB - Deep Learning-based tissue classification may support pathologists in analyzing digitized whole slide images. However, in such critical tasks, only approaches that can be validated by medical experts in advance to deployment, are suitable. We present an approach that contributes to making automated tissue classification more transparent. We step beyond broadly used visualizations for last layers of a convolutional neural network by identifying most relevant intermediate layers applying Grad-CAM. A visual evaluation by a pathologist shows that these layers assign relevance, where important morphological structures are present in case of correct class decisions. We introduce a tool that can be easily used by medical experts for such validation purposes for any convolutional neural network and any layer. Visual explanations for intermediate layers provide insights into a neural network’s decision for histopathological tissue classification. In future research also the context of the input data must be considered. KW - Digital Pathology KW - Deep Learning KW - XAI KW - Evaluation KW - Grad-CAM KW - Intelligent user interface ER - TY - JOUR TI - Toward human-level concept learning : Pattern benchmarking for AI algorithms AU - Holzinger, Andreas AU - Saranti, Anna AU - Angerschmid, Alessa AU - Finzel, Bettina AU - Schmid, Ute AU - Mueller, Heimo PY - 2023 JO - Patterns VL - 4 IS - 8 SP - 1-21 PB - Elsevier CY - Amsterdam SN - 2666-3899 DO - 10.1016/j.patter.2023.100788 UR - https://fis.uni-bamberg.de/handle/uniba/106517 LA - eng AB - Artificial intelligence (AI) today is very successful at standard pattern-recognition tasks due to the availability of large amounts of data and advances in statistical data-driven machine learning. However, there is still a large gap between AI pattern recognition and human-level concept learning. Humans can learn amazingly well even under uncertainty from just a few examples and are capable of generalizing these concepts to solve new conceptual problems. The growing interest in explainable machine intelligence requires experimental environments and diagnostic/benchmark datasets to analyze existing approaches and drive progress in pattern analysis and machine intelligence. In this paper, we provide an overview of current AI solutions for benchmarking concept learning, reasoning, and generalization; discuss the state-of-the-art of existing diagnostic/ benchmark datasets (such as CLEVR, CLEVRER, CLOSURE, CURI, Bongard-LOGO, V-PROM, RAVEN, Kandinsky Patterns, CLEVR-Humans, CLEVRER-Humans, and their extension containing human language); and provide an outlook of some future research directions in this exciting research domain. ER - TY - JOUR TI - Generating Explanations for Conceptual Validation of Graph Neural Networks : An Investigation of Symbolic Predicates Learned on Relevance-Ranked Sub-Graphs AU - Finzel, Bettina AU - Saranti, Anna AU - Angerschmid, Alessa AU - Tafler, David AU - Pfeifer, Bastian AU - Holzinger, Andreas PY - 2023 SP - 271–285 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/91640 LA - eng AB - Graph Neural Networks (GNN) show good performance in relational data classification. However, their contribution to concept learning and the validation of their output from an application domain’s and user’s perspective have not been thoroughly studied. We argue that combining symbolic learning methods, such as Inductive Logic Programming (ILP), with statistical machine learning methods, especially GNNs, is an essential forward-looking step to perform powerful and validatable relational concept learning. In this contribution, we introduce a benchmark for the conceptual validation of GNN classification outputs. It consists of the symbolic representations of symmetric and non-symmetric figures that are taken from a well-known Kandinsky Pattern data set. We further provide a novel validation framework that can be used to generate comprehensible explanations with ILP on top of the relevance output of GNN explainers and human-expected relevance for concepts learned by GNNs. Our experiments conducted on our benchmark data set demonstrate that it is possible to extract symbolic concepts from the most relevant explanations that are representative of what a GNN has learned. Our findings open up a variety of avenues for future research on validatable explanations for GNNs. KW - Graph neural networks (GNN) KW - Explainable AI (xAI) KW - Inductive logic programming (ILP) KW - Symbolic AI KW - Kandinsky pattern (KP) ER - TY - CPAPER TI - Domain-Specific Evaluation of Visual Explanations for Application-Grounded Facial Expression Recognition AU - Finzel, Bettina AU - Rieger, Ines AU - Kuhn, Simon AU - Schmid, Ute Maria PY - 2023 T2 - Machine learning and knowledge extraction : 7th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2023, Benevento, Italy, August 29 – September 1, 2023, proceedings SP - 31-44 PB - Springer Nature Switzerland CY - Cham, Switzerland SN - 978-3-031-40836-6 DO - 10.1007/978-3-031-40837-3_3 UR - https://fis.uni-bamberg.de/handle/uniba/93116 LA - eng AB - Research in the field of explainable artificial intelligence has produced a vast amount of visual explanation methods for deep learning-based image classification in various domains of application. However, there is still a lack of domain-specific evaluation methods to assess an explanation’s quality and a classifier’s performance with respect to domain-specific requirements. In particular, evaluation methods could benefit from integrating human expertise into quality criteria and metrics. Such domain-specific evaluation methods can help to assess the robustness of deep learning models more precisely. In this paper, we present an approach for domain-specific evaluation of visual explanation methods in order to enhance the transparency of deep learning models and estimate their robustness accordingly. As an example use case, we apply our framework to facial expression recognition. We can show that the domain-specific evaluation is especially beneficial for challenging use cases such as facial expression recognition and provides application-grounded quality criteria that are not covered by standard evaluation methods. Our comparison of the domain-specific evaluation method with standard approaches thus shows that the quality of the expert knowledge is of great importance for assessing a model’s performance precisely. KW - Application-Grounded Facial Expression Recognition ER - TY - JOUR TI - Explaining and Evaluating Deep Tissue Classification by Visualizing Activations of Most Relevant Intermediate Layers AU - Mohammed, Aliya AU - Geppert, Carol AU - Hartmann, Arnd AU - Kuritcyn, Petr AU - Bruns, Volker AU - Schmid, Ute AU - Wittenberg, Thomas AU - Benz, Michaela AU - Finzel, Bettina PY - 2022 JO - Current directions in biomedical engineering VL - 8 IS - 2 SP - 229-232 PB - De Gruyter CY - Berlin SN - 2364-5504 DO - 10.1515/cdbme-2022-1059 UR - https://fis.uni-bamberg.de/handle/uniba/93000 LA - eng AB - Deep Learning-based tissue classification may support pathologists in analyzing digitized whole slide images. However, in such critical tasks, only approaches that can be validated by medical experts in advance to deployment, are suitable. We present an approach that contributes to making automated tissue classification more transparent. We step beyond broadly used visualizations for last layers of a convolutional neural network by identifying most relevant intermediate layers applying Grad-CAM. A visual evaluation by a pathologist shows that these layers assign relevance, where important morphological structures are present in case of correct class decisions. We introduce a tool that can be easily used by medical experts for such validation purposes for any convolutional neural network and any layer. Visual explanations for intermediate layers provide insights into a neural network’s decision for histopathological tissue classification. In future research also the context of the input data must be considered. KW - Digital Pathology KW - Deep Learning KW - XAI KW - Evaluation KW - Grad-CAM KW - Intelligent user interface ER - TY - JOUR TI - Generating Explanations for Conceptual Validation of Graph Neural Networks : An Investigation of Symbolic Predicates Learned on Relevance-Ranked Sub-Graphs AU - Finzel, Bettina AU - Saranti, Anna AU - Angerschmid, Alessa AU - Tafler, David AU - Pfeifer, Bastian AU - Holzinger, Andreas PY - 2022 JO - Künstliche Intelligenz : KI ; Forschung, Entwicklung, Erfahrungen ; Organ des Fachbereichs 1 Künstliche Intelligenz der Gesellschaft für Informatik e.V., GI VL - 36 IS - 3-4 SP - 271-285 PB - Springer CY - Berlin SN - 0933-1875 DO - 10.1007/s13218-022-00781-7 UR - https://fis.uni-bamberg.de/handle/uniba/91634 LA - eng AB - Graph Neural Networks (GNN) show good performance in relational data classification. However, their contribution to concept learning and the validation of their output from an application domain’s and user’s perspective have not been thoroughly studied. We argue that combining symbolic learning methods, such as Inductive Logic Programming (ILP), with statistical machine learning methods, especially GNNs, is an essential forward-looking step to perform powerful and validatable relational concept learning. In this contribution, we introduce a benchmark for the conceptual validation of GNN classification outputs. It consists of the symbolic representations of symmetric and non-symmetric figures that are taken from a well-known Kandinsky Pattern data set. We further provide a novel validation framework that can be used to generate comprehensible explanations with ILP on top of the relevance output of GNN explainers and human-expected relevance for concepts learned by GNNs. Our experiments conducted on our benchmark data set demonstrate that it is possible to extract symbolic concepts from the most relevant explanations that are representative of what a GNN has learned. Our findings open up a variety of avenues for future research on validatable explanations for GNNs. KW - Graph neural networks (GNN) KW - Explainable AI (xAI) KW - Inductive logic programming (ILP) KW - Symbolic AI KW - Kandinsky pattern (KP) ER - TY - CPAPER TI - CorrLoss : Integrating Co-Occurrence Domain Knowledge for Affect Recognition AU - Rieger, Ines AU - Pahl, Jaspar AU - Finzel, Bettina AU - Schmid, Ute PY - 2022 T2 - 2022 26th International Conference on Pattern Recognition (ICPR): Proceedings SP - 798-804 PB - IEEE SN - 978-1-6654-9062-7 DO - 10.1109/ICPR56361.2022.9956319 UR - https://fis.uni-bamberg.de/handle/uniba/58025 LA - eng ER - TY - JOUR TI - Mutual Explanations for Cooperative Decision Making in Medicine AU - Schmid, Ute AU - Finzel, Bettina PY - 2022 SP - 227-233 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/55221 LA - eng AB - Exploiting mutual explanations for interactive learning is presented as part of an interdisciplinary research project on transparent machine learning for medical decision support. Focus of the project is to combine deep learning black box approaches with interpretable machine learning for classification of different types of medical images to combine the predictive accuracy of deep learning and the transparency and comprehensibility of interpretable models. Specifically, we present an extension of the Inductive Logic Programming system Aleph to allow for interactive learning. Medical experts can ask for verbal explanations. They can correct classification decisions and in addition can also correct the explanations. Thereby, expert knowledge can be taken into account in form of constraints for model adaption. KW - Human-AI partnership KW - Inductive Logic Programming KW - Explanations as constraints ER - TY - CPAPER TI - Deriving Temporal Prototypes from Saliency Map Clusters for the Analysis of Deep-Learning-based Facial Action Unit Classification AU - Finzel, Bettina AU - Kollmann, Rene AU - Rieger, Ines AU - Pahl, Jaspar AU - Schmid, Ute PY - 2022 SP - 86-97 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/55380 LA - eng AB - Reliably determining the emotional state of a person is a difficult task for both humans as well as machines. Automatic detection and evaluation of facial expressions is particularly important if people are unable to express their emotional state themselves, for example due to cognitive impairments. Identifying the presence of Action Units in a human’s face is a psychologically validated approach of quantifying which emotion is expressed. To automate the detection process of Action Units Neural Networks have been trained. However, the black-box nature of Deep Neural Networks provides no insight on the relevant features identified during the decision process. Approaches of Explainable Artificial Intelligence have to be applied to provide an explanation why the network came to a certain conclusion. In this work "Layer-Wise Relevance Propagation" (LRP) in combination with the meta analysis approach "Spectral Relevance Analysis" (SpRAy) is used to derive temporal prototypes from predictions in video sequences. Temporal prototypes provide an aggregated view on the prediction of the network by grouping together similar frames by considering relevance. Additionally, a specific visualization method for temporal prototypes is presented that highlights the most relevant areas for a prediction of an Action Unit. A quantitative evaluation of our approach shows that temporal prototypes aggregate temporal information well. The proposed method can be used to generate concise visual explanations for a sequence of interpretable saliency maps. Based on the above, this work shall provide the foundation for a new temporal analysis method as well as an explanation approach that is supposed to help researchers and experts to gain a deeper understanding of how the underlying network decides which Action Units are active in a particular emotional state. KW - Temporal Prototypes KW - Facial Action Unit Detection KW - Layer-wise Relevance Propagation KW - Spectral Clustering KW - Affective Computing KW - Correlation Analysis ER - TY - CPAPER TI - Explanation as a Process : User-Centric Construction of Multi-level and Multi-modal Explanations AU - Finzel, Bettina AU - Tafler, David E. AU - Scheele, Stephan M. AU - Schmid, Ute PY - 2021 T2 - KI 2021: Advances in Artificial Intelligence : 44th German Conference on AI, Virtual Event, September 27 – October 1, 2021, Proceedings SP - 80–94 PB - Springer International Publishing CY - Cham T3 - Lecture Notes in Computer Science SN - 978-3-030-87626-5 DO - 10.1007/978-3-030-87626-5_7 UR - https://fis.uni-bamberg.de/handle/uniba/111166 LA - eng ER - TY - CPAPER TI - Uncovering the Bias in Facial Expressions AU - Deuschel, Jessica AU - Finzel, Bettina AU - Rieger, Ines PY - 2021 T2 - Kolloquium Forschende Frauen 2020 - Gender in Gesellschaft 4.0 : Beiträge Bamberger Nachwuchswissenschaftlerinnen SP - 15-42 PB - University of Bamberg Press CY - Bamberg SN - 978-3-86309-853-7 DO - 10.20378/irb-90482 UR - https://fis.uni-bamberg.de/handle/uniba/90482 LA - eng AB - Over the past decades the machine and deep learning community has celebrated great achievements in challenging tasks such as image classification. The deep architecture of artificial neural networks together with the plenitude of available data makes it possible to describe highly complex relations. Yet, it is still impossible to fully capture what the deep learning model has learned and to verify that it operates fairly and without creating bias, especially in critical tasks, for instance those arising in the medical field. One example for such a task is the detection of distinct facial expressions, called Action Units, in facial images. Considering this specific task, our research aims to provide transparency regarding bias, specifically in relation to gender and skin color. We train a neural network for Action Unit classification and analyze its performance quantitatively based on its accuracy and qualitatively based on heatmaps. A structured review of our results indicates that we are able to detect bias. Even though we cannot conclude from our results that lower classification performance emerged solely from gender and skin color bias, these biases must be addressed, which is why we end by giving suggestions on how the detected bias can be avoided. KW - - ER - TY - CPAPER TI - Deriving Temporal Prototypes from Saliency Map Clusters for the Analysis of Deep-Learning-based Facial Action Unit Classification AU - Finzel, Bettina AU - Kollmann, Rene AU - Rieger, Ines AU - Pahl, Jaspar AU - Schmid, Ute PY - 2021 T2 - Proceedings of the LWDA 2021 Workshops: FGWM, KDML, FGWI-BIA, and FGIR SP - 86-97 PB - RWTH Aachen CY - Aachen, Germany UR - https://fis.uni-bamberg.de/handle/uniba/55037 LA - eng AB - Reliably determining the emotional state of a person is a difficult task for both humans as well as machines. Automatic detection and evaluation of facial expressions is particularly important if people are unable to express their emotional state themselves, for example due to cognitive impairments. Identifying the presence of Action Units in a human’s face is a psychologically validated approach of quantifying which emotion is expressed. To automate the detection process of Action Units Neural Networks have been trained. However, the black-box nature of Deep Neural Networks provides no insight on the relevant features identified during the decision process. Approaches of Explainable Artificial Intelligence have to be applied to provide an explanation why the network came to a certain conclusion. In this work "Layer-Wise Relevance Propagation" (LRP) in combination with the meta analysis approach "Spectral Relevance Analysis" (SpRAy) is used to derive temporal prototypes from predictions in video sequences. Temporal prototypes provide an aggregated view on the prediction of the network by grouping together similar frames by considering relevance. Additionally, a specific visualization method for temporal prototypes is presented that highlights the most relevant areas for a prediction of an Action Unit. A quantitative evaluation of our approach shows that temporal prototypes aggregate temporal information well. The proposed method can be used to generate concise visual explanations for a sequence of interpretable saliency maps. Based on the above, this work shall provide the foundation for a new temporal analysis method as well as an explanation approach that is supposed to help researchers and experts to gain a deeper understanding of how the underlying network decides which Action Units are active in a particular emotional state. KW - Temporal Prototypes KW - Facial Action Unit Detection KW - Layer-wise Relevance Propagation KW - Spectral Clustering KW - Affective Computing KW - Correlation Analysis ER - TY - GEN TI - Uni.vers Forschung : das Magazin der Otto-Friedrich-Universität Bamberg AU - Achter, Patricia AU - Braches-Chyrek, Rita AU - Düchs, Martin AU - Färber, Tanja AU - Finzel, Bettina AU - Ganesch, Franziska AU - Graichen, Joanna AU - Horn, Matthias AU - Oehler, Andreas AU - Posegga, Oliver AU - Proaño Acosta, Christian AU - Scheltjens, Werner AU - Schmid, Ute AU - Schneider, Pauline AU - Staake, Thorsten AU - Steins-Löber, Sabine AU - Stingl, Carlo AU - Stöckl, Andreas AU - Struck, Olaf AU - Velde, Caroline van der AU - Wolstein, Jörg ED - Fischbach, Kai PY - 2021 IS - April 2021, Mensch und Maschine - Bamberger Beiträge zur Digitalisierung = Man and Machine - Bamberg Scholars on Digitalisation PB - Otto-Friedrich-Universität CY - Bamberg SN - 1618-9019 UR - https://fis.uni-bamberg.de/handle/uniba/50041 LA - deu AB - Wenn Menschen die Technik weiterentwickeln, entstehen ständig neue Möglichkeiten und Herausforderungen. Die Gesellschaft ist gefragt mitzudiskutieren, wie eine immer weiter digitalisierte Welt künftig aussehen könnte. Und die Forschung hilft dabei, dass sich jede und jeder Einzelne eine fundierte Meinung bilden kann. In dieser Ausgabe schreiben die Autorinnen und Autoren über den Einsatz der Technik im Alltag, die Zusammenarbeit von Menschen und Maschinen, das Pro und Contra der Digitalisierung. KW - Digitalisierung KW - Technik KW - KI KW - Gesellschaft KW - App KW - Internet KW - Arbeitsmarkt KW - Geschichte KW - Psychologie KW - Architektur KW - Kinder KW - Medien KW - Finanzen KW - Gesundheit ER - TY - CHAP TI - Uncovering the Bias in Facial Expressions AU - Deuschel, Jessica AU - Finzel, Bettina AU - Rieger, Ines PY - 2021 SP - 1-28 PB - Otto-Friedrich-Universität CY - Bamberg DO - 10.20378/irb-50304 UR - https://fis.uni-bamberg.de/handle/uniba/50304 LA - eng AB - Over the past decades the machine and deep learning community has celebrated great achievements in challenging tasks such as image classification. The deep architecture of artificial neural networks together with the plenitude of available data makes it possible to describe highly complex relations. Yet, it is still impossible to fully capture what the deep learning model has learned and to verify that it operates fairly and without creating bias, especially in critical tasks, for instance those arising in the medical field. One example for such a task is the detection of distinct facial expressions, called Action Units, in facial images. Considering this specific task, our research aims to provide transparency regarding bias, specifically in relation to gender and skin color. We train a neural network for Action Unit classification and analyze its performance quantitatively based on its accuracy and qualitatively based on heatmaps. A structured review of our results indicates that we are able to detect bias. Even though we cannot conclude from our results that lower classification performance emerged solely from gender and skin color bias, these biases must be addressed, which is why we end by giving suggestions on how the detected bias can be avoided. KW - Facial Expressions, Bias, Neural Networks, Deep Learning, Facial Action Units ER - TY - GEN TI - XAI Method Properties : A (Meta-)study AU - Schwalbe, Gesina AU - Finzel, Bettina PY - 2021 IS - Submitted on 15 May 2021 UR - https://fis.uni-bamberg.de/handle/uniba/51285 LA - eng ER - TY - CPAPER TI - Kolloquium Forschende Frauen 2020 : Gender in Gesellschaft 4.0 : Beiträge Bamberger Nachwuchswissenschaftlerinnen AU - Rauscher, Judith AU - Deuschel, Jessica AU - Finzel, Bettina AU - Rieger, Ines AU - Wagner, Anne AU - Velde, Caroline van der AU - Oehlhorn, Caroline AU - Hlukhovych, Adrianna ED - Rauscher, Judith ED - Hess, Mona ED - Schütz, Astrid ED - Szczepaniak, Renata PY - 2021 PB - University of Bamberg Press CY - Bamberg T3 - Forschende Frauen in Bamberg SN - 978-3-86309-853-7 DO - 10.20378/irb-50050 UR - https://fis.uni-bamberg.de/handle/uniba/50050 LA - other AB - Die Buchreihe „Forschende Frauen in Bamberg“ begleitet das gleichnamige Forschungskolloquium der Frauenbeauftragten der Otto-Friedrich-Universität Bamberg. Das Thema für 2020 ist "Gender in Gesellschaft 4.0". KW - Bias, Facial Expressions, Facial Action Units, Deep Learning, Neural Network, Skin color, Gender, Layer-wise Relevance Propagation, Machine Learning, Artificial Intelligence, FairAI, Interpretable AI KW - IT-Fachkräfte, Arbeitgeberattraktivität, Kano-Analyse KW - Gender, Medien, Geschlechtsidentitäten, Cyberfeminismus, Neuer Materialismus, kritischer Posthumanismus KW - I-GENDO, obesity, mHealth, gender, psychological therapy KW - I-GENDO, Adipositas, mHealth, gender, Psychotherapie KW - Digitalisierung, Berufliche Bildung, Berufsschule, Geschlechterunterschiede, Digitale Medien, Unterricht, Tablet ER - TY - JOUR TI - The Next Generation of Medical Decision Support : A Roadmap Toward Transparent Expert Companions AU - Bruckert, Sebastian AU - Finzel, Bettina AU - Schmid, Ute PY - 2020 JO - Frontiers in artificial intelligence VL - 3 PB - Frontiers Media CY - Lausanne SN - 2624-8212 DO - 10.3389/frai.2020.507973 UR - https://fis.uni-bamberg.de/handle/uniba/57747 LA - eng AB - Increasing quality and performance of artificial intelligence (AI) in general and machine learning (ML) in particular is followed by a wider use of these approaches in everyday life. As part of this development, ML classifiers have also gained more importance for diagnosing diseases within biomedical engineering and medical sciences. However, many of those ubiquitous high-performing ML algorithms reveal a black-box-nature, leading to opaque and incomprehensible systems that complicate human interpretations of single predictions or the whole prediction process. This puts up a serious challenge on human decision makers to develop trust, which is much needed in life-changing decision tasks. This paper is designed to answer the question how expert companion systems for decision support can be designed to be interpretable and therefore transparent and comprehensible for humans. On the other hand, an approach for interactive ML as well as human-in-the-loop-learning is demonstrated in order to integrate human expert knowledge into ML models so that humans and machines act as companions within a critical decision task. We especially address the problem of Semantic Alignment between ML classifiers and its human users as a prerequisite for semantically relevant and useful explanations as well as interactions. Our roadmap paper presents and discusses an interdisciplinary yet integrated Comprehensible Artificial Intelligence (cAI)-transition-framework with regard to the task of medical diagnosis. We explain and integrate relevant concepts and research areas to provide the reader with a hands-on-cookbook for achieving the transition from opaque black-box models to interactive, transparent, comprehensible and trustworthy systems. To make our approach tangible, we present suitable state of the art methods with regard to the medical domain and include a realization concept of our framework. The emphasis is on the concept of Mutual Explanations (ME) that we introduce as a dialog-based, incremental process in order to provide human ML users with trust, but also with stronger participation within the learning process. KW - explainable artificial intelligence KW - interactive ML KW - interpretability KW - trust KW - medical diagnosis KW - medical decision support KW - companion ER - TY - CPAPER TI - Verifying Deep Learning-based Decisions for Facial Expression Recognition AU - Rieger, Ines AU - Kollmann, Rene AU - Finzel, Bettina AU - Seuß, Dominik AU - Schmid, Ute PY - 2020 T2 - ESANN 2020 proceedings SP - 139-144 SN - 978-2-87587-074-2 UR - https://fis.uni-bamberg.de/handle/uniba/49224 LA - eng ER - TY - JOUR TI - Korrigierbares maschinelles Lernen mithilfe wechselseitiger Erklärungen am Beispiel der Medizin AU - Finzel, Bettina PY - 2020 JO - Frauen machen Informatik IS - 44 SP - 14-17 CY - Bonn SN - 2364-6152 UR - https://fis.uni-bamberg.de/handle/uniba/49223 LA - deu ER - TY - JOUR TI - The Next Generation of Medical Decision Support : A Roadmap Toward Transparent Expert Companions AU - Bruckert, Sebastian AU - Finzel, Bettina AU - Schmid, Ute PY - 2020 IS - 507973 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/49222 LA - eng AB - Increasing quality and performance of artificial intelligence (AI) in general and machine learning (ML) in particular is followed by a wider use of these approaches in everyday life. As part of this development, ML classifiers have also gained more importance for diagnosing diseases within biomedical engineering and medical sciences. However, many of those ubiquitous high-performing ML algorithms reveal a black-box-nature, leading to opaque and incomprehensible systems that complicate human interpretations of single predictions or the whole prediction process. This puts up a serious challenge on human decision makers to develop trust, which is much needed in life-changing decision tasks. This paper is designed to answer the question how expert companion systems for decision support can be designed to be interpretable and therefore transparent and comprehensible for humans. On the other hand, an approach for interactive ML as well as human-in-the-loop-learning is demonstrated in order to integrate human expert knowledge into ML models so that humans and machines act as companions within a critical decision task. We especially address the problem of Semantic Alignment between ML classifiers and its human users as a prerequisite for semantically relevant and useful explanations as well as interactions. Our roadmap paper presents and discusses an interdisciplinary yet integrated Comprehensible Artificial Intelligence (cAI)-transition-framework with regard to the task of medical diagnosis. We explain and integrate relevant concepts and research areas to provide the reader with a hands-on-cookbook for achieving the transition from opaque black-box models to interactive, transparent, comprehensible and trustworthy systems. To make our approach tangible, we present suitable state of the art methods with regard to the medical domain and include a realization concept of our framework. The emphasis is on the concept of Mutual Explanations (ME) that we introduce as a dialog-based, incremental process in order to provide human ML users with trust, but also with stronger participation within the learning process. KW - explainable artificial intelligence KW - interactive ML KW - interpretability KW - trust KW - medical diagnosis KW - medical decision support KW - companion ER - TY - JOUR TI - Mutual Explanations for Cooperative Decision Making in Medicine AU - Schmid, Ute AU - Finzel, Bettina PY - 2020 JO - Künstliche Intelligenz : KI ; Forschung, Entwicklung, Erfahrungen VL - 34 IS - 2 SP - 227-233 PB - Springer CY - Berlin ; Heidelberg SN - 1610-1987 DO - 10.1007/s13218-020-00633-2 UR - https://fis.uni-bamberg.de/handle/uniba/47120 LA - eng AB - Exploiting mutual explanations for interactive learning is presented as part of an interdisciplinary research project on transparent machine learning for medical decision support. Focus of the project is to combine deep learning black box approaches with interpretable machine learning for classification of different types of medical images to combine the predictive accuracy of deep learning and the transparency and comprehensibility of interpretable models. Specifically, we present an extension of the Inductive Logic Programming system Aleph to allow for interactive learning. Medical experts can ask for verbal explanations. They can correct classification decisions and in addition can also correct the explanations. Thereby, expert knowledge can be taken into account in form of constraints for model adaption. KW - Human-AI partnership KW - Inductive Logic Programming KW - Explanations as constraints ER - TY - RPRT TI - Unternehmenskulturen im Wandel - Karriereentwicklung im Informatikbereich. Ergebnisse des Forschungsprojektes "Alumnae Tracking" AU - Förtsch, Silvia AU - Gärtig-Daugs, Anja AU - Schmid, Ute PY - 2017 PB - opus CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/42406 LA - deu AB - Unternehmenskulturen im Wandel beinhaltet quantitative Ergebnisse aus der Studierenden- und Ehemaligenbefragung sowie qualitative Ergebnisse aus der Unternehmensbefragung im Forschungsprojekt "Alumnae Tracking". ER -