TY - CPAPER TI - Stylizing ViT : Anatomy-Preserving Instance Style Transfer for Domain Generalization AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Alle, Jonas AU - Ledig, Christian PY - 2026 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/116855 LA - eng AB - Deep learning models in medical image analysis often struggle with generalizability across domains and demographic groups due to data heterogeneity and scarcity. Traditional augmentation improves robustness, but fails under substantial domain shifts. Recent advances in stylistic augmentation enhance domain generalization by varying image styles but fall short in terms of style diversity or by introducing artifacts into the generated images. To address these limitations, we propose Stylizing ViT, a novel Vision Transformer encoder that utilizes weight-shared attention blocks for both self- and cross-attention. This design allows the same attention block to maintain anatomical consistency through self-attention while performing style transfer via cross-attention. We assess the effectiveness of our method for domain generalization by employing it for data augmentation on three distinct image classification tasks in the context of histopathology and dermatology. Results demonstrate an improved robustness (up to +13% accuracy) over the state of the art while generating perceptually convincing images without artifacts. Additionally, we show that Stylizing ViT is effective beyond training, achieving a 17% performance improvement during inference when used for test-time augmentation. The source code is available at https://github.com/sdoerrich97/stylizing-vit . KW - style transfer KW - domain generalization KW - crossattention KW - data augmentation KW - vision transformer ER - TY - CPAPER TI - Stylizing ViT : Anatomy-Preserving Instance Style Transfer for Domain Generalization AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Alle, Jonas AU - Ledig, Christian PY - 2026 T2 - 2026 IEEE 23rd International Symposium on Biomedical Imaging (ISBI) PB - IEEE CY - Piscataway, NJ DO - 10.1109/isbi61048.2026.11515498 UR - https://fis.uni-bamberg.de/handle/uniba/117126 LA - eng AB - Deep learning models in medical image analysis often struggle with generalizability across domains and demographic groups due to data heterogeneity and scarcity. Traditional augmentation improves robustness, but fails under substantial domain shifts. Recent advances in stylistic augmentation enhance domain generalization by varying image styles but fall short in terms of style diversity or by introducing artifacts into the generated images. To address these limitations, we propose Stylizing ViT, a novel Vision Transformer encoder that utilizes weight-shared attention blocks for both self- and cross-attention. This design allows the same attention block to maintain anatomical consistency through self-attention while performing style transfer via cross-attention. We assess the effectiveness of our method for domain generalization by employing it for data augmentation on three distinct image classification tasks in the context of histopathology and dermatology. Results demonstrate an improved robustness (up to +13% accuracy) over the state of the art while generating perceptually convincing images without artifacts. Additionally, we show that Stylizing ViT is effective beyond training, achieving a 17% performance improvement during inference when used for test-time augmentation. The source code is available at https://github.com/sdoerrich97/stylizing-vit . KW - style transfer KW - domain generalization KW - crossattention KW - data augmentation KW - vision transformer ER - TY - CPAPER TI - An End-to-End AI Pipeline for Wood Knot Detection to Enhance Structural Assessment in Historic Timber Structures AU - Pan, Junquan AU - Chizhova, Maria AU - Ebener, Frank AU - Luhmann, Thomas AU - Ledig, Christian AU - Maiwald, Ferdinand AU - Eißing, Thomas PY - 2026 SP - 267-274 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/113148 LA - eng AB - The accurate detection and assessment of wood surface defects in historic timber structures, particularly knots, is essential for effective conservation and strengthening planning. However, the application of automated visual grading methods to aged timber remains underexplored due to the irregular texture, weathering and lack of relevant datasets. In this study, we propose an end-to-end deep learning-based pipeline that integrates wood surface segmentation, perspective correction, and knot detection to estimate structural grading factors. A dedicated raw data collection of over 10,000 high-resolution images of historic timber surfaces was compiled using both DSLR cameras and mobile devices, resulting in multiple datasets with approximately 3,000 annotated samples. Three model families, YOLO, Detectron2 and DeepLabV3, were evaluated under different experimental setups. Beyond model benchmarking, we further compared the AI-derived results with expert manual measurements. The model for segmentation of timber surface achieved a mean IoU of over 0.85 and the model for detection of historical wood knots reached F1 scores of up to 0.9. The structural assessment factors estimated by the AI pipeline achieved a Pearson correlation coefficient of 0.641 compared to manual measurements, indicating a moderate level of consistency in knot factor estimation. This research highlights the potential of vision-based AI systems in supporting structural diagnosis and conservation of heritage timber elements. KW - Photogrammetry KW - Deep Learning KW - Historic Timber Structures KW - Wood Knot Detection KW - Structural Assessment ER - TY - JOUR TI - Incorporating label uncertainty during the training of convolutional neural networks improves performance for the discrimination between certain and inconclusive cases in dopamine transporter SPECT AU - Kucerenko, Aleksej AU - Buddenkotte, Thomas AU - Apostolova, Ivayla AU - Klutmann, Susanne AU - Ledig, Christian AU - Buchert, Ralph PY - 2025 SP - 1535-1548 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/111618 LA - eng AB - Purpose: Deep convolutional neural networks (CNN) hold promise for assisting the interpretation of dopamine transporter (DAT)-SPECT. For improved communication of uncertainty to the user it is crucial to reliably discriminate certain from inconclusive cases that might be misclassified by strict application of a predefined decision threshold on the CNN output. This study tested two methods to incorporate existing label uncertainty during the training to improve the utility of the CNN sigmoid output for this task. Methods: Three datasets were used retrospectively: a “development” dataset (n = 1740) for CNN training, validation and testing, two independent out-of-distribution datasets (n = 640, 645) for testing only. In the development dataset, binary classification based on visual inspection was performed carefully by three well-trained readers. A ResNet-18 architecture was trained for binary classification of DAT-SPECT using either a randomly selected vote (“random vote training”, RVT), the proportion of “reduced” votes ( “average vote training”, AVT) or the majority vote (MVT) across the three readers as reference standard. Balanced accuracy was computed separately for “inconclusive” sigmoid outputs (within a predefined interval around the 0.5 decision threshold) and for “certain” (non-inconclusive) sigmoid outputs. Results: The proportion of “inconclusive” test cases that had to be accepted to achieve a given balanced accuracy in the “certain” test case was lower with RVT and AVT than with MVT in all datasets (e.g., 1.9% and 1.2% versus 2.8% for 98% balanced accuracy in “certain” test cases from the development dataset). In addition, RVT and AVT resulted in slightly higher balanced accuracy in all test cases independent of their certainty (97.3% and 97.5% versus 97.0% in the development dataset). Conclusion: Making between-readers-discrepancy known to CNN during the training improves the utility of their sigmoid output to discriminate certain from inconclusive cases that might be misclassified by the CNN when the predefined decision threshold is strictly applied. This does not compromise on overall accuracy. KW - Deep learning KW - Convolutional neural network KW - Uncertainty KW - FP-CIT KW - Dopamine transporter KW - SPECT ER - TY - JOUR TI - An Embedding is Worth a Thousand Noisy Labels AU - Di Salvo, Francesco AU - Doerrich, Sebastian AU - Ledig, Christian AU - Rieger, Ines PY - 2025 SP - 1-19 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/109531 LA - eng AB - The performance of deep neural networks scales with dataset size and label quality, rendering the efficient mitigation of low-quality data annotations crucial for building robust and cost-effective systems. Existing strategies to address label noise exhibit severe limitations due to computational complexity and application dependency. In this work, we propose WANN, a Weighted Adaptive Nearest Neighbor approach that builds on self-supervised feature representations obtained from foundation models. To guide the weighted voting scheme, we introduce a reliability score, which measures the likelihood of a data label being correct. WANN outperforms reference methods, including a linear layer trained with robust loss functions, on diverse datasets of varying size and under various noise types and severities. WANN also exhibits superior generalization on imbalanced data compared to both Adaptive-NNs (ANN) and fixed k-NNs. Furthermore, the proposed weighting scheme enhances supervised dimensionality reduction under noisy labels. This yields a significant boost in classification performance with 10x and 100x smaller image embeddings, minimizing latency and storage requirements. Our approach, emphasizing efficiency and explainability, emerges as a simple, robust solution to overcome inherent limitations of deep neural network training. KW - Adaptive kNN KW - Label noise KW - Robustness KW - Foundation models ER - TY - CPAPER TI - MedMNIST-C : Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions AU - Di Salvo, Francesco AU - Doerrich, Sebastian AU - Ledig, Christian PY - 2025 SP - 1-11 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/109532 LA - eng AB - The integration of neural-network-based systems into clinical practice is limited by challenges related to domain generalization and robustness. The computer vision community established benchmarks such as ImageNet-C as a fundamental prerequisite to measure progress towards those challenges. Similar datasets are largely absent in the medical imaging community which lacks a comprehensive benchmark that spans across imaging modalities and applications. To address this gap, we create and open-source MedMNIST-C, a benchmark dataset based on the MedMNIST+ collection covering 12 datasets and 9 imaging modalities. We simulate task and modality-specific image corruptions of varying severity to comprehensively evaluate the robustness of established algorithms against real-world artifacts and distribution shifts. We further provide quantitative evidence that our simple-to-use artificial corruptions allow for highly performant, lightweight data augmentation to enhance model robustness. Unlike traditional, generic augmentation strategies, our approach leverages domain knowledge, exhibiting significantly higher robustness when compared to widely adopted methods. By introducing MedMNIST-C and open-sourcing the corresponding library allowing for targeted data augmentations, we contribute to the development of increasingly robust methods tailored to the challenges of medical imaging. KW - Robustness KW - Corruptions KW - Data augmentation ER - TY - CPAPER TI - Unsupervised Feature Orthogonalization for Learning Distortion-Invariant Representations AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2025 SP - 1-13 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/109529 LA - eng AB - This study introduces unORANIC+, a novel method that integrates unsupervised feature orthogonalization with the ability of a Vision Transformer to capture both local and global relationships for improved robustness and generalizability. The streamlined architecture of unORANIC+ effectively separates anatomical and image-specific attributes, resulting in robust and unbiased latent representations that allow the model to demonstrate excellent performance across various medical image analysis tasks and diverse datasets. Extensive experimentation demonstrates unORANIC+’s reconstruction proficiency, corruption resilience, as well as capability to revise existing image distortions. Additionally, the model exhibits notable aptitude in downstream tasks such as disease classification and corruption detection. We confirm its adaptability to diverse datasets of varying image sources and sample sizes which positions the method as a promising algorithm for advanced medical image analysis, particularly in resourceconstrained environments lacking large, tailored datasets. The source code is available at github.com/sdoerrich97/unoranic-plus. KW - feature orthogonalization KW - robustness KW - corruption revision KW - unsupervised learning KW - generalization ER - TY - CPAPER TI - Privacy-preserving datasets by capturing feature distributions with Conditional VAEs AU - Di Salvo, Francesco AU - Tafler, David E. AU - Doerrich, Sebastian AU - Ledig, Christian PY - 2025 SP - 1-14 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/109530 LA - eng AB - Large and well-annotated datasets are essential for advancing deep learning applications, however often costly or impossible to obtain by a single entity. In many areas, including the medical domain, approaches relying on data sharing have become critical to address those challenges. While effective in increasing dataset size and diversity, data sharing raises significant privacy concerns. Commonly employed anonymization methods based on the k-anonymity paradigm often fail to preserve data diversity, affecting model robustness. This work introduces a novel approach using Conditional Variational Autoencoders (CVAEs) trained on feature vectors extracted from large pre-trained vision foundation models. Foundation models effectively detect and represent complex patterns across diverse domains, allowing the CVAE to faithfully capture the embedding space of a given data distribution to generate (sample) a diverse, privacy-respecting, and potentially unbounded set of synthetic feature vectors. Our method notably outperforms traditional approaches in both medical and natural image domains, exhibiting greater dataset diversity and higher robustness against perturbations while preserving sample privacy. These results underscore the potential of generative models to significantly impact deep learning applications in data-scarce and privacy-sensitive environments. KW - Generative models KW - Privacy KW - Foundation models KW - Robustness ER - TY - CPAPER TI - An End-to-End AI Pipeline for Wood Knot Detection to Enhance Structural Assessment in Historic Timber Structures AU - Pan, Junquan AU - Chizhova, Maria AU - Ebener, Frank AU - Luhmann, Thomas AU - Ledig, Christian AU - Maiwald, Ferdinand AU - Eißing, Thomas PY - 2025 VL - X-M-2-2025 SP - 267-274 PB - Copernicus Publications CY - Katlenburg-Lindau SN - 2194-9050 DO - 10.5194/isprs-annals-x-m-2-2025-267-2025 UR - https://fis.uni-bamberg.de/handle/uniba/110553 LA - eng AB - The accurate detection and assessment of wood surface defects in historic timber structures, particularly knots, is essential for effective conservation and strengthening planning. However, the application of automated visual grading methods to aged timber remains underexplored due to the irregular texture, weathering and lack of relevant datasets. In this study, we propose an end-to-end deep learning-based pipeline that integrates wood surface segmentation, perspective correction, and knot detection to estimate structural grading factors. A dedicated raw data collection of over 10,000 high-resolution images of historic timber surfaces was compiled using both DSLR cameras and mobile devices, resulting in multiple datasets with approximately 3,000 annotated samples. Three model families, YOLO, Detectron2 and DeepLabV3, were evaluated under different experimental setups. Beyond model benchmarking, we further compared the AI-derived results with expert manual measurements. The model for segmentation of timber surface achieved a mean IoU of over 0.85 and the model for detection of historical wood knots reached F1 scores of up to 0.9. The structural assessment factors estimated by the AI pipeline achieved a Pearson correlation coefficient of 0.641 compared to manual measurements, indicating a moderate level of consistency in knot factor estimation. This research highlights the potential of vision-based AI systems in supporting structural diagnosis and conservation of heritage timber elements. KW - Photogrammetry KW - Deep Learning KW - Historic Timber Structures KW - Wood Knot Detection KW - Structural Assessment ER - TY - JOUR TI - An Embedding is Worth a Thousand Noisy Labels AU - Di Salvo, Francesco AU - Doerrich, Sebastian AU - Rieger, Ines AU - Ledig, Christian PY - 2025 JO - Transactions on machine learning research IS - 3 SP - 1-19 PB - OpenReview.net CY - Amherst, Massachusetts SN - 2835-8856 UR - https://fis.uni-bamberg.de/handle/uniba/109535 LA - eng AB - The performance of deep neural networks scales with dataset size and label quality, rendering the efficient mitigation of low-quality data annotations crucial for building robust and cost-effective systems. Existing strategies to address label noise exhibit severe limitations due to computational complexity and application dependency. In this work, we propose WANN, a Weighted Adaptive Nearest Neighbor approach that builds on self-supervised feature representations obtained from foundation models. To guide the weighted voting scheme, we introduce a reliability score, which measures the likelihood of a data label being correct. WANN outperforms reference methods, including a linear layer trained with robust loss functions, on diverse datasets of varying size and under various noise types and severities. WANN also exhibits superior generalization on imbalanced data compared to both Adaptive-NNs (ANN) and fixed k-NNs. Furthermore, the proposed weighting scheme enhances supervised dimensionality reduction under noisy labels. This yields a significant boost in classification performance with 10x and 100x smaller image embeddings, minimizing latency and storage requirements. Our approach, emphasizing efficiency and explainability, emerges as a simple, robust solution to overcome inherent limitations of deep neural network training. KW - Adaptive kNN KW - Label noise KW - Robustness KW - Foundation models ER - TY - JOUR TI - Rethinking model prototyping through the MedMNIST+ dataset collection AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Brockmann, Julius AU - Ledig, Christian PY - 2025 SP - 1-15 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/107030 LA - eng AB - The integration of deep learning based systems in clinical practice is often impeded by challenges rooted in limited and heterogeneous medical datasets. In addition, the field has increasingly prioritized marginal performance gains on a few, narrowly scoped benchmarks over clinical applicability, slowing down meaningful algorithmic progress. This trend often results in excessive fine-tuning of existing methods on selected datasets rather than fostering clinically relevant innovations. In response, this work introduces a comprehensive benchmark for the MedMNIST+ dataset collection, designed to diversify the evaluation landscape across several imaging modalities, anatomical regions, classification tasks and sample sizes. We systematically reassess commonly used Convolutional Neural Networks (CNNs) and Vision Transformer (ViT) architectures across distinct medical datasets, training methodologies, and input resolutions to validate and refine existing assumptions about model effectiveness and development. Our findings suggest that computationally efficient training schemes and modern foundation models offer viable alternatives to costly end-to-end training. Additionally, we observe that higher image resolutions do not consistently improve performance beyond a certain threshold. This highlights the potential benefits of using lower resolutions, particularly in prototyping stages, to reduce computational demands without sacrificing accuracy. Notably, our analysis reaffirms the competitiveness of CNNs compared to ViTs, emphasizing the importance of comprehending the intrinsic capabilities of different architectures. Finally, by establishing a standardized evaluation framework, we aim to enhance transparency, reproducibility, and comparability within the MedMNIST+ dataset collection as well as future research. Code is available at github.com/sdoerrich97/rethinking-model-prototyping-MedMNISTPlus. KW - Benchmarking KW - Prototyping recommendations KW - Medical image classification KW - Standardized evaluation framework KW - Foundation models ER - TY - CPAPER TI - unORANIC : Unsupervised Orthogonalization of Anatomy and Image-Characteristic Features AU - Dörrich, Sebastian AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2025 SP - 1-11 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/106934 LA - eng AB - We introduce unORANIC, an unsupervised approach that uses an adapted loss function to drive the orthogonalization of anatomy and image-characteristic features. The method is versatile for diverse modalities and tasks, as it does not require domain knowledge, paired data samples, or labels. During test time unORANIC is applied to potentially corrupted images, orthogonalizing their anatomy and characteristic components, to subsequently reconstruct corruption-free images, showing their domain-invariant anatomy only. This feature orthogonalization further improves generalization and robustness against corruptions. We confirm this qualitatively and quantitatively on 5 distinct datasets by assessing unORANIC’s classification accuracy, corruption detection and revision capabilities. Our approach shows promise for enhancing the generalizability and robustness of practical applications in medical image analysis. The source code is available at github.com/sdoerrich97/unORANIC. KW - feature orthogonalization KW - robustness KW - corruption revision KW - unsupervised learning KW - generalization ER - TY - JOUR TI - Rethinking model prototyping through the MedMNIST+ dataset collection AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Brockmann, Julius AU - Ledig, Christian PY - 2025 JO - Scientific reports VL - 15 IS - 1, 7669 SP - 1-15 PB - Springer Nature CY - London SN - 2045-2322 DO - 10.1038/s41598-025-92156-9 UR - https://fis.uni-bamberg.de/handle/uniba/107047 LA - eng AB - The integration of deep learning based systems in clinical practice is often impeded by challenges rooted in limited and heterogeneous medical datasets. In addition, the field has increasingly prioritized marginal performance gains on a few, narrowly scoped benchmarks over clinical applicability, slowing down meaningful algorithmic progress. This trend often results in excessive fine-tuning of existing methods on selected datasets rather than fostering clinically relevant innovations. In response, this work introduces a comprehensive benchmark for the MedMNIST+ dataset collection, designed to diversify the evaluation landscape across several imaging modalities, anatomical regions, classification tasks and sample sizes. We systematically reassess commonly used Convolutional Neural Networks (CNNs) and Vision Transformer (ViT) architectures across distinct medical datasets, training methodologies, and input resolutions to validate and refine existing assumptions about model effectiveness and development. Our findings suggest that computationally efficient training schemes and modern foundation models offer viable alternatives to costly end-to-end training. Additionally, we observe that higher image resolutions do not consistently improve performance beyond a certain threshold. This highlights the potential benefits of using lower resolutions, particularly in prototyping stages, to reduce computational demands without sacrificing accuracy. Notably, our analysis reaffirms the competitiveness of CNNs compared to ViTs, emphasizing the importance of comprehending the intrinsic capabilities of different architectures. Finally, by establishing a standardized evaluation framework, we aim to enhance transparency, reproducibility, and comparability within the MedMNIST+ dataset collection as well as future research. Code is available at github.com/sdoerrich97/rethinking-model-prototyping-MedMNISTPlus. KW - Benchmarking KW - Prototyping recommendations KW - Medical image classification KW - Standardized evaluation framework KW - Foundation models ER - TY - CPAPER TI - Privacy-preserving datasets by capturing feature distributions with Conditional VAEs AU - Di Salvo, Francesco AU - Tafler, David AU - Doerrich, Sebastian AU - Ledig, Christian PY - 2024 T2 - Proceedings : The 35th British Machine Vision Conference 2024 SP - 1-14 UR - https://fis.uni-bamberg.de/handle/uniba/109551 LA - eng AB - Large and well-annotated datasets are essential for advancing deep learning applications, however often costly or impossible to obtain by a single entity. In many areas, including the medical domain, approaches relying on data sharing have become critical to address those challenges. While effective in increasing dataset size and diversity, data sharing raises significant privacy concerns. Commonly employed anonymization methods based on the k-anonymity paradigm often fail to preserve data diversity, affecting model robustness. This work introduces a novel approach using Conditional Variational Autoencoders (CVAEs) trained on feature vectors extracted from large pre-trained vision foundation models. Foundation models effectively detect and represent complex patterns across diverse domains, allowing the CVAE to faithfully capture the embedding space of a given data distribution to generate (sample) a diverse, privacy-respecting, and potentially unbounded set of synthetic feature vectors. Our method notably outperforms traditional approaches in both medical and natural image domains, exhibiting greater dataset diversity and higher robustness against perturbations while preserving sample privacy. These results underscore the potential of generative models to significantly impact deep learning applications in data-scarce and privacy-sensitive environments. KW - Generative models KW - Privacy KW - Foundation models KW - Robustness ER - TY - CPAPER TI - Unsupervised Feature Orthogonalization for Learning Distortion-Invariant Representations AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2024 T2 - Workshop Proceedings : International Workshop on Robust Recognition in the Open World at The 35th British Machine Vision Conference SP - 1-13 UR - https://fis.uni-bamberg.de/handle/uniba/109550 LA - eng AB - This study introduces unORANIC+, a novel method that integrates unsupervised feature orthogonalization with the ability of a Vision Transformer to capture both local and global relationships for improved robustness and generalizability. The streamlined architecture of unORANIC+ effectively separates anatomical and image-specific attributes, resulting in robust and unbiased latent representations that allow the model to demonstrate excellent performance across various medical image analysis tasks and diverse datasets. Extensive experimentation demonstrates unORANIC+’s reconstruction proficiency, corruption resilience, as well as capability to revise existing image distortions. Additionally, the model exhibits notable aptitude in downstream tasks such as disease classification and corruption detection. We confirm its adaptability to diverse datasets of varying image sources and sample sizes which positions the method as a promising algorithm for advanced medical image analysis, particularly in resourceconstrained environments lacking large, tailored datasets. The source code is available at github.com/sdoerrich97/unoranic-plus. KW - feature orthogonalization KW - robustness KW - corruption revision KW - unsupervised learning KW - generalization ER - TY - CPAPER TI - MedMNIST-C : Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions AU - Di Salvo, Francesco AU - Doerrich, Sebastian AU - Ledig, Christian PY - 2024 SP - 1-11 DO - 10.48550/arXiv.2406.17536 UR - https://fis.uni-bamberg.de/handle/uniba/109539 LA - eng AB - The integration of neural-network-based systems into clinical practice is limited by challenges related to domain generalization and robustness. The computer vision community established benchmarks such as ImageNet-C as a fundamental prerequisite to measure progress towards those challenges. Similar datasets are largely absent in the medical imaging community which lacks a comprehensive benchmark that spans across imaging modalities and applications. To address this gap, we create and open-source MedMNIST-C, a benchmark dataset based on the MedMNIST+ collection covering 12 datasets and 9 imaging modalities. We simulate task and modality-specific image corruptions of varying severity to comprehensively evaluate the robustness of established algorithms against real-world artifacts and distribution shifts. We further provide quantitative evidence that our simple-to-use artificial corruptions allow for highly performant, lightweight data augmentation to enhance model robustness. Unlike traditional, generic augmentation strategies, our approach leverages domain knowledge, exhibiting significantly higher robustness when compared to widely adopted methods. By introducing MedMNIST-C and open-sourcing the corresponding library allowing for targeted data augmentations, we contribute to the development of increasingly robust methods tailored to the challenges of medical imaging. KW - Robustness KW - Corruptions KW - Data augmentation ER - TY - JOUR TI - Incorporating label uncertainty during the training of convolutional neural networks improves performance for the discrimination between certain and inconclusive cases in dopamine transporter SPECT AU - Kucerenko, Aleksej AU - Buddenkotte, Thomas AU - Apostolova, Ivayla AU - Klutmann, Susanne AU - Ledig, Christian AU - Buchert, Ralph PY - 2024 JO - European Journal of Nuclear Medicine and Molecular Imaging VL - 52 IS - 4 SP - 1535-1548 PB - Springer CY - Heidelberg [u.a.] SN - 1619-7089 DO - 10.1007/s00259-024-06988-0 UR - https://fis.uni-bamberg.de/handle/uniba/106141 LA - eng AB - Purpose: Deep convolutional neural networks (CNN) hold promise for assisting the interpretation of dopamine transporter (DAT)-SPECT. For improved communication of uncertainty to the user it is crucial to reliably discriminate certain from inconclusive cases that might be misclassified by strict application of a predefined decision threshold on the CNN output. This study tested two methods to incorporate existing label uncertainty during the training to improve the utility of the CNN sigmoid output for this task. Methods: Three datasets were used retrospectively: a “development” dataset (n = 1740) for CNN training, validation and testing, two independent out-of-distribution datasets (n = 640, 645) for testing only. In the development dataset, binary classification based on visual inspection was performed carefully by three well-trained readers. A ResNet-18 architecture was trained for binary classification of DAT-SPECT using either a randomly selected vote (“random vote training”, RVT), the proportion of “reduced” votes ( “average vote training”, AVT) or the majority vote (MVT) across the three readers as reference standard. Balanced accuracy was computed separately for “inconclusive” sigmoid outputs (within a predefined interval around the 0.5 decision threshold) and for “certain” (non-inconclusive) sigmoid outputs. Results: The proportion of “inconclusive” test cases that had to be accepted to achieve a given balanced accuracy in the “certain” test case was lower with RVT and AVT than with MVT in all datasets (e.g., 1.9% and 1.2% versus 2.8% for 98% balanced accuracy in “certain” test cases from the development dataset). In addition, RVT and AVT resulted in slightly higher balanced accuracy in all test cases independent of their certainty (97.3% and 97.5% versus 97.0% in the development dataset). Conclusion: Making between-readers-discrepancy known to CNN during the training improves the utility of their sigmoid output to discriminate certain from inconclusive cases that might be misclassified by the CNN when the predefined decision threshold is strictly applied. This does not compromise on overall accuracy. KW - Deep learning KW - Convolutional neural network KW - Uncertainty KW - FP-CIT KW - Dopamine transporter KW - SPECT ER - TY - CPAPER TI - Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2024 SP - 1-10 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/104846 LA - eng AB - Despite notable advancements, the integration of deep learning (DL) techniques into impactful clinical applications, particularly in the realm of digital histopathology, has been hindered by challenges associated with achieving robust generalization across diverse imaging domains and characteristics. Traditional mitigation strategies in this field such as data augmentation and stain color normalization have proven insufficient in addressing this limitation, necessitating the exploration of alternative methodologies. To this end, we propose a novel generative method for domain generalization in histopathology images. Our method employs a generative, self-supervised Vision Transformer to dynamically extract characteristics of image patches and seamlessly infuse them into the original images, thereby creating novel, synthetic images with diverse attributes. By enriching the dataset with such synthesized images, we aim to enhance its holistic nature, facilitating improved generalization of DL models to unseen domains. Extensive experiments conducted on two distinct histopathology datasets demonstrate the effectiveness of our proposed approach, outperforming the state of the art substantially, on the Camelyon17-wilds challenge dataset (+2%) and on a second epithelium-stroma dataset (+26%). Furthermore, we emphasize our method's ability to readily scale with increasingly available unlabeled data samples and more complex, higher parametric architectures. Source code is available at https://github.com/sdoerrich97/vits-are-generative-models. KW - domain generalization KW - self-supervised learning KW - feature orthogonalization KW - generative image synthesis ER - TY - CPAPER TI - Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2024 T2 - Medical Image Computing and Computer Assisted Intervention – MICCAI 2024 SP - 644-654 PB - Springer Nature Switzerland CY - Cham SN - 978-3-031-72117-5 DO - 10.1007/978-3-031-72117-5_60 UR - https://fis.uni-bamberg.de/handle/uniba/105289 LA - eng AB - Despite notable advancements, the integration of deep learning (DL) techniques into impactful clinical applications, particularly in the realm of digital histopathology, has been hindered by challenges associated with achieving robust generalization across diverse imaging domains and characteristics. Traditional mitigation strategies in this field such as data augmentation and stain color normalization have proven insufficient in addressing this limitation, necessitating the exploration of alternative methodologies. To this end, we propose a novel generative method for domain generalization in histopathology images. Our method employs a generative, self-supervised Vision Transformer to dynamically extract characteristics of image patches and seamlessly infuse them into the original images, thereby creating novel, synthetic images with diverse attributes. By enriching the dataset with such synthesized images, we aim to enhance its holistic nature, facilitating improved generalization of DL models to unseen domains. Extensive experiments conducted on two distinct histopathology datasets demonstrate the effectiveness of our proposed approach, outperforming the state of the art substantially, on the Camelyon17-wilds challenge dataset (+2%) and on a second epithelium-stroma dataset (+26%). Furthermore, we emphasize our method's ability to readily scale with increasingly available unlabeled data samples and more complex, higher parametric architectures. Source code is available at https://github.com/sdoerrich97/vits-are-generative-models. KW - domain generalization KW - self-supervised learning KW - feature orthogonalization KW - generative image synthesis ER - TY - CPAPER TI - Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification AU - Doerrich, Sebastian AU - Archut, Tobias AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2024 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/104752 LA - eng AB - Traditional deep learning models implicitly encode knowledge limiting their transparency and ability to adapt to data changes. Yet, this adaptability is vital for addressing user data privacy concerns. We address this limitation by storing embeddings of the underlying training data independently of the model weights, enabling dynamic data modifications without retraining. Specifically, our approach integrates the k-Nearest Neighbor (k-NN) classifier with a vision-based foundation model, pre-trained self-supervised on natural images, enhancing interpretability and adaptability. We share open-source implementations of a previously unpublished baseline method as well as our performance-improving contributions. Quantitative experiments confirm improved classification across established benchmark datasets and the method’s applicability to distinct medical image classification tasks. Additionally, we assess the method’s robustness in continual learning and data removal scenarios. The approach exhibits great promise for bridging the gap between foundation models’ performance and challenges tied to data privacy. The source code is available at github.com/TobArc. KW - k-NN classifier KW - continual learning KW - transfer learning KW - few-shot classification KW - explainability ER - TY - CPAPER TI - Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification AU - Doerrich, Sebastian AU - Archut, Tobias AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2024 T2 - 2024 IEEE International Symposium on Biomedical Imaging (ISBI) SP - 1-5 PB - IEEE DO - 10.1109/isbi56570.2024.10635560 UR - https://fis.uni-bamberg.de/handle/uniba/104176 LA - eng AB - Traditional deep learning models implicitly encode knowledge limiting their transparency and ability to adapt to data changes. Yet, this adaptability is vital for addressing user data privacy concerns. We address this limitation by storing embeddings of the underlying training data independently of the model weights, enabling dynamic data modifications without retraining. Specifically, our approach integrates the k-Nearest Neighbor (k-NN) classifier with a vision-based foundation model, pre-trained self-supervised on natural images, enhancing interpretability and adaptability. We share open-source implementations of a previously unpublished baseline method as well as our performance-improving contributions. Quantitative experiments confirm improved classification across established benchmark datasets and the method’s applicability to distinct medical image classification tasks. Additionally, we assess the method’s robustness in continual learning and data removal scenarios. The approach exhibits great promise for bridging the gap between foundation models’ performance and challenges tied to data privacy. The source code is available at github.com/TobArc. KW - k-NN classifier KW - continual learning KW - transfer learning KW - few-shot classification KW - explainability ER - TY - CPAPER TI - unORANIC : Unsupervised Orthogonalization of Anatomy and Image-Characteristic Features AU - Doerrich, Sebastian AU - Di Salvo, Francesco AU - Ledig, Christian PY - 2023 T2 - Machine Learning in Medical Imaging : 14th International Workshop, MLMI 2023, Held in Conjunction with MICCAI 2023, Vancouver, BC, Canada, October 8, 2023 ; Proceedings, Part I SP - 62-71 PB - Springer Nature Switzerland CY - Cham SN - 978-3-031-45672-5 DO - 10.1007/978-3-031-45673-2_7 UR - https://fis.uni-bamberg.de/handle/uniba/91842 LA - eng AB - We introduce unORANIC, an unsupervised approach that uses an adapted loss function to drive the orthogonalization of anatomy and image-characteristic features. The method is versatile for diverse modalities and tasks, as it does not require domain knowledge, paired data samples, or labels. During test time unORANIC is applied to potentially corrupted images, orthogonalizing their anatomy and characteristic components, to subsequently reconstruct corruption-free images, showing their domain-invariant anatomy only. This feature orthogonalization further improves generalization and robustness against corruptions. We confirm this qualitatively and quantitatively on 5 distinct datasets by assessing unORANIC’s classification accuracy, corruption detection and revision capabilities. Our approach shows promise for enhancing the generalizability and robustness of practical applications in medical image analysis. The source code is available at github.com/sdoerrich97/unORANIC. KW - feature orthogonalization KW - robustness KW - corruption revision KW - unsupervised learning KW - generalization ER - TY - JOUR TI - Differences Between MR Brain Region Segmentation Methods: Impact on Single-Subject Analysis AU - Huizinga, W. AU - Poot, D. H. J. AU - Vinke, E. J. AU - Wenzel, F. AU - Bron, E. E. AU - Toussaint, N. AU - Ledig, Christian AU - Vrooman, H. AU - Ikram, M. A. AU - Niessen, W. J. AU - Vernooij, M. W. AU - Klein, S. PY - 2021 JO - Frontiers in big data VL - 4 IS - 577164 PB - Frontiers Media CY - Lausanne SN - 2624-909X DO - 10.3389/fdata.2021.577164 UR - https://fis.uni-bamberg.de/handle/uniba/54056 LA - eng ER - TY - CHAP TI - Deep learning: Generative adversarial networks and adversarial methods AU - Wolterink, Jelmer M. AU - Kamnitsas, Konstantinos AU - Ledig, Christian AU - Išgum, Ivana PY - 2020 T2 - Handbook of medical image computing and computer assisted intervention SP - 547-574 PB - Academic Press CY - London ET - 1 SN - 9780128165867 DO - 10.1016/B978-0-12-816176-0.00028-4 UR - https://fis.uni-bamberg.de/handle/uniba/55000 LA - eng KW - Medizin KW - Bildverarbeitung ER - TY - JOUR TI - Integrative Analysis of Circulating Metabolite Profiles and Magnetic Resonance Imaging Metrics in Patients with Traumatic Brain Injury AU - Thomas, Ilias AU - Dickens, Alex M. AU - Posti, Jussi P. AU - Mohammadian, Mehrbod AU - Ledig, Christian AU - Takala, Riikka S. K. AU - Hyötyläinen, Tuulia AU - Tenovuo, Olli AU - Orešič, Matej PY - 2020 JO - International journal of molecular sciences VL - 21 IS - 4 SN - 1422-0067 DO - 10.3390/ijms21041395 UR - https://fis.uni-bamberg.de/handle/uniba/54994 LA - eng ER - TY - JOUR TI - Explainable Anatomical Shape Analysis Through Deep Hierarchical Generative Models AU - Biffi, Carlo AU - Cerrolaza, Juan J. AU - Tarroni, Giacomo AU - Bai, Wenjia AU - De Marvao, Antonio AU - Oktay, Ozan AU - Ledig, Christian AU - Le Folgoc, Loic AU - Kamnitsas, Konstantinos AU - Doumou, Georgia AU - Duan, Jinming AU - Prasad, Sanjay K. AU - Cook, Stuart A AU - O'Regan, Declan P. AU - Rueckert, Daniel PY - 2020 JO - IEEE transactions on medical imaging VL - 39 IS - 6 SP - 2088-2099 SN - 1558-254X DO - 10.1109/TMI.2020.2964499 UR - https://fis.uni-bamberg.de/handle/uniba/54993 LA - eng ER - TY - JOUR TI - Volume Change in Frontal Cholinergic Structures After Traumatic Brain Injury and Cognitive Outcome AU - Östberg, Anna AU - Ledig, Christian AU - Katila, Ari AU - Maanpää, Henna-Riikka AU - Posti, Jussi P. AU - Takala, Riikka AU - Tallus, Jussi AU - Glocker, Ben AU - Rueckert, Daniel AU - Tenovuo, Olli PY - 2020 JO - Frontiers in neurology VL - 11 IS - 832 SN - 1664-2295 DO - 10.3389/fneur.2020.00832 UR - https://fis.uni-bamberg.de/handle/uniba/54992 LA - eng ER - TY - JOUR TI - Assessment of a deep-learning system for fracture detection in musculoskeletal radiographs AU - Jones, Rebecca M. AU - Sharma, Anuj AU - Hotchkiss, Robert AU - Sperling, John W. AU - Hamburger, Jackson AU - Ledig, Christian AU - O'Toole, Robert AU - Gardner, Michael AU - Venkatesh, Srivas AU - Roberts, Matthew M. AU - Sauvestre, Romain AU - Shatkhin, Max AU - Gupta, Anant AU - Chopra, Sumit AU - Kumaravel, Manickam AU - Daluiski, Aaron AU - Plogger, Will AU - Nascone, Jason AU - Potter, Hollis G. AU - Lindsey, Robert V. PY - 2020 JO - NPJ digital medicine VL - 3 IS - 144 PB - Macmillan Publishers Limited CY - Basingstoke SN - 2398-6352 DO - 10.1038/s41746-020-00352-w UR - https://fis.uni-bamberg.de/handle/uniba/54057 LA - eng ER - TY - JOUR TI - Quantitative assessment of myelination patterns in preterm neonates using T2-weighted MRI AU - Wang, Siying AU - Ledig, Christian AU - Hajnal, Joseph V. AU - Counsell, Serena J. AU - Schnabel, Julia A. AU - Deprez, Maria PY - 2019 JO - Scientific reports VL - 9 IS - 1 SN - 2045-2322 DO - 10.1038/s41598-019-49350-3 UR - https://fis.uni-bamberg.de/handle/uniba/54995 LA - eng ER - TY - GEN TI - Segmentation of Myelin-like Signals on Clinical MR Images for Age Estimation in Preterm Infants AU - Deprez, Maria AU - Wang, Siying AU - Ledig, Christian AU - Hajnal, Joseph V. AU - Counsell, Serena J. AU - Schnabel, Julia A. PY - 2018 DO - 10.1101/357749 UR - https://fis.uni-bamberg.de/handle/uniba/59140 LA - eng ER - TY - GEN TI - Generative adversarial networks and adversarial methods in biomedical image analysis AU - Wolterink, Jelmer M. AU - Kamnitsas, Konstantinos AU - Ledig, Christian AU - Išgum, Ivana PY - 2018 SP - 1-29 PB - arXiv DO - 10.48550/ARXIV.1810.10352 UR - https://fis.uni-bamberg.de/handle/uniba/59134 LA - eng ER - TY - JOUR TI - Data-Driven Differential Diagnosis of Dementia Using Multiclass Disease State Index Classifier AU - Tolonen, Antti AU - Rhodius-Meester, Hanneke F. M. AU - Bruun, Marie AU - Koikkalainen, Juha AU - Barkhof, Frederik AU - Lemstra, Afina W. AU - Koene, Teddy AU - Scheltens, Philip AU - Teunissen, Charlotte E. AU - Tong, Tong AU - Guerrero, Ricardo AU - Schuh, Andreas AU - Ledig, Christian AU - Baroni, Marta AU - Rueckert, Daniel AU - Soininen, Hilkka AU - Remes, Anne M. AU - Waldemar, Gunhild AU - Hasselbalch, Steen G. AU - Mecocci, Patrizia AU - Van der Flier, Wiesje M. AU - Lötjönen, Jyrki PY - 2018 JO - Frontiers in aging neuroscience VL - 10 SN - 1663-4365 DO - 10.3389/fnagi.2018.00111 UR - https://fis.uni-bamberg.de/handle/uniba/54997 LA - eng ER - TY - JOUR TI - Structural brain imaging in Alzheimer's disease and mild cognitive impairment: biomarker analysis and shared morphometry database AU - Ledig, Christian AU - Schuh, Andreas AU - Guerrero, Ricardo AU - Heckemann, Rolf A. AU - Rueckert, Daniel PY - 2018 JO - Scientific reports VL - 8 IS - 1 SN - 2045-2322 DO - 10.1038/s41598-018-29295-9 UR - https://fis.uni-bamberg.de/handle/uniba/54996 LA - eng ER - TY - CHAP TI - Semantic Parsing of Brain MR Images AU - Ledig, Christian AU - Rueckert, Daniel PY - 2016 SP - 307-335 PB - Elsevier CY - Amsterdam SN - 9780128025819 DO - 10.1016/B978-0-12-802581-9.00014-7 UR - https://fis.uni-bamberg.de/handle/uniba/55001 LA - deu ER - TY - CPAPER TI - Hippocampal atrophy rate using an expectation maximization classifier with a disease-specific prior AU - Lötjönen, Jyrki AU - Wolz, Robin AU - Koikkalainen, Juha AU - Manna, Valeria AU - Ledig, Christian AU - Thurfjell, Lennart AU - Lundqvist, Roger AU - Waldemar, Gunhild AU - Soininen, Hilkka AU - Rueckert, Daniel PY - 2012 SP - 1164-1167 DO - 10.1109/ISBI.2012.6235767 UR - https://fis.uni-bamberg.de/handle/uniba/58622 LA - eng ER -