Skip to content

Read-only version: For security reasons, the Research Information System (FIS) has been temporarily replaced by this read-only version. All public content and full texts remain available; login, editing, submission and statistics are not available. Questions: fis@uni-bamberg.de

Home › Publications › Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Publication

Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Doerrich, Sebastian; Archut, Tobias; Di Salvo, Francesco; u. a. (2024): Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification, in: Bamberg: Otto-Friedrich-Universität.

Details

Faculty/Chair
Faculty/Professorship
Author
Archut, Tobias
By
Doerrich, Sebastian; ...
Publisher Information
Otto-Friedrich-Universität : Bamberg
Year of publication
2024
Pages
6
Source/Other editions
2024 IEEE International Symposium on Biomedical Imaging (ISBI) / IEEE, 2024, S. 1–5.
Year of first publication
2024
Language
English
Licence
URN
Abstract

Traditional deep learning models implicitly encode knowledge limiting their transparency and ability to adapt to data changes. Yet, this adaptability is vital for addressing user data privacy concerns. We address this limitation by storing embeddings of the underlying training data independently of the model weights, enabling dynamic data modifications without retraining. Specifically, our approach integrates the k-Nearest Neighbor (k-NN) classifier with a vision-based foundation model, pre-trained self-supervised on natural images, enhancing interpretability and adaptability. We share open-source implementations of a previously unpublished baseline method as well as our performance-improving contributions. Quantitative experiments confirm improved classification across established benchmark datasets and the method’s applicability to distinct medical image classification tasks. Additionally, we assess the method’s robustness in continual learning and data removal scenarios. The approach exhibits great promise for bridging the gap between foundation models’ performance and challenges tied to data privacy. The source code is available at github.com/TobArc.

GND Keywords
Neuronales Netz
Klassifikator <Informatik>
Maschinelles Lernen
Explainable Artificial Intelligence
Keywords
k-NN classifier
continual learning
transfer learning
few-shot classification
explainability
DDC Classification
004 Computer science
RVK Classification
ST 301
Peer Reviewed
yes
International Distribution
yes
Open Access Journal
no
Type
Conferenceobject
URI
Activation date
2024-11-22

Permalink: https://fis.uni-bamberg.de/handle/uniba/104752 · Data as JSON · Cite: BibTeX · RIS

Data as of: 2026-11-16

Restricted read-only access: login, editing and submission are temporarily disabled.