Zum Inhalt springen

Nur-Lesen-Fassung: Das Forschungsinformationssystem (FIS) ist aus Sicherheitsgründen vorübergehend durch diese Nur-Lesen-Fassung ersetzt. Alle öffentlichen Inhalte und Volltexte sind weiterhin abrufbar; Anmeldung, Bearbeitung, Einreichung und Statistiken stehen nicht zur Verfügung. Bei Fragen: fis@uni-bamberg.de

Start › Publikationen › A Three-Level Evaluation Protocoll for Acoustic Scene Understanding of Large Language Audio Models

Publikation

A Three-Level Evaluation Protocoll for Acoustic Scene Understanding of Large Language Audio Models

Dilip, Harish; Abeßer, Jakob (2025): A Three-Level Evaluation Protocoll for Acoustic Scene Understanding of Large Language Audio Models, in: Emmanouil Benetos, Frederic Font, Magdalena Fuentes, u. a. (Hrsg.), Proceedings of the 10th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2025), October 2025, zenodo, S. 185–189, doi: 10.5281/zenodo.17251589.

Details

Fakultät/Lehrstuhl
Autorin(nen)/Autor(en)
Dilip, Harish
Titel des Sammelwerks
Proceedings of the 10th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2025), October 2025
Herausgeberin(nen)/Herausgeber des Sammelwerks
Benetos, Emmanouil
Font, Frederic
Fuentes, Magdalena
Martin Morato, Irene
Rocamora, Martín
Konferenz
10th Workshop on Detection and Classification of Acoustic Scenes and Events, DCASE 2025, Barcelona
Verlagsangabe
zenodo
Erscheinungsjahr
2025
ISBN
978-84-09-77652-8
Sprache(n)
Englisch
DOI
Abstract

Reaching a semantic understanding of complex acoustic scenes requires computational models to capture the temporal-spatial sound source composition as well as individual sound events. This is a great challenge for computational models due to the large variety of everyday sound events and the extensive temporal-spectral overlap in real-life acoustic scenes. In this work, we aim to evaluate the acoustic scene understanding capabilities of two large audio-language models (LALMs). As a challenging scenario, we use the USM dataset, which features synthetic urban soundscapes with 2-6 overlapping sound sources per mixture. Our main contribution is a novel three-layer evaluation protocol, which includes four analysis tasks for low-level sound event understanding (sound event tagging), mid-level understanding and reasoning (sound polyphony estimation, sound source loudness ranking), as well as high-level scene understanding (audio captioning). We apply standardized metrics to assess the model's performances for each task. The proposed mulit-layer protocol allows for a fine-grained analysis of model behavior across soundscapes of various complexity levels. Our results indicate that despite their rmarkable controllability using textual instructions, the ability of state-of-the-art LALMs to understand acoustic scenes is still limited as the performance on individual analysis tasks desgrades with increasing sound polyphony.

Freie Schlagwort(e)
audio captioning
large audio-language models
sound event tagging
sound polyphony estimation
sound source loudness ranking
Peer Reviewed
ja
Internationale Verbreitung
ja
Dokumenttyp
Konferenzveröffentlichung
URI
Datum der Freischaltung
24.11.2025

Permalink: https://fis.uni-bamberg.de/handle/uniba/111565 · Daten als JSON · Zitieren: BibTeX · RIS

Datenstand: 15.11.2026

Eingeschränkter Lesezugriff: Anmeldung, Bearbeitung und Einreichung sind vorübergehend deaktiviert.