Skip to content

Read-only version: For security reasons, the Research Information System (FIS) has been temporarily replaced by this read-only version. All public content and full texts remain available; login, editing, submission and statistics are not available. Questions: fis@uni-bamberg.de

Home › Publications › A Three-Level Evaluation Protocoll for Acoustic Scene Understanding of Large Language Audio Models

Publication

A Three-Level Evaluation Protocoll for Acoustic Scene Understanding of Large Language Audio Models

Dilip, Harish; Abeßer, Jakob (2025): A Three-Level Evaluation Protocoll for Acoustic Scene Understanding of Large Language Audio Models, in: Emmanouil Benetos, Frederic Font, Magdalena Fuentes, u. a. (Hrsg.), Proceedings of the 10th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2025), October 2025, zenodo, S. 185–189, doi: 10.5281/zenodo.17251589.

Details

Faculty/Chair
Faculty/Professorship
Author
Dilip, Harish
Title of the compilation
Proceedings of the 10th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2025), October 2025
Editors
Benetos, Emmanouil
Font, Frederic
Fuentes, Magdalena
Martin Morato, Irene
Rocamora, Martín
Conference
10th Workshop on Detection and Classification of Acoustic Scenes and Events, DCASE 2025, Barcelona
Publisher Information
zenodo
Year of publication
2025
ISBN
978-84-09-77652-8
Language
English
DOI
Abstract

Reaching a semantic understanding of complex acoustic scenes requires computational models to capture the temporal-spatial sound source composition as well as individual sound events. This is a great challenge for computational models due to the large variety of everyday sound events and the extensive temporal-spectral overlap in real-life acoustic scenes. In this work, we aim to evaluate the acoustic scene understanding capabilities of two large audio-language models (LALMs). As a challenging scenario, we use the USM dataset, which features synthetic urban soundscapes with 2-6 overlapping sound sources per mixture. Our main contribution is a novel three-layer evaluation protocol, which includes four analysis tasks for low-level sound event understanding (sound event tagging), mid-level understanding and reasoning (sound polyphony estimation, sound source loudness ranking), as well as high-level scene understanding (audio captioning). We apply standardized metrics to assess the model's performances for each task. The proposed mulit-layer protocol allows for a fine-grained analysis of model behavior across soundscapes of various complexity levels. Our results indicate that despite their rmarkable controllability using textual instructions, the ability of state-of-the-art LALMs to understand acoustic scenes is still limited as the performance on individual analysis tasks desgrades with increasing sound polyphony.

Keywords
audio captioning
large audio-language models
sound event tagging
sound polyphony estimation
sound source loudness ranking
Peer Reviewed
yes
International Distribution
yes
Type
Conferenceobject
URI
Activation date
2025-11-24

Permalink: https://fis.uni-bamberg.de/handle/uniba/111565 · Data as JSON · Cite: BibTeX · RIS

Data as of: 2026-11-16

Restricted read-only access: login, editing and submission are temporarily disabled.