Skip to content

Read-only version: For security reasons, the Research Information System (FIS) has been temporarily replaced by this read-only version. All public content and full texts remain available; login, editing, submission and statistics are not available. Questions: fis@uni-bamberg.de

Home › Publications › Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs

Publication

Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs

Greschner, Lynn; Klinger, Roman (2026): Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs, in: Mika Hämäläinen, Emily Öhman, Yuri Bizzoni, u. a. (Hrsg.), Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities, Bamberg: Otto-Friedrich-Universität, S. 628–646.

Details

Faculty/Chair
Faculty/Professorship
Author
Title of the compilation
Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities
Editors
Hämäläinen, Mika
Öhman, Emily
Bizzoni, Yuri
Miyagawa, So
Alnajjar, Khalid
Conference
International Conference on Natural Language Processing for Digital Humanities, Albuquerque, USA
Publisher Information
Otto-Friedrich-Universität : Bamberg
Year of publication
2026
ISBN
979-8-89176-234-3
Source/Other editions
Mika Hämäläinen, Emily Öhman, Yuri Bizzoni, u. a. (Hrsg.), Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities, Stroudsburg, PA: Association for Computational Linguistics (ACL), 2025, S. 628–646, ISBN: 979-8-89176-234-3
Year of first publication
2025
Language
English
Licence
URN
Abstract

Arguments evoke emotions, influencing the effect of the argument itself. Not only the emotional intensity but also the category influences the argument’s effects, for instance, the willingness to adapt stances. While binary emotionality has been studied in argumentative texts, there is no work on discrete emotion categories (e.g., ‘anger’) in such data. To fill this gap, we crowdsource subjective annotations of emotion categories in a German argument corpus and evaluate automatic LLM-based labeling methods. Specifically, we compare three prompting strategies (zero-shot, one-shot, chain-of-thought) on three large instruction-tuned language models (Falcon-7b-instruct, Llama-3.1-8B-instruct, GPT-4o-mini). We further vary the definition of the output space to be binary (is there emotionality in the argument?), closed-domain (which emotion from a given label set is in the argument?), or open-domain (which emotion is in the argument?). We find that emotion categories enhance the prediction of emotionality in arguments, emphasizing the need for discrete emotion annotations in arguments. Across all prompt settings and models, automatic predictions show a high recall but low precision for predicting anger and fear, indicating a strong bias toward negative emotions.

Keywords
argument
emotion
corpus
large language model
Peer Reviewed
yes
International Distribution
yes
Open Access Journal
yes
Type
Conferenceobject
URI
Activation date
2026-05-22
Project(s)

Permalink: https://fis.uni-bamberg.de/handle/uniba/115224 · Data as JSON · Cite: BibTeX · RIS

Data as of: 2026-11-16

Restricted read-only access: login, editing and submission are temporarily disabled.