Skip to content

Read-only version: For security reasons, the Research Information System (FIS) has been temporarily replaced by this read-only version. All public content and full texts remain available; login, editing, submission and statistics are not available. Questions: fis@uni-bamberg.de

Home › Publications › Emotion Ratings : How Intensity, Annotation Confidence and Agreements are Entangled

Publication

Emotion Ratings : How Intensity, Annotation Confidence and Agreements are Entangled

Troiano, Enrica; Padó, Sebastian; Klinger, Roman (2025): Emotion Ratings : How Intensity, Annotation Confidence and Agreements are Entangled, in: Bamberg: Otto-Friedrich-Universität, S. 40–49.

Details

Faculty/Chair
Faculty/Professorship
Author
Troiano, Enrica
Padó, Sebastian
Publisher Information
Otto-Friedrich-Universität : Bamberg
Year of publication
2025
Source/Other editions
Orphée De Clercq, Alexandra Balahur, João Sedoc, u. a. (Hrsg.), Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, Association for Computational Linguistics, 2021, S. 40–49
Year of first publication
2021
Language
English
Licence
URN
Abstract

When humans judge the affective content of texts, they also implicitly assess the correctness of such judgment, that is, their confidence. We hypothesize that people’s (in)confidence that they performed well in an annotation task leads to (dis)agreements among each other. If this is true, confidence may serve as a diagnostic tool for systematic differences in annotations. To probe our assumption, we conduct a study on a subset of the Corpus of Contemporary American English, in which we ask raters to distinguish neutral sentences from emotion-bearing ones, while scoring the confidence of their answers. Confidence turns out to approximate inter-annotator disagreements. Further, we find that confidence is correlated to emotion intensity: perceiving stronger affect in text prompts annotators to more certain classification performances. This insight is relevant for modelling studies of intensity, as it opens the question wether automatic regressors or classifiers actually predict intensity, or rather human’s self-perceived confidence.

GND Keywords
Computerlinguistik
Emotion
Keywords
Emotion Ratings
DDC Classification
004 Computer science
RVK Classification
ST 306
Type
Conferenceobject
URI
Activation date
2025-05-30
Project(s)

Permalink: https://fis.uni-bamberg.de/handle/uniba/108340 · Data as JSON · Cite: BibTeX · RIS

Data as of: 2026-11-16

Restricted read-only access: login, editing and submission are temporarily disabled.