TY - CPAPER TI - Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs AU - Greschner, Lynn AU - Klinger, Roman PY - 2026 T2 - Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities SP - 628–646 PB - Otto-Friedrich-Universität CY - Bamberg SN - 979-8-89176-234-3 UR - https://fis.uni-bamberg.de/handle/uniba/115224 LA - eng AB - Arguments evoke emotions, influencing the effect of the argument itself. Not only the emotional intensity but also the category influences the argument’s effects, for instance, the willingness to adapt stances. While binary emotionality has been studied in argumentative texts, there is no work on discrete emotion categories (e.g., ‘anger’) in such data. To fill this gap, we crowdsource subjective annotations of emotion categories in a German argument corpus and evaluate automatic LLM-based labeling methods. Specifically, we compare three prompting strategies (zero-shot, one-shot, chain-of-thought) on three large instruction-tuned language models (Falcon-7b-instruct, Llama-3.1-8B-instruct, GPT-4o-mini). We further vary the definition of the output space to be binary (is there emotionality in the argument?), closed-domain (which emotion from a given label set is in the argument?), or open-domain (which emotion is in the argument?). We find that emotion categories enhance the prediction of emotionality in arguments, emphasizing the need for discrete emotion annotations in arguments. Across all prompt settings and models, automatic predictions show a high recall but low precision for predicting anger and fear, indicating a strong bias toward negative emotions. KW - argument KW - emotion KW - corpus KW - large language model ER - TY - CPAPER TI - Trust Me, I Can Convince You: The Contextualized Argument Appraisal Framework and the ContArgA Corpus AU - Greschner, Lynn AU - Weber, Sabine AU - Klinger, Roman PY - 2026 T2 - Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026) SP - 8327–8346 PB - European Languages Resources Association (ELRA) CY - Paris SN - 978-2-493814-49-4 DO - 10.63317/484rpnvebop5 UR - https://fis.uni-bamberg.de/handle/uniba/115018 LA - eng ER - TY - CPAPER TI - Categorical Emotions or Appraisals: Which Emotion Model Explains Argument Convincingness Better? AU - Greschner, Lynn AU - Bauer, Meike AU - Weber, Sabine AU - Klinger, Roman PY - 2026 T2 - Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026) SP - 8190–8203 PB - European Languages Resources Association (ELRA) CY - Paris SN - 978-2-493814-49-4 DO - 10.63317/3vrvrgvtnvhn UR - https://fis.uni-bamberg.de/handle/uniba/115020 LA - eng ER - TY - CPAPER TI - Less Is More?: The Role of Demographic Author Information in Emotion Classification of Ambiguous Text AU - Weber, Sabine AU - Greschner, Lynn AU - Klinger, Roman PY - 2026 T2 - Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026) SP - 8147–8161 PB - European Languages Resources Association (ELRA) CY - Paris SN - 978-2-493814-49-4 DO - 10.63317/2cw8tpo82h55 UR - https://fis.uni-bamberg.de/handle/uniba/115023 LA - eng ER - TY - CPAPER TI - Says Who? : Argument Convincingness and Reader Stance Are Correlated with Perceived Author Personality AU - Weber, Sabine AU - Greschner, Lynn AU - Klinger, Roman PY - 2026 T2 - The Proceedings for the 15th Workshop on Computational Approaches to Subjectivity, Sentiment Social Media Analysis (WASSA 2026) SP - 265–277 PB - Association for Computational Linguistics SN - 979-8-89176-378-4 UR - https://fis.uni-bamberg.de/handle/uniba/114414 LA - eng ER - TY - CPAPER TI - Emotionally Charged, Logically Blurred : AI-driven Emotional Framing Impairs Human Fallacy Detection AU - Chen, Yanran AU - Greschner, Lynn AU - Klinger, Roman AU - Klenk, Michael AU - Eger, Steffen PY - 2026 T2 - Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics SP - 6709–6732 PB - Association for Computational Linguistics SN - 979-8-89176-380-7 UR - https://fis.uni-bamberg.de/handle/uniba/114415 LA - eng ER - TY - CPAPER TI - Which Demographics do LLMs Default to During Annotation? AU - Schäfer, Johannes AU - Combs, Aidan AU - Bagdon, Christopher Doyle AU - Li, Jiahui AU - Probol, Nadine AU - Greschner, Lynn AU - Papay, Sean AU - Menchaca Resendiz, Yarik AU - Velutharambath, Aswathy AU - Wuehrl, Amelie AU - Weber, Sabine AU - Klinger, Roman PY - 2026 SP - 17331–17348 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/114413 LA - eng AB - Demographics and cultural background of annotators influence the labels they assign in text annotation – for instance, an elderly woman might find it offensive to read a message addressed to a “bro”, but a male teenager might find it appropriate. It is therefore important to acknowledge label variations to not underrepresent members of a society. Two research directions developed out of this observation in the context of using large language models (LLM) for data annotations, namely (1) studying biases and inherent knowledge of LLMs and (2) injecting diversity in the output by manipulating the prompt with demographic information. We combine these two strands of research and ask the question to which demographics an LLM resorts when no demographics is given. To answer this question, we evaluate which attributes of human annotators LLMs inherently mimic. Furthermore, we compare non-demographic conditioned prompts and placebo-conditioned prompts (e.g., “you are an annotator who lives in house number 5”) to demographics-conditioned prompts (“You are a 45 year old man and an expert on politeness annotation. How do you rate {instance}”). We study these questions for politeness and offensiveness annotations on the POPQUORN data set, a corpus created in a controlled manner to investigate human label variations based on demographics which has not been used for LLM-based analyses so far. We observe notable influences related to gender, race, and age in demographic prompting, which contrasts with previous studies that found no such effects KW - - ER - TY - CPAPER TI - QoLAS : A Reddit Corpus of Health-Related Quality of Life Aspects of Mental Disorders AU - Greschner, Lynn AU - Wührl, Amelie AU - Klinger, Roman PY - 2026 SP - 201–216 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/114409 LA - eng AB - ‘Quality of Life’ (QoL) refers to a person’s subjective perception of various aspects of their life. For medical practitioners, it is one of the most important concepts for treatment decisions. Therefore, it is essential to understand in which aspects a medical condition affects a patient’s subjective perception of their life. With this paper, we focus on the under-resourced domain of mental health-related QoL, and contribute the first corpus to study and model this concept: We (1) annotate 240 Reddit posts with a set of 11 QoL aspects (such as ‘independence’, ‘mood’, or ‘relationships’) and their sentiment polarity. Based on this novel corpus, we (2) evaluate a pipeline to detect QoL mentions and classify them into aspects using open domain aspect-based sentiment analysis. We find that users frequently discuss healthrelated QoL in their posts, focusing primarily on the aspects ‘relationships’ and ‘selfimage’. Our method reliably predicts such mentions and their sentiment, however, detecting finegrained individual aspects remains challenging. An analysis of a large corpus of automatically labeled data reveals that social media content contains novel aspects pertinent to patients that are not covered by existing QoL taxonomies KW - - ER - TY - CPAPER TI - IMS_medicALY at #SMM4H 2024 : Detecting Impacts of Outdoor Spaces on Social Anxiety with Data Augmented Ensembling AU - Wührl, Amelie AU - Greschner, Lynn AU - Menchaca Resendiz, Yarik AU - Klinger, Roman PY - 2025 SP - 83–87 PB - Otto-Friedrich-Universität CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/111077 LA - eng AB - Many individuals affected by Social Anxiety Disorder turn to social media platforms to share their experiences and seek advice. This includes discussing the potential benefits of engaging with outdoor environments. As part of #SMM4H 2024, Shared Task 3 focuses on classifying the effects of outdoor spaces on social anxiety symptoms in Reddit posts. In our contribution to the task, we explore the effectiveness of domain-specific models (trained on social media data – SocBERT) against general domain models (trained on diverse datasets – BERT, RoBERTa, GPT-3.5) in predicting the sentiment related to outdoor spaces. Further, we assess the benefits of augmenting sparse human-labeled data with synthetic training instances and evaluate the complementary strengths of domain-specific and general classifiers using an ensemble model. Our results show that (1) fine-tuning small, domain-specific models generally outperforms large general language models in most cases. Only one large language model (GPT-4) exhibits performance comparable to the fine-tuned models (52% F1). Further, we find that (2) synthetic data does improve the performance of fine-tuned models in some cases, and (3) models do not appear to complement each other in our ensemble setup. KW - Data Augmented Ensembling ER - TY - CPAPER TI - Which Demographics do LLMs Default to During Annotation? AU - Schäfer, Johannes AU - Combs, Aidan AU - Bagdon, Christopher Doyle AU - Li, Jiahui AU - Probol, Nadine AU - Greschner, Lynn AU - Papay, Sean AU - Menchaca Resendiz, Yarik AU - Velutharambath, Aswathy AU - Wuehrl, Amelie AU - Weber, Sabine AU - Klinger, Roman PY - 2025 T2 - Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics SP - 17331–17348 PB - Association for Computational Linguistics SN - 979-8-89176-251-0 DO - 10.18653/v1/2025.acl-long.848 UR - https://fis.uni-bamberg.de/handle/uniba/109541 LA - eng AB - Demographics and cultural background of annotators influence the labels they assign in text annotation – for instance, an elderly woman might find it offensive to read a message addressed to a “bro”, but a male teenager might find it appropriate. It is therefore important to acknowledge label variations to not underrepresent members of a society. Two research directions developed out of this observation in the context of using large language models (LLM) for data annotations, namely (1) studying biases and inherent knowledge of LLMs and (2) injecting diversity in the output by manipulating the prompt with demographic information. We combine these two strands of research and ask the question to which demographics an LLM resorts when no demographics is given. To answer this question, we evaluate which attributes of human annotators LLMs inherently mimic. Furthermore, we compare non-demographic conditioned prompts and placebo-conditioned prompts (e.g., “you are an annotator who lives in house number 5”) to demographics-conditioned prompts (“You are a 45 year old man and an expert on politeness annotation. How do you rate {instance}”). We study these questions for politeness and offensiveness annotations on the POPQUORN data set, a corpus created in a controlled manner to investigate human label variations based on demographics which has not been used for LLM-based analyses so far. We observe notable influences related to gender, race, and age in demographic prompting, which contrasts with previous studies that found no such effects KW - - ER - TY - CPAPER TI - QoLAS : A Reddit Corpus of Health-Related Quality of Life Aspects of Mental Disorders AU - Greschner, Lynn AU - Wührl, Amelie AU - Klinger, Roman PY - 2025 T2 - Proceedings of the 24th Workshop on Biomedical Language Processing SP - 201–216 PB - Association for Computational Linguistics SN - 979-8-89176-275-6 DO - 10.18653/v1/2025.bionlp-1.18 UR - https://fis.uni-bamberg.de/handle/uniba/109545 LA - eng AB - ‘Quality of Life’ (QoL) refers to a person’s subjective perception of various aspects of their life. For medical practitioners, it is one of the most important concepts for treatment decisions. Therefore, it is essential to understand in which aspects a medical condition affects a patient’s subjective perception of their life. With this paper, we focus on the under-resourced domain of mental health-related QoL, and contribute the first corpus to study and model this concept: We (1) annotate 240 Reddit posts with a set of 11 QoL aspects (such as ‘independence’, ‘mood’, or ‘relationships’) and their sentiment polarity. Based on this novel corpus, we (2) evaluate a pipeline to detect QoL mentions and classify them into aspects using open domain aspect-based sentiment analysis. We find that users frequently discuss healthrelated QoL in their posts, focusing primarily on the aspects ‘relationships’ and ‘selfimage’. Our method reliably predicts such mentions and their sentiment, however, detecting finegrained individual aspects remains challenging. An analysis of a large corpus of automatically labeled data reveals that social media content contains novel aspects pertinent to patients that are not covered by existing QoL taxonomies KW - - ER - TY - CPAPER TI - Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs AU - Greschner, Lynn AU - Klinger, Roman PY - 2025 T2 - Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities SP - 628–646 PB - Association for Computational Linguistics (ACL) CY - Stroudsburg, PA SN - 979-8-89176-234-3 DO - 10.18653/v1/2025.nlp4dh-1.52 UR - https://fis.uni-bamberg.de/handle/uniba/107836 LA - eng AB - Arguments evoke emotions, influencing the effect of the argument itself. Not only the emotional intensity but also the category influences the argument’s effects, for instance, the willingness to adapt stances. While binary emotionality has been studied in argumentative texts, there is no work on discrete emotion categories (e.g., ‘anger’) in such data. To fill this gap, we crowdsource subjective annotations of emotion categories in a German argument corpus and evaluate automatic LLM-based labeling methods. Specifically, we compare three prompting strategies (zero-shot, one-shot, chain-of-thought) on three large instruction-tuned language models (Falcon-7b-instruct, Llama-3.1-8B-instruct, GPT-4o-mini). We further vary the definition of the output space to be binary (is there emotionality in the argument?), closed-domain (which emotion from a given label set is in the argument?), or open-domain (which emotion is in the argument?). We find that emotion categories enhance the prediction of emotionality in arguments, emphasizing the need for discrete emotion annotations in arguments. Across all prompt settings and models, automatic predictions show a high recall but low precision for predicting anger and fear, indicating a strong bias toward negative emotions. KW - argument KW - emotion KW - corpus KW - large language model ER - TY - CPAPER TI - IMS_medicALY at #SMM4H 2024 : Detecting Impacts of Outdoor Spaces on Social Anxiety with Data Augmented Ensembling AU - Wührl, Amelie AU - Greschner, Lynn AU - Menchaca Resendiz, Yarik AU - Klinger, Roman PY - 2024 T2 - Proceedings of The 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks SP - 83–87 PB - Association for Computational Linguistics CY - Bangkok, Thailand UR - https://fis.uni-bamberg.de/handle/uniba/98196 LA - eng AB - Many individuals affected by Social Anxiety Disorder turn to social media platforms to share their experiences and seek advice. This includes discussing the potential benefits of engaging with outdoor environments. As part of #SMM4H 2024, Shared Task 3 focuses on classifying the effects of outdoor spaces on social anxiety symptoms in Reddit posts. In our contribution to the task, we explore the effectiveness of domain-specific models (trained on social media data – SocBERT) against general domain models (trained on diverse datasets – BERT, RoBERTa, GPT-3.5) in predicting the sentiment related to outdoor spaces. Further, we assess the benefits of augmenting sparse human-labeled data with synthetic training instances and evaluate the complementary strengths of domain-specific and general classifiers using an ensemble model. Our results show that (1) fine-tuning small, domain-specific models generally outperforms large general language models in most cases. Only one large language model (GPT-4) exhibits performance comparable to the fine-tuned models (52% F1). Further, we find that (2) synthetic data does improve the performance of fine-tuned models in some cases, and (3) models do not appear to complement each other in our ensemble setup. KW - Data Augmented Ensembling ER -