TY - THES TI - Disengaged Responding in Educational Large-Scale Assessments : New Insights into its Identification, Modeling, and Consequences AU - Welling, Jana PY - 2026 PB - Otto-Friedrich-Universität CY - Bamberg DO - 10.20378/irb-116761 UR - https://fis.uni-bamberg.de/handle/uniba/116761 LA - eng AB - Educational large-scale assessments (LSAs) provide information on competencies across cohorts, populations, and test domains. However, when examinees lack test-taking motivation, they can respond to test items without investing their best effort, knowledge, and abilities, a behavior called disengaged responding. Because disengaged responses do not reflect the proficiency of the examinees, these responses can pose a threat to the validity of assessment outcomes. To ensure the interpretability of findings from LSAs, disengaged responding needs to be accurately identified and accounted for in the measurement of competencies. Moreover, it is important to understand in which contexts disengaged responding arises and which consequences it entails. Therefore, this dissertation includes three articles that addressed major, yet unresolved issues in the context of disengaged responding concerning its identification (Article 1), accounting for it in the measurement of competencies (Article 2), and the generalizability of established findings to child samples (Article 3). Most approaches to identifying disengaged responding rely solely on item responses and response times, thereby risking the misclassification of fast engaged or slow disengaged responses. Article 1 aimed to improve the classification accuracy of response-time based identification methods by introducing and evaluating three novel process data indicators of disengaged responding in reading comprehension tasks: text reread, item revisit, and answer change. Findings from an empirical study suggest that the improvement in classification accuracy was marginal to non-existent, thereby confirming response times as an important indicator of disengagement. Although many LSAs provide proficiency estimates as plausible values, no study has yet investigated how disengagement should be accounted for in their estimation. Article 2 addressed this gap by proposing and evaluating different approaches to (not) accounting for disengaged responding in the estimation of plausible values. A simulation study demonstrated that achievement gaps estimated with plausible values can be biased by disengaged responding. While models that accounted for disengagement in the measurement model effectively reduced these biases, models accounting for it solely in the population model did not. An additional empirical study found no difference between the models, supporting recent studies which claim that the impact of disengaged responding on achievement gaps may be negligible in practice. Most empirical findings on disengaged responding are based on adolescent or adult samples, ignoring the fact that many LSAs also include child samples. Article 3 challenged the generalizability of established findings on disengaged responding to child samples by providing theoretical considerations on why the test-taking behavior of children may differ from that of older examinees. Additionally, the findings from an empirical study supported recent research indicating that disengaged responding may be less common among children. Only proficiency could be replicated as predictor of disengaged responding, while gender and socio-economic status could not. Further results suggested that the impact of disengaged responding on achievement gaps may be limited in this age group. Taken together, this dissertation combines theoretical, methodological, and empirical approaches to improve our understanding of disengaged responding and the validity of LSAs. The central findings bear important implications for the identification, modeling, and consequences of disengaged responding. First, they support response-time based measures as strong indicators of disengagement while suggesting that process data may not always provide sufficient information on test-taking engagement to inform new identification methods. Second, they highlight the need to accurately account for disengaged responding also in plausible value estimation. Third, this dissertation suggests that and discusses why the threat (rapid) disengagement poses to the validity of group-based assessment outcomes may be less pronounced than often assumed. Fourth, it challenges the generalizability of established findings to child samples. By laying conceptual and methodological groundwork, this dissertation provides a foundation for future research aimed at developing more robust, valid, and context-sensitive approaches to measuring competencies in educational LSAs. KW - disengaged responding KW - rapid guessing KW - test-taking engagement KW - test-taking behavior KW - large-scale assessments KW - process data ER - TY - THES TI - Die Abbildung latenter Merkmale im individuellen Entwicklungsverlauf : Vergleich von IRT-Linkmethoden unter Verwendung Rasch-skalierter Kompetenztestdaten AU - Fischer, Luise PY - 2023 PB - Otto-Friedrich-Universität CY - Bamberg DO - 10.20378/irb-91014 UR - https://fis.uni-bamberg.de/handle/uniba/91014 LA - other AB - Die vorliegende Forschungsarbeit möchte die Psychologie in ihrem Bestreben unterstützen, das Erleben und Verhalten des Menschen über die gesamte Lebensspanne abzubilden. Sie erweitert den Forschungsstand betreffend die Modellierung kognitiver Entwicklungsverläufe und damit einhergehende Herausforderungen wie z.B. starke Fähigkeitsveränderung zwischen Messzeitpunkten, kurze Testlänge, geringe Anzahl an Ankeritems und moderat eingeschränkte Modellgeltung. Das Verlinken angrenzender Messzeitpunkte erfolgt im Rahmen von Item Response Theory und dem Rasch Modell. Konkret liegen drei Beiträge vor: In zwei Beiträgen (Fischer, Gnambs, Rohm, & Carstensen, 2019, Fischer, Rohm, Carstensen, & Gnambs, 2021) werden zwei Linkdesigns (Ankeritem, Ankergruppe) und vier im Querschnittdesign etablierte IRT-Linkmethoden (Concurrent calibration, Fixed Parameter Calibration, mean/mean linking, weighted mean/mean linking) anhand von empirischen als auch simulierten Daten untersucht und in ihrer Performanz verglichen. Ein dritter Beitrag ist schließlich stärker praxisorientiert: Der korrekte Prozess des Verlinkens wird anschaulich dargestellt und in einer Schritt-für-Schritt Anleitung für weniger methodisch versierte Personen nachvollziehbar beschrieben. Während die vorliegende Forschungsarbeit nur marginale Unterschiede zwischen drei IRT-Linkmethoden (Fixed Parameter Calibration, mean/mean linking, weighted mean/mean linking) gefunden hat, so zeigten sich massive Konvergenzprobleme bei der Concurrent Calibration. Deutlich hervorzuheben sind die jeweils unterschiedlichen Herausforderungen der zwei gegenübergestellten Linkdesigns. So muss bei einem Ankeritem-Design besonderes Augenmerk auf einen möglichen systematischen Stichprobenausfall im Panel sowie auf Modellverletzungen durch abweichende Itemparameter von Ankeritems gelegt werden. Es haben sich keine Anhaltspunkte für Erinnerungseffekte durch die wiederholte Vorgabe von Items ergeben. Bei einem Ankergruppe-Design ist hingegen die Repräsentativität der Ankergruppe sowie – bedingt durch eine geringere Stichprobengröße – der Stichprobenfehler entscheidend für die Reliabilität und Validität des Linkergebnisses. Zusammenfassend verdeutlicht die vorliegende Forschungsarbeit die Bedeutung einer guten Modellpassung von Ankeritems sowie einem guten test targeting, um den systematischen Fehler im Linkergebnis beim Ankeritem-Design gering zu halten. KW - Linking KW - IRT KW - Rasch model KW - longitudinal KW - simulation study ER - TY - GEN TI - Theorie und Messung in der Psychologie : Eine evolutionäre Perspektive AU - Borgstede, Matthias PY - 2022 PB - University of Bamberg Press CY - Bamberg T3 - Schriften aus der Fakultät Humanwissenschaften der Otto-Friedrich-Universität Bamberg SN - 978-3-86309-892-6 DO - 10.20378/irb-56781 UR - https://fis.uni-bamberg.de/handle/uniba/56781 LA - deu AB - In dieser Habilitationsschrift geht es um die wissenschaftstheoretische, methodische und inhaltliche Bedeutung formaler Modellierung in der Psychologie. Vor dem Hintergrund einer semantischen Theoriekonzeption wird argumentiert, dass valide psychologische Messoperationen nicht unabhängig von substanzieller formaler Theorie definiert werden können. Theoriebildung wird dabei nicht als statistische Modellierung, sondern als Formalisierung allgemeiner theoretischer Prinzipien verstanden. Als möglicher Ausgangspunkt für eine derartige formale Theoriebildung in der Psychologie wird ein evolutionstheoretischer Rahmen für die Erklärung individueller Verhaltensanpassungen an wechselnde Umweltbedingungen vorgeschlagen. Lernen und Evolution sind nach dieser Auffassung keine Gegensätze, sondern integrale Bestandteile eines allgemeinen Selektionsprozesses. KW - Theoriebildung KW - Psychologische Messung KW - Evolutionstheorie KW - Verhaltensselektion KW - Formale Modellierung KW - Theory building KW - Psychological measurement KW - Evolutionary theory KW - Behavioral selection KW - Formal modeling ER - TY - THES TI - Analyzing the context of large-scale educational assessments using multilevel latent variable modeling AU - Rohm, Theresa PY - 2022 PB - Otto-Friedrich-Universität CY - Bamberg DO - 10.20378/irb-55904 UR - https://fis.uni-bamberg.de/handle/uniba/55904 LA - eng AB - Large-scale assessments in education often involve the measurement of latent competence, with the aim to perform comparisons between groups in subsequent analyses. The assessment of latent competence is thereby often performed within the institutionalized context of schools, or with the help of interviewers visiting respondents’ homes. In addition, sample selection for educational assessments might focus on groups as sampling units. For example, German secondary school types are used as primary sampling units in a multi-stage sampling procedure in educational studies. Subsequently, several schools per school type are selected for student competence assessment and in consequence, students are nested within clusters of schools. Hence, hierarchical data structures in large-scale educational assessments are caused by contexts of assessments, the sampling design or the combination of both. Regarding the resulting measures of latent competence, construct-irrelevant variance could be introduced by the context of educational assessments in the form of variance between clusters. This can potentially reduce the quality of measurement and impair a fair and valid interpretation of test scores. Further uncertainty in latent competence assessment arises by using as set of item indicators to measure the latent trait of interest. Besides the overall latent trait, also single items can vary by assessment contexts, indicated by random item effects that denote item variation across clusters. In addition, item differences can occur between groups (i.e., fixed item effects) and, if severe, indicate measurement non-invariance in the form of differential item functioning between groups. Item bias is indicated when item differences between groups favor a particular group. If item bias is found between groups that were used as sampling units (e.g., school type), item bias is associated with the context of competence assessment. Hence, group differences of items are then found on the cluster-level of the hierarchical data structure. For the detection of such sources of construct-irrelevant variance stemming from the assessment context, as well as item bias and item variances associated with the assessment context, multilevel latent variable models are presented and applied to competence and cognitive ability assessments. Competence measurements in the domains mathematic and reading are examined, as well as a measure of the cognitive ability perceptual speed, all assessed within the National Educational Panel Study. In the first study, interviewer and area clusters were investigated in an adult mathematic competence assessment, as hierarchical structures that might introduce construct-irrelevant variance. The examination of cross-classified multilevel item response theory models showed substantial interviewer variance in mathematic competence, while area effects were small. Subsequent analyses revealed some interviewers with undue influence, that were in addition associated with the respondents’ number of missing values in the assessed test and participation rates at the subsequent competence assessment. The second study investigated consequences of item bias, in the form of cluster-level group differences in items by school type, for students reading competence development from fifth to ninth grade of German secondary school. Measurement non-invariance occurred especially between the highest and lowest school type of German secondary schools at all measurement occasions. Nevertheless, the school type comparisons of reading competence development were not sensitive to found measurement non-invariance between school types and a parallel development of reading competence between German secondary school types was presented. In the third study, in addition to cluster-level group differences in item estimates by school type (i.e., fixed item effects), also random item effects across school clusters per school type were investigated for three items measuring perceptual speed for German secondary school students at ninth grade. Fixed- and random-group differential item functioning was investigated in comparison of students from several types of regular schools and students with special educational needs. Random-group differential item functioning was found for two out of the three items, indicating that estimated item difficulties differed across school clusters. Such differences across assessment contexts (i.e., school clusters) might stem from problems of standardized test assessment. Finally, the results of the three studies are discussed with regard to test standards for educational and psychological testing. The results are furthermore compared to empirical evaluations of context effects in other educational large-scale assessments. KW - assessment context, item response theory, multilevel latent variable modeling KW - large-scale educational assessments, National Educational Panel Study ER - TY - THES TI - Is There a Self-Regulated Learning Model in East Asian Educational Systems? : a Cross-Sectional Study Based on PISA 2012 AU - Wu, Yi-Jhen PY - 2019 PB - Otto-Friedrich-Universität CY - Bamberg DO - 10.20378/irb-46617 UR - https://fis.uni-bamberg.de/handle/uniba/46617 LA - eng AB - The common stereotype of East Asian learning behavior is that East Asian student use memorization extensively, which is called rote-learning behavior. However, East Asian students' performance has been consistently ranked at the top on international large-scale assessments such as Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). Such striking performance is unlikely to be attributable to rote-learning behavior. In this dissertation, we aimed to disentangle East Asian students' learning behavior by drawing on PISA 2012 with the following aims: (1) the frequency of use of memorization among East Asian students, (2) the relationships between learning strategies and mathematics performance, (3) the relationships between self-efficacy and learning strategies, (4) the relationships between intrinsic motivation, extrinsic motivation, and learning strategies. To investigate the frequency of use of memorization, latent class analysis (LCA) were fitted to seven East Asian educational systems (i.e. Hong Kong, Japan, Korea, Macau, Shanghai of China, Singapore, and Taiwan) as well as the comparisons of the use of memorization across cultures, namely Taiwan and America by using the multiple-group LCA. This showed that East Asian students reported more control strategies, yet less memorization; only a few Taiwanese students (5%) reported memorization and more American students (19%) reported memorization compared to Taiwanese students. Examining other factors, performance, self-efficacy, and motivation, related to learning strategies, showed that when East Asian students have lower performance, lower self-efficacy or lower motivation, they tend to report the use of memorization. In contrast, when students had higher performance, higher self-efficacy or higher motivation, they were likely to report control strategies. In sum, we examined different angles to understand East Asian students' learning behavior by using PISA 2012 within a self-regulated learning framework. It is worth noting that the serotype of East Asian learning behavior was challenged. Moreover, it seemed that East Asian students' self-regulated learning behavior was in line with the self-regulated learning theory (SRL). Implications of these findings are discussed. KW - Learning Strategies, Self-efficacy, Intrinsic Motivation, Extrinsic Motivation, LCA, PISA ER - TY - THES TI - Leserlichkeit der Handschrift von Schreibanfängern : eine empirische Studie zur Erfassung und Bedeutung der Leserlichkeit AU - Rüb, Angelika PY - 2018 PB - University of Bamberg Press CY - Bamberg T3 - Schriften aus der Fakultät Humanwissenschaften der Otto-Friedrich-Universität Bamberg SN - 978-3-86309-597-0 DO - 10.20378/irbo-52476 UR - https://fis.uni-bamberg.de/handle/uniba/44172 LA - deu AB - Die Leserlichkeit der Handschrift – als ein Merkmal des Produkts der graphomotorischen Fähigkeiten – wurde in ihrer Bedeutung für zahlreichen Facetten der Schreibkompetenz im Anfangsunterricht bislang unzureichend untersucht, obgleich theoretisch konzipierte und empirisch begründete Argumentationen für eine besondere Beachtung zu Beginn des Schreiblernprozesses plädieren. Auch fordern die Bildungsstandards im Fach Deutsch für den Primarbereich: Die Kinder sollen „eine gut lesbare [präzise: leserliche; A.R.] Handschrift flüssig schreiben“ (KMK 2004, S. 10). Doch handhabbare und gleichzeitig empirisch überprüfte Evaluationsmethoden fehlen. Die Forschungsarbeit umfasst drei Fragestellungen: (1) Lässt sich ein für Forschung und Unterrichtspraxis handhabbares Instrument für die Erfassung der Leserlichkeit von Schülerhandschriften anhand freier Texte im Anfangsunterricht entwickeln und zufriedenstellend evaluieren? (2) Inwiefern und in welchem Maß beeinflussen individuelle und familiäre sowie didaktische und schulbezogene Bedingungen die Leserlichkeit der Kinderhandschriften im Anfangsunterricht? (3) Welche Zusammenhänge bestehen zwischen der Leserlichkeit der Handschrift sowie den orthografischen Fähigkeiten und der Textqualität in der Mitte des ersten Schuljahres? Um diese Fragen zu beantworten, wurden 612 Texte aus der Längsschnittstudie PERLE („Persönlichkeits- und Lernentwicklung von Grundschulkindern“) mithilfe eines induktiv-deduktiv entwickelten Ratingsystems zur holistischen sowie kriterialen Erfassung der Leserlichkeit von Kinderhandschriften ausgewertet. Die Evaluation des entwickelten Instruments liefert zufriedenstellende Ergebnisse. Die Analysen bezüglich des Einflusses individueller und familiärer Bedingungen sowie didaktischer und schulbezogener Voraussetzungen für die Leserlichkeit der Handschriften im Anfangsunterricht können bisherige Forschungsergebnisse, unbestätigte Thesen sowie verbreitete Vorurteile teils bestätigen (z.B. positiver Einfluss des Alters; Überlegenheit der verbundenen Schrift), teils weisen sie in eine andere Richtung (z.B. Vorteile der Mädchen). Zusammenfassend zeigen die Ergebnisse, dass die Leserlichkeit der Handschrift im Anfangsunterricht ein, aber nicht der einzige Faktor ist, um eine positive Entwicklung der allgemeinen (Schreib)kompetenzen zu befördern. KW - Leserlichkeit, Schreiben, Handschrift ER - TY - THES TI - Valide und vergleichbare Erfassung bildungsrelevanter Konstrukte bei Schülerinnen und Schülern mit sonderpädagogischem Förderbedarf Lernen AU - Nusser, Lena PY - 2018 PB - opus CY - Bamberg DO - 10.20378/irbo-51103 UR - https://fis.uni-bamberg.de/handle/uniba/43138 LA - deu AB - Zunehmend werden auch Schülerinnen und Schüler mit einem sonderpädagogischen Förderbedarf Lernen (SPF-L) in Large-Scale-Assessments einbezogen. Hierbei stellt die valide und vergleichbare Erfassung bildungsrelevanter Konstrukte eine besondere Herausforderung dar. Die vorliegende Arbeit gliedert sich in drei Teile. Zunächst wird in einer theoretischen Einbettung die Zielgruppe der Schülerschaft mit SPF-L näher beschrieben. Zur Erklärung der Herausforderungen, die sich für eine valide Erfassung bildungsrelevanter Konstrukte abzeichnen, werden zwei kognitive Modelle herangezogen: Das Modell in Anlehnung an die Prinzipien des dynamischen Testens nach Carlson und Wiedl (1992), welches die Probleme einer suboptimalen Anwendung der tatsächlichen Kompetenz und der daraus resultierenden Testperformanz aufgreift sowie das Modell der Accessibilty Theory (Beddow et al., 2013), welches die Testsituation als eine Interaktion zwischen Charakteristiken einer Testperson und den Anforderungen eines Tests rahmt. Zudem wird das Modell nach Tourangeau, Rips und Rasinksi (2000), welches die kognitiven Prozessschritte, die ein/e Befragte/r bei der Beantwortung von Fragebogenitems durchläuft, aufgegriffen. Die Ausgangslage der SchülerInnen mit SPF-L wird den theoretischen Modellen gegenübergestellt und somit die Herausforderungen bei der Bearbeitung von Fragebögen oder Tests und die Implikationen für Large-Scale-Studien herausgearbeitet. Anschließend werden die übergeordneten Forschungsfragen dieser Arbeit und die zugrundeliegenden Schriften vorgestellt. Die Ergebnisse werden in Bezug auf die dargelegten Forschungsfragen diskutiert und entsprechend der theoretischen Einbettung eingeordnet, bevor abschließend die Grenzen dieser Arbeit und die Implikationen für Large-Scale-Assessments unter Einbezug von SchülerInnen mit SPF-L dargelegt werden. ER - TY - THES TI - Unterrichtliche Determinanten der Schreibkompetenz in der Primarstufe : wie wirken sich Aufgabenstellung und individuelle Lernunterstützung auf die Textqualität im Anfangsunterricht aus? AU - Kürzinger, Anja PY - 2018 PB - opus CY - Bamberg DO - 10.20378/irbo-48822 UR - https://fis.uni-bamberg.de/handle/uniba/41706 LA - deu AB - Den Ausgangspunkt der vorliegenden Dissertation bildet die Frage, welche unterrichtlichen Bedingungsfaktoren als Erklärung für die individuelle Schreibperformanz in der Primarstufe herangezogen werden können. Vor dem Hintergrund eines elementaren Forschungsdesiderats werden im Rahmen von fünf Einzelbeiträgen Schreibfähigkeiten in Klasse 1 auf breiter Basis empirisch-quantitativ analysiert und mit der Prozessqualität von Schreibunterricht in Verbindung gebracht. Durch die Integration theoriebasierter, schreibdidaktischer Überlegungen mit der empirischen, videogestützten Unterrichtsforschung lässt sich das Forschungsvorhaben interdisziplinär verorten. So erfolgt die Modellierung der Konstrukte auf Basis fachdidaktischer Grundlagen, die mit psychometrischen Anforderungen an die Operationalisierung verbunden werden. Dabei greift die Arbeit, die in das DFG-Projekt NaSch1 (Narrative Schreibkompetenz in Klasse 1) eingebettet ist, auf 49 Unterrichtsvideos und 540 Texte aus der ersten Jahrgangsstufe der PERLE-Studie (Persönlichkeits- und Lernentwicklung von Grundschulkindern) zurück. Die Befunde zeigen, dass sich bereits im Anfangsunterricht komplexere, funktional-pragmatische Schreibfähigkeiten entwickeln, die sich in zwei Subdimensionen (semantisch-pragmatischer und sprachsystematischer Faktor) von Schreibkompetenz differenzieren lassen. Unter Kontrolle des Geschlechts der Lernenden sowie ihrer schriftsprachbezogenen Vorläuferfähigkeit (Buchstabenkenntnis) wirken sich vor allem präzise, verständliche Aufgabenstellungen und evaluative sowie lösungsorientierte Lernunterstützungen auf die Qualität der individuellen Schreibleistungen aus. Neben diesen Wirkungsanalysen liegen auch detaillierte deskriptive Ergebnisse zur unterrichtlichen Sicht- und Tiefenstruktur vor, die Einblicke in das Lehrerhandeln und die Praxis des Schreibunterrichts in Klasse 1 ermöglichen. Diese Befunde werden in dem vorliegenden Rahmentext unter Berücksichtigung der methodischen Vorgehensweise kritisch diskutiert und um Implikationen für die Schreibdidaktik und Schreib- bzw. Unterrichtsforschung erweitert. ER - TY - THES TI - Distal and Proximal Indicators of Teacher Effectiveness : As Predictors of Students’ Mathematics Competence in Germany (Grade 5 – 7) AU - Hanif, Saba PY - 2016 PB - University of Bamberg Press CY - Bamberg T3 - Schriften aus der Fakultät Humanwissenschaften der Otto-Friedrich-Universität Bamberg SN - 978-3-86309-452-2 DO - 10.20378/irb-41135 UR - https://fis.uni-bamberg.de/handle/uniba/41135 LA - eng AB - The focus of research on educational effectiveness has turned from school effectiveness towards the effectiveness of the teachers as research in the recent past has shown that teachers are the most important determinant of student learning. However, there is still a lack of consensus as to whether teacher variables (distal) or teaching variables (proximal) are more important. A strong body of knowledge has shown the importance of teacher variables in student learning. However, Seidel and Shavelson (2007) in their recent meta-analyses claimed that teaching factors are more important than teacher factors because they are proximal to the teaching-learning process. They argued that past studies on teacher effectiveness using quasi experimental and experimental studies have shown stronger effect on student learning than correlational studies because they have measured teacher effectiveness proximally. Therefore, they suggested correlational studies should also measure teacher effectiveness proximally to get better results. Although, it may be true that proximal indicators are more important than distal indicators, but findings from a single meta-analysis study are not enough to diminish the significance of teacher factors. More research is needed in correlational studies to explore this issue. The current study aims to measure the effectiveness of both distal and proximal indicators of teacher effectiveness in order to see which factors strongly predict student competence in the domain of mathematics. The study aims to investigate the role of teacher factors (teacher belief, professional training, planning, cooperation among teachers, job satisfaction and stress in planning and classroom) and their instruction (cognitive activation, cooperative learning, cognitively challenging tasks, student engagement and differentiation.) in the mathematics competence development of students. The design of my study is longitudinal in nature. National Educational Panel (NEPS) data is used to investigate the effect of teachers on students’ mathematics competence development from grade 5-7 with a sample of 151 teachers and their 1706 students. Confirmatory factor analysis was carried out on the data in order to confirm the construct validity of the questionnaires. Mathematics competence was measured by NEPS through a standardized test at the beginning of grade 5 and grade 7. Multiple regression analyses and multi-group analyses were carried out in Mplus 7.11 to analyze the research questions. The study found that the following distal indicators are significant predictors of mathematics competence belief about deep learning, professional collaboration, individual needs of students, and stress in planning and classroom. Among proximal indicators student engagement, cognitively challenging tasks and differentiation showed the significant effect. Distal indicators explained 11% and proximal indicators explained 6% of the total variance in mathematics competence of students. The study found that distal indicators are stronger predictors of mathematics competence than proximal indicators and the importance of distal factors cannot be undermined. Once student previous mathematics competence was controlled for professional collaboration was no longer significant predictor and the amount of variance explained by distal and proximal factors decreased. A multi-group analysis was performed in order to examine do teachers in different school tracks (lower secondary school, middle secondary school, multi-track school, comprehensive school and grammar school) vary in effectiveness. However, no meaningful pattern of difference in teacher effectiveness across schools types was found. Implications of the findings are discussed. KW - - ER - TY - THES TI - Modeling Competence Data in Large-Scale Educational Assessments AU - Haberkorn, Kerstin PY - 2016 PB - opus CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/40176 LA - eng AB - Over the last decades, large-scale assessments focusing on skills and knowledge of individuals have expanded significantly in order to obtain information about conditions for and consequences of competence acquisition. To provide valid and accurate scores of the subjects under investigation, it is necessary to thoroughly check the psychometric properties of the competence instruments that are administered and to choose appropriate scaling models for the competence data. In this thesis, various challenges in modeling competence data were addressed that arose from different recently developed competence tests in large-scale assessments. The different tests posed specific demands on the scaling of the data such as dealing with multidimensionality, incorporating different response formats, or linking competence scores. By investigating these challenges associated with each of the competence tests, the aim of the thesis was to draw implications for the specification of the scaling models for the competence data. First, a new metacognitive knowledge test for early elementary school children was investigated. As earlier findings on metacognitive knowledge in secondary school pointed to empirically distinguishable components of metacognitive knowledge, especially the dimensionality of the newly developed test was studied. Therefore, uni- and multidimensional models were applied to the competence data and their model fit was compared. By applying multidimensional latent-change models, the homogeneity of change was observed as further indicator for dimensionality. Overall, the new test instrument exhibited good psychometric properties including fairness of the items for various subgroups. In accordance with previous studies in other age groups the results indicated a multidimensional structure of the newly developed test instrument. In the discussion, theoretical as well as empirical arguments were compiled that should be considered for the choice of a uni- or a multidimensional model for the metacognitive knowledge data. The next objective in the thesis was to study a series of reading competence tests intended to measure the same latent trait across a large age span. The different reading competence tests, developed in a longitudinal large-scale study, were based on the same conceptual framework and were administered from fifth grade to adulthood. We specifically investigated whether the test scores were comparable across such a large age span enabling to interpret change across time. The analyses on the reading competence tests showed that the coherence of measurement could not fully be assured across the wide age range. The application of strict linking models allowing for the interpretation of developmental progress seemed to be justified within secondary school, but not between secondary school and adulthood. The last purpose in the thesis was to find out how to adequately incorporate different response formats in a scaling model. Therefore, multiple choice (MC) and complex multiple choice (CMC) items were regarded as they are most frequently used in large-scale assessments. Specifically, we explored whether the two response formats form distinct empirical dimensions and which a priori scoring schemes for the two response formats appropriately model the competence data. The results demonstrated that the response formats built a unidimensional measure across domains, studies, and age cohorts justifying to use a unidimensional scale score. A differentiated scoring of the CMC items yielded a better discrimination between persons and was, thus, preferred. The a priori weighting scheme of giving each subtask of a CMC item half the weight of a MC item described the empirical competence data well. ER - TY - THES TI - Investigating response styles and item homogeneity using item response models AU - Wetzel, Eunike PY - 2013 PB - opus CY - Bamberg UR - https://fis.uni-bamberg.de/handle/uniba/1852 LA - eng AB - Measurement invariance is a pre-requisite for drawing accurate and valid inferences concerning individuals’ trait levels from questionnaire data. However, several factors exist that can influence a person’s item responses in addition to his or her latent trait level. The research in this dissertation was aimed at investigating three of these factors: 1) individual differences in response styles, 2) the measurement invariance of items between subgroups of respondents, and 3) the measurement invariance of items across assessment periods. Mixed Rasch analyses of data from the German NEO-PI-R showed that respondents differed systematically in their response scale use: some preferred extreme categories while others preferred moderate categories. Multidimensional item response models showed that response styles (especially extreme response style) explained variance in item responses that was incremental to the variance explained by the traits. Thus, individual differences in response styles had an influence on item responses. The measurement invariance of items between subgroups of respondents was investigated with respect to differential item functioning for gender in the German NEO-PI-R. Several NEO-PI-R facets especially on neuroticism, agreeableness, and conscientiousness contained items that were not measurement invariant for men and women. The measurement invariance of items across assessment periods was investigated for link items from the reading and science domains in the Programme for International Student Assessment (PISA). Measurement invariance was violated for both item sets. Some items showed large differences in item difficulty between assessments which may in part be attributed to changes in item wording and position effects. In sum, it was shown that individual differences in response styles, the lack of measurement invariance of items between subgroups of respondents, and the lack of measurement invariance of items across assessment periods can impair the measurement of the intended traits and in consequence render trait inferences and comparisons between individuals or groups invalid. Thus, measures should be taken to reduce the impact of factors that interfere with measurement invariance. These measures can be aimed at test construction where, for example, the item or response format can be adjusted to elicit response styles to a lesser degree and items can be selected that have invariance properties across subgroups of participants and across assessment periods. KW - response styles KW - item response models KW - measurement invariance ER -