Development of Economics Students’ Domain-Specific Critical Online Reasoning Skills over the Course of Their Bachelor Studies and Key Influencing Factors
Study Background and Focus
Research indicates that students increasingly use online information for study-related learning and that the ability to search for, evaluate, and integrate online information is relevant for successful learning. There is also evidence of considerable variation in students’ ability to critically use subject-related online information, including in economics.
These developments highlight the relevance of integrating critical online reasoning (COR) more systematically into economics education. Valid assessments are needed to diagnose students’ COR skills and provide an empirical basis for instructional approaches that support competent, self-directed learning with online information.
Concept and Research Objective
A02 develops and validates a domain-specific framework and digital performance assessment for measuring how economics students acquire, evaluate, and reason with online information, and investigates how these competencies develop during the course of university studies.
Based on prior research, A02 defines domain-specific (DOM)-COR in economics as the necessary skill set to critically select, evaluate, and use economics-related information from the Internet to derive warranted knowledge and reasoned decisions.
The DOM-COR construct is modeled in accordance with the three facets:
- online information acquisition (OIA),
- critical information evaluation (CIE),
- reasoning with evidence, argumentation, and synthesis (REAS).
OIA, CIE, and REAS are interrelated cognitive abilities that interact dynamically during Internet-based task solving and are activated and regulated by metacognitive abilities (MCA).
A02 further differentiates between three requirement contexts, in which DOM-COR can be applied:
- theoretical/fundamental reasoning for determining correct meanings of concepts (e.g., the break-even point),
- applied reasoning required in economic decision-making under real business conditions (e.g., preparing a financial plan based on conflicting parameters),
- and transdisciplinary reasoning for complex problem-solving (e.g., making decisions about costly life-saving measures with limited resources).
Based on relevant work on the theoretical modeling of the development of critical thinking as well as economic understanding, in line with the Model of Domain Learning three developmental stages in the development of DOM-COR skills are assumed: acclimation (basic level); competence (advanced level); proficiency (proficient level).
A02 aims to develop and validate an objective and reliable performance-based assessment of students’ DOM-COR development that supports valid interpretations of test scores and can inform diagnostic and research purposes in higher education.
Measurement and Analyses
To validly assess DOM-COR skills, A02 uses scenario-based tasks that immerse students in a realistic economics problem, employing newly developed and validated performance assessments in an open format with unrestricted Internet access and a closed format with Internet-like simulations with preselected sources.
Following the conceptual basis, every task in the DOM-COR assessment is designed to cover
- all three DOM-COR skill facets and
- all three reasoning contexts.
All DOM-COR tasks focus on seven problem scenarios in economics in these three contexts related to reasoning derived from found (open information space, OIS) or given online information (closed information space, CIS) using open or closed task formats.
Both the response texts and task-solving processes (incl. search history, clicks, and scrolling) are recorded in the digital DOM-COR assessment environment in real time. These multimodal student performance and response process data are then evaluated by multiple independent, trained raters using a newly developed rating scheme based on the COR construct definition.
The various DOM-COR measures are then analyzed alongside additional survey information, such as socio-demographic data, media use habits (incl. AI use), professional knowledge, intelligence, and characteristics of the online sources students used, to gain a comprehensive understanding of students' DOM-COR skills and their determinants.
The effects of curricula and learning opportunities on the development of DOM-COR skills, as well as other key personal and contextual theoretically assumed covariates, are longitudinally analyzed in A02 based on data from three panel cohorts with students of economics and economics education (and social sciences students serving as a comparison group) assessed over the course of their bachelor studies at three measurement points. Across the three waves, A02 combines repeated participation with newly recruited students to investigate development during undergraduate study.
At the first measurement in winter term 2023/24, overall 250 economics and social sciences students took part in the DOM-COR assessment. In addition, participants completed the economics knowledge test and the intelligence test, which were assessed as key personal covariates.
For the second measurement in the winter term 2024/25, 253 economics and social sciences students overall took part in the DOM-COR assessment as well as the economics knowledge test and the intelligence test. Of these, 126 students participated in the survey for the second time and 127 students participated for the first time.
For the third measurement in the winter term 2025/26, 258 economics and social sciences students overall took part in the DOM-COR assessment as well as the economics knowledge test and the intelligence test. Of these, 155 students participated for the second or third time, and 103 participated for the first time.
Research Outcomes
A02 generates longitudinal and cross-sectional data and findings
- on the development of students’ DOM-COR in economics,
- on key personal and institutional covariates and
- on the relation to key learning outcomes (such as exam grades and achieved Credit Points).
Contribution within the Research Unit CORE
A02 contributes data, assessment instruments, and methodological expertise to cross-project analyses across all three CORE research areas. For example, A02 data and task materials are linked with analyses in the B projects to examine the characteristics and quality of online information used in task solving and their relevance for students’ DOM-COR performance.
Publications and Presentations
Publications
Martin de los Santos, L., Toepper, M., Nagel, M.-T., Trierweiler, L. & Zlatkin-Troitschanskaia, O. (in press). Kritisches Denken im digitalen Wandel: Informationspraktiken, epistemologische Überzeugungen und Critical Online Reasoning bei Studierenden der Wirtschaftspädagogik. Zeitschrift für Berufs- und Wirtschaftspädagogik, 122(4).
Molerov, D., Federiakin, D., Zlatkin-Troitschanskaia, O., Shenavai, K., Trierweiler, L., & Nagel, M. (2026). The Relationship Between AI-Chatbot Use and Student Assessment Performance and Learning Outcomes in Higher Education. Unterrichtswissenschaft. https://doi.org/10.1007/s42010-026-00242-2
Nagel, M.-T., & Zlatkin-Troitschanskaia, O. (2025). The competence to deal critically with online information among economics education students and teacher trainees in economics: implications for teacher training. Empirical Research in Vocational Education and Training, 17(11). https://doi.org/10.1186/s40461-025-00186-4
Reichert-Schlax, J. & Zlatkin-Troitschanskaia, O. (2026). Tracking Knowledge and Confidence Trajectories: A Cross-Domain Longitudinal Study in Higher Education Economics and Social Sciences. International Journal of Educational Research (138). https://doi.org/10.1016/j.ijer.2026.103007
Zlatkin-Troitschanskaia, O., Hartig, J., Schmidt, U., Schmidt, S., Drake, P., & Nagel, M. (2023). Development of Economics Students’ Domain-Specific Critical Online Reasoning Skills over the Course of Their Bachelor Studies and Key Influencing Factors. In Zlatkin-Troitschanskaia, O., Fischer, J., & Molerov, D. (Eds.), Critical Online Reasoning in Higher Education (CORE) – Overview of the Research Unit: Objectives, framework, structure, and research projects [Working Paper No. 1].
Zlatkin-Troitschanskaia, O., Hartig, J., Schmidt, S., Drake, P., & Nagel, M. (2025). Development of Students’ Domain-Specific Critical Online Reasoning Skills in Economics. In Zlatkin-Troitschanskaia, O., Toepper, M., & Molerov, D. (Eds.), Critical Online Reasoning in Higher Education (CORE) – Portfolio of Assessment Instruments [Working Paper No. 3].
International Presentations
Hodes, P., Federiakin, D., Zlatkin-Troitschanskaia, O., Martin de los Santos Kleinz, L., Gabler, J., & Molerov, D. (2025). Impact of Vocational Education on Student Media Usage Skills: A Quasi-Experimental Design [Paper presentation]. Presentation at the 2025 National Council on Measurement in Education (NCME) Annual Meeting, April 2025, Denver, US.
Hodes, P., Nagel, M.-T., & Zlatkin-Troitschanskaia, O. (2026). Modeling critical information evaluation within the Critical Online Reasoning framework: A digital performance-based measurement approach [symposium presentation]. Presentation at International Test Commission Conference 2026, July 2026, Auckland, New Zealand.
Nagel, M.-T., Kohmer, A., Zlatkin-Troitschanskaia, O., Martin de los Santos Kleinz, L., Fischer, J. & Maur, A. (2023). Development and Validation of a Critical Online Reasoning Assessment in Higher Education - An International Study [Paper presentation]. Presentation at the 2023 WERA Focal Meeting, November 2023, Singapore.
Nagel, M., Zlatkin-Troitschanskaia, O., Braunheim, D., Martin de los Santos, L., Drake, P., & Hartig, J. (2024). Assessment of Economic Critical Online Reasoning Skills in First-semester Bachelor's Economic Students [Poster presentation]. Presentation at EARLI SIG 20 & 26 Conference, September 2024, Berlin, Germany.
Nagel, M.-T., Zlatkin-Troitschanskaia, O., Schmidt, S., Trierweiler, L., Hodes, P., & Martin de los Santos Kleinz, L. (2026). Validating Digital Performance Tasks for Assessing Domain-specific Critical Online Reasoning in Higher Education [Symposium presentation]. Presentation at the International Test Commission Conference (ITC) 2026, July 2026, Auckland, New Zealand.
Nagel, M.-T., Zlatkin-Troitschanskaia, O., Trierweiler, L., Drake, P., Hartig, J., & Walstad, W. (2025). Developing Domain-Specific Critical Online Reasoning Skills of Economics Students in their First Year [Symposium presentation]. Presentation at the EARLI 2025 Conference, August 2025, Graz, Austria.
Nagel, M., Zlatkin-Troitschanskaia, Trierweiler, L., Martin de los Santos Kleinz, L., Drake, P., & Hartig, J. (2024). Economics students’ Domain-Specific Critical Online Reasoning (DOM-COR) skills at the beginning of their studies [Symposium presentation]. Presentation at BERA Conference & WERA Focal Meeting, September 2024, Manchester, UK.
Schmidt, S., Hodes, P., Shenavai, K., Wieland, J., Zlatkin-Troitschanskaia, O., & Nagel, M.-T. (2026). Understanding human–LLM disagreement: Text features, rubric criteria, and edge cases in automated essay scoring [Symposium presentation]. Presentation at International Test Commission Conference 2026, Auckland, New Zealand.
Schmidt, S., Hodes, P., Shenavai, K., Zlatkin-Troitschanskaia, O., & Nagel, M.-T. (2026). Psychometric properties of LLM-based ratings in domain-specific assessment: Agreement, bias patterns, and context-specific effects [Symposium presentation]. Presentation at International Test Commission Conference 2026, July 2026, Auckland, New Zealand.
Trierweiler, L., Nagel, M.-T., Zlatkin-Troitschanskaia, O., Hartig, J., & Drake, P. (2025). Valid Assessment of Economics Students’ Domain-Specific Critical Online Reasoning Skills [Paper presentation]. Presentation at the 2025 National Council on Measurement in Education (NCME) Annual Meeting, April 2025, Denver, US.
Zlatkin-Troitschanskaia, O., Schmidt, S., Hartig, H., Drake, P., Nagel, M., & Fischer, J. (2024, April 13). Performance assessments to validly measure cross-nationally university economics students’ domain-specific critical online reasoning [Paper presentation]. Presentation at the Annual Meeting of the American Educational Research Association (AERA), Denver, CO, USA.
National Presentations
Hodes, P., Nagel, M.-T., & Zlatkin-Troitschanskaia, O. (2026). Kritische Online-Informationsbewertung bei Studierenden: Fähigkeitsprofile und epistemologische Überzeugungen [Poster presentation]. Presentation at 54. DGPs Kongress, September 2026, Luxembourg.
Martin de los Santos Kleinz, L., Zlatkin-Troitschanskaia, O., Trierweiler, L., & Nagel, M.-T. (2025). Die Entwicklung der Mediennutzung und Critical Online Reasoning bei Studierenden der Wirtschaftswissenschaften und Wirtschaftspädagogik im ersten Studienjahr [poster presentation]. Presentation at 25. annual meeting of the DFG Section Vocational and Business Education (BWP), September 2025, Darmstadt.
Nagel, M.-T., Zlatkin-Troitschanskaia, O., Trierweiler, Hodes, P., & Schmidt, S. (2026). Entwicklung ökonomischen Critical Online Reasoning bei Studierenden der Wirtschaftswissenschaften und Wirtschaftspädagogik im ersten & zweiten Studienjahr [symposium presentation]. Presentation at 26. annual meeting of the DFG Section Vocational and Business Education (BWP), September 2026, Rostock.
Nagel, M., Zlatkin-Troitschanskaia, O., Trierweiler, L., Scherer, T., Maurer, M., & Schemer, C. (2026). Zusammenhang zwischen domänenspezifischem Critical Online Reasoning (COR) von Studierenden der Wirtschaftswissenschaften/-pädagogik und ihrer Nutzung digitaler Medien im Studium. Presentation at 30. Kongress der Deutschen Gesellschaft für Erziehungswissenschaft (DGfE), March 2026, Munich.
Nagel, M., Zlatkin-Troitschanskaia, O., Schmidt, S., Hartig, H., & Drake, P. (2024). Entwicklung des domänenspezifischen kritischen Online Reasoning (DOM-COR) im wirtschaftswissenschaftlichen Bachelorstudium und Einflussfaktoren auf DOM-COR [Poster presentation]. Presentation at 29. Kongress der Deutschen Gesellschaft für Erziehungswissenschaft (DGfE), March 2024, Halle.
Nagel, M.-T., Zlatkin-Troitschanskaia, O., Trierweiler, L., & Schmidt, S. (2025). Validierung von Aufgaben zur Erfassung des domänenspezifischen Critical Online Reasoning (DOM-COR) bei StudienanfängerInnen der Wirtschaftswissenschaften und Wirtschaftspädagogik [symposium presentation]. Presentation at 25. annual meeting of the DFG Section Vocational and Business Education (BWP), September 2025, Darmstadt.
Reichert-Schlax, J., Zlatkin-Troitschanskaia, O., Trierweiler, L., Nagel, M.-T., Martin de los Santos Kleinz, L. & Hodes, P. (2026). Erfassung zentraler Studieneingangsmerkmale im digitalen Zeitalter: Ein Kohortenvergleich bei Studierenden der Wirtschaftspädagogik, Wirtschafts- und Sozialwissenschaften [symposium presentation]. Presentation at 26. annual meeting of the DFG Section Vocational and Business Education (BWP), September 2026, Rostock.
Zlatkin-Troitschanskaia, O. & Hartig, J. (2023). Generische und fachspezifische Kompetenzen von Hochschulstudierenden im Umgang mit Online-Informationen - Erfassung und Entwicklung im Studium – Projekte A01 und A02 [symposium presentation]. Presentation at AEPF, September 2023, Potsdam.
Zlatkin-Troitschanskaia, O., Nagel, M., Hodes, P., Schmidt, S., Beck, K., & Martin de los Santos Kleinz, L. (2024). Domänenspezifisches Critical Online Reasoning (DOM-COR) bei StudienanfängerInnen der Wirtschaftswissenschaften und WiPäd [symposium presentation]. Presentation at 24. annual meeting of the DFG Section Vocational and Business Education (BWP), September 2024, Dresden.
Zlatkin-Troitschanskaia, O., Toepper, M., Molerov, D., Hodes, P., & Martin de los Santos, L. (2025). Critical Online Reasoning bei Studierenden der Wirtschaftswissenschaften und Wirtschaftspädagogik in Zeiten von ChatGPT [symposium presentation]. Presentation at 25. annual meeting of the DFG Section Vocational and Business Education (BWP), September 2025, Darmstadt.