Development of Domain-Specific Critical Online Reasoning Skills in Medical Students during Their Preclinical Studies
Background and Study Focus
Given the recent changes in the medical information landscape, such as the emergence of data-intensive precision medicine and clinical data science methods, medical education faces the challenge of equipping students with the necessary skills to navigate this information landscape and make well-reasoned decisions. Developing critical online reasoning (COR) skills as part of medical studies is therefore essential – such skills are crucial when students use the Internet to retrieve information about patient cases and solve professional (e.g. diagnostic) problems. A lack of domain-specific (DOM-)COR skills can lead to an acquisition of misconceptions or misapplication of knowledge, which ultimately jeopardises patient safety.
Concept and Research Objective
To identify factors influencing COR, medical teachers urgently need valid assessments that reliably capture students’ skills in critical use of online information. This represents a central prerequisite for the development of DOM-COR training interventions. To contribute to a better understanding of how medical education needs to be designed to overcome the apparent lack of COR skills among medical students, project A03 has four overarching objectives:
- Description of DOM-COR in medical students
- Prediction of DOM-COR development during academic studies
- Prediction of learning outcomes
- Use of research results and findings/transfer to practice
Measurement and Analyses
A longitudinal study over the first three years of medical studies will examine the effects of the medical curriculum on the development of DOM-COR skills based on a sample of medical students, with physics students serving as a comparison group – in the first assessment in the winter semester 2023/24, a total of 170 medicine and physics students took part in the DOM-COR assessment. Further assessments are currently being carried out.
Of particular interest in project A03 are the different reasoning contexts in which DOM-COR can be applied in medicine: biomedical (i.e., fundamental scientific) reasoning requires understanding of biomedical, chemical, or physical mechanisms to understand the effects of medication on specific diseases (e.g., radiation in cancer treatment); practical clinical reasoning leading to clinical decision-making regarding the needs of a specific patient (e.g., establishing a differential diagnosis and an according treatment plan); transdisciplinary reasoning to solve complex information problems that need to be considered from different perspectives at an integrated level (e.g., global health).
Scenario-based tasks are continuously developed to validly measure three DOM-COR cognitive skill facets across reasoning contexts (searching for online information; critical evaluation of the information; evidence-based reasoning and synthesizing of the information). Using Internet-based tasks as well as Internet-like simulation-based tasks, we investigate the impact of the medical curriculum by assessing which courses and research-related activities students specifically take during their studies. We also consider other key influencing factors such as individual learning prerequisites and interactions with specific properties of DOM-COR tasks or online materials used.
Research Outcome and Contribution within the research unit CORE
A03 generates evidence on how medical students’ DOM-COR skills develop longitudinally. We want to clarify which role the curriculum and used learning opportunities play during studies, and which impact key personal and contextual factors have in this regard. We further want to clarify to what extent distinct DOM-COR skill profiles can be found in medical students. Finally, the question of how crucial DOM-COR skills are for medical study success is going to be answered. Results should substantiate a 2nd research phase in which targeted (and possibly adaptive) interventions will be developed to foster medical DOM-COR skills. The unique data set generated in A03 will allow for comprehensive cross-project interdisciplinary data analyses by providing psychometric data on DOM-COR performance of medical (and physics) students, data of online materials used by the students – analyzed as ‘information landscapes' in detail in B-projects, and DOM-COR process data in the form of log files – analyzed in detail in collaboration with C-projects.
Publications
Zottmann, J., Kuhn, J., & Fischer, M. R. (Guest eds.). (2025). Special issue on “Critical reasoning and argumentation for informed decisions in health care.” GMS Journal for Medical Education.Fischer, M. R., Kuhn, J., van Gog, T., Zottmann, J., Dinc, Y., Horrer, A., & Ruf, V. (2025). A03: Development of domain-specific critical online reasoning (DOM-COR) skills in medical students during their preclinical studies. In O. Zlatkin-Troitschanskaia, M. Toepper, & D. Molerov (Eds.), Critical Online Reasoning in Higher Education (CORE) – Portfolio of Assessment Instruments (CORE Working Papers, 3). Johannes Gutenberg University Mainz.
Paper and Poster Presentations
Stroganova, A., Horrer, A., Weber, M., Klose, V., Kuhn, J., Fischer, M. R., & Zottmann, J. (in press). Potenziale generativer KI bei der Erstellung komplexer Fallszenarien zur Erfassung kritischer Online-Reasoning-Fähigkeiten Studierender [Conference presentation]. 30. Kongress der Deutschen Gesellschaft für Erziehungswissenschaft (DGfE), Munich, Germany.
Horrer, A., Stroganova, A., Dinc, Y., Ruf, V., Küchemann, S., Kuhn, J., Fischer, M. R., & Zottmann, J. (2025). Investigating the interplay of scientific reasoning skills and epistemological beliefs of medical and physics students [Conference presentation]. 21st Biennial Conference of the European Association for Research on Learning and Instruction (EARLI), Graz, Austria.
Illmann, J., Baumartz, D., Drake, P., Hahnel, C., Hartig, J., Köhler, C., Schrickel, M., Stroganova, A., & Goldhammer, F. (2025, January). Die Definition und Validierung von Logdaten-Indikatoren für die Messung von Critical Online Reasoning [Conference presentation]. 12. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Mannheim, Germany.
Martin de los Santos, L., Klopp, E., Horrer, A., Zottmann, J., & Zlatkin-Troitschanskaia, O. (2025, January). The relation between critical online reasoning skills of incoming university students in Germany and their epistemological beliefs [Conference presentation]. 12. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Mannheim, Germany.
Horrer, A., Küchemann, S., Kuhn, J., Fischer, M. R., & Zottmann, J. (2024). The interplay of scientific reasoning skills and epistemological beliefs of medical and physics students [Symposium presentation]. Presentation at BERA Conference 2024 and WERA Focal Meeting, 08.–12 September, Manchester, United Kingdom.
Horrer, A., Küchemann, S., Kuhn, J., Fischer, M. R., & Zottmann, J. (2024, September). The interplay of scientific reasoning skills and epistemological beliefs of medical and physics students [Paper presentation]. EARLI SIG 20 & 26 Conference, Berlin, Germany.
Horrer, A., Küchemann, S., Kuhn, J. Fischer, M. R., & Zottmann, J. (2024). On the interplay of scientific reasoning and epistemological beliefs of medical and physics students [Paper presentation]. Presentation at EARLI SIG 4 & 17 Conference 2024. 25.-27. September, Utrecht, The Netherlands.
Stroganova, A., Horrer, A., Kuhn, J., Fischer, M. R., & Zottmann, J. (2024). Task-Entwicklung für die Messung von Fähigkeiten zum Critical Online Reasoning in der medizinischen Ausbildung [Symposium presentation]. Presentation at GMA, 05.-09. August, Fribourg, Swiss.
Zottmann, J., Kuhn, J., & Fischer, M. R. (2024, August). Critical online reasoning in medical education [Symposium]. GMA Annual Conference, Fribourg, Switzerland.
Zottmann, J., Horrer, A., Kuhn, J., Jünger, J., Roeper, J., & Fischer, M. R. (2023). Fachspezifische Kompetenzen von Hochschulstudierenden im Umgang mit Online – Erfassung und Entwicklung im Studium – Projekt A03 [Symposium presentation]. Presentation at AEPF, 15. September, Potsdam, Germany.