Development of Domain-Specific Critical Online Reasoning Skills in Medical Students during Their Preclinical Studies
Background and Study Focus
Given the recent changes in the medical information landscape, such as the emergence of data-intensive precision medicine and clinical data science methods, medical education faces the challenge of equipping students with the necessary skills to navigate this information landscape and make well-reasoned decisions. Developing critical online reasoning (COR) skills as part of medical studies is therefore essential – such skills are crucial when students use the Internet to retrieve information about patient cases and solve professional (e.g. diagnostic) problems. A lack of domain-specific (DOM-)COR skills can lead to an acquisition of misconceptions or misapplication of knowledge, which ultimately jeopardises patient safety.
Concept and Research Objective
To identify factors influencing COR, medical teachers urgently need valid assessments that reliably capture students’ skills in critical use of online information. This represents a central prerequisite for the development of DOM-COR training interventions. To contribute to a better understanding of how medical education needs to be designed to overcome the apparent lack of COR skills among medical students, project A03 has four overarching objectives:
- Description of DOM-COR in medical students
- Prediction of DOM-COR development during academic studies
- Prediction of learning outcomes
- Use of research results and findings/transfer to practice
Measurement and Analyses
A longitudinal study over the first three years of medical studies will examine the effects of the medical curriculum on the development of DOM-COR skills based on a sample of medical students, with physics students serving as a comparison group – in the first assessment in the winter semester 2023/24, a total of 170 medicine and physics students took part in the DOM-COR assessment. Further assessments are currently being carried out.
Of particular interest in project A03 are the different reasoning contexts in which DOM-COR can be applied in medicine: biomedical (i.e., fundamental scientific) reasoning requires understanding of biomedical, chemical, or physical mechanisms to understand the effects of medication on specific diseases (e.g., radiation in cancer treatment); practical clinical reasoning leading to clinical decision-making regarding the needs of a specific patient (e.g., establishing a differential diagnosis and an according treatment plan); transdisciplinary reasoning to solve complex information problems that need to be considered from different perspectives at an integrated level (e.g., global health).
Scenario-based tasks are continuously developed to validly measure three DOM-COR cognitive skill facets across reasoning contexts (searching for online information; critical evaluation of the information; evidence-based reasoning and synthesizing of the information). Using Internet-based tasks as well as Internet-like simulation-based tasks, we investigate the impact of the medical curriculum by assessing which courses and research-related activities students specifically take during their studies. We also consider other key influencing factors such as individual learning prerequisites and interactions with specific properties of DOM-COR tasks or online materials used.
Research Outcome and Contribution within the research unit CORE
A03 generates evidence on how medical students’ DOM-COR skills develop longitudinally. We want to clarify which role the curriculum and used learning opportunities play during studies, and which impact key personal and contextual factors have in this regard. We further want to clarify to what extent distinct DOM-COR skill profiles can be found in medical students. Finally, the question of how crucial DOM-COR skills are for medical study success is going to be answered. Results should substantiate a 2nd research phase in which targeted (and possibly adaptive) interventions will be developed to foster medical DOM-COR skills. The unique data set generated in A03 will allow for comprehensive cross-project interdisciplinary data analyses by providing psychometric data on DOM-COR performance of medical (and physics) students, data of online materials used by the students – analyzed as ‘information landscapes' in detail in B-projects, and DOM-COR process data in the form of log files – analyzed in detail in collaboration with C-projects.
