Analysis of the Accuracy and Comprehensibility of Media Content Used by Medical and Economics Students in Higher Education Online Learning Settings
Background and Study Focus
Recent research indicates that news media and social media play a remarkable role in students’ information acquisition and learning in higher education. While some students learn from these media, other students learn nothing or even adopt misconceptions. Although previous research has provided valuable evidence on various factors ingrained in the information content and student learning characteristics, we know comparably little about the interaction of such influences in natural settings (i.e., the Internet).
Concept and Research Objective
B04 aims to characterize the information landscape students use in generic and domain-specific critical online reasoning (COR) tasks by focusing on the accuracy (i.e., incompleteness, bias, incorrectness) and comprehensibility (i.e., linguistic and stylistic level) of the online content used by medical and economics students when completing these tasks. Additionally, B04 aims to develop a theoretically sound and empirically driven framework to explain the interplay between different types and characteristics of information that students use in COR tasks and the processes that govern students’ information selection, evaluation and reasoning. Finally, we are interested in the long-term relations of types of information sources that students habitually use and their learning outcomes in higher education.
Measurements and analyses
The description of the information landscape is based on a quantitative content analysis of websites that students access when solving COR tasks. The accuracy of information is assessed manually, the comprehensibility of information is measured partly automatically, partly by human coders. The content analytics data are then combined with data from COR tasks assessed in the A-projects of this research unit (FOR). Finally, we rely on the FOR panel survey data to investigate how the media and information use of students is related to COR development and learning outcomes over time.
Research Outcomes
The study examines to what extent information sources differ with respect to accuracy and comprehensibility and how this variation affects students’ COR performance. The examination of short-term and long-term influences of students’ information diet on COR performance and learning outcomes provides a unique basis for the conceptual development of instructional interventions, e.g. using Internet-like simulations, to effectively foster the DOM-COR skills (in the 2nd research phase).
Contribution within the research unit CORE
B04 uses data from the COR-assessments and panel survey conducted in A-projects. The examination of inter-individual differences and of strategies of high-performers and deficiencies of low-performers in COR-tasks is shared with C07. Together with B05 and B06, which analyze the same information sources using different methodologies (B05: linguistic analysis, B06: qualitative-reconstructive analysis), we contribute to an in-depth understanding of the beneficial and detrimental role of various online information sources for higher education learning among students in medicine and economics.
Analyses and results to date
In the first assessment phase, we analyzed 343 unique websites used by students during the GEN COR tasks, as well as 1901 unique websites used by students during the DOM COR tasks (1080 in the economics context and 821 in the medicine context). We analyzed these websites using a quantitative content analysis with 14 categories regarding information quality (e.g., completeness etc.).
Our results revealed vast differences in the genres of websites used by students to complete GEN and DOM COR tasks. Information quality varies, with most websites providing only limited relevant information. We found that generalized statements regarding information quality cannot be made based on website genre alone. Furthermore, we found differences in completeness depending on the reasoning context of the DOM COR task for which the websites were used.