Learning How to Measure STEM Student Critical Thinking Through Undergraduate Research Engagement Across a Three-Institution Consortium

Abstract

Undergraduate research is a high-impact practice associated with positive student outcomes, including improved retention, graduation rates, and post-graduation success. It is also linked to gains in intellectual skills, such as critical thinking; however, these gains are typically inferred from student self-reports and faculty observations rather than direct measurement. A key challenge is the lack of robust methods to directly assess critical thinking development through undergraduate research experiences in Science, Technology, Engineering, and Math (STEM). To address this gap, the Learning Environment and Academic Research Network® (LEARN) Consortium was established across three institutions. The consortium implemented a year-long, faculty-mentored research experience through two parallel programs: one for underrepresented first-time-in-college (FTIC) students and another on Associate of Arts (AA) or Associate of Science (AS) transfer students. To directly measure changes in critical thinking the consortium employed a pre-post assessment design using two instruments: 1. the Critical Thinking Assessment Tool (CAT©) and 2. STEM writing assignments evaluated with a specifically designed critical thinking rubric. Results from the CAT© indicated gains in critical thinking for both FTIC and transfer cohorts, with statistically significant gains for the FTIC students at two institutions in two of the three years. Similarly, rubric-based evaluations of STEM writing, showed gains for both cohorts, though these were not statistically significant. This article examines the challenges of directly assessing critical thinking, including issues of rubric development, and shares lessons learned from implementing and evaluating this transferable undergraduate research model across three different institutions.