Developing HOTS-Based Assessment Instrument for Science Learning in Junior High School: Validity, Reliability, and Item Analysis

Abstract

Despite growing policy demands for Higher Order Thinking Skills (HOTS) assessment in Indonesian science education, psychometrically validated instruments aligned with the Merdeka Curriculum remain scarce. This study aims to develop and validate a HOTS-based assessment instrument for eighth-grade junior high school (SMP) science students. Employing the Research and Development (R&D) approach with the 4-D model (Define, Design, Develop, Disseminate), the instrument was designed to comprehensively measure HOTS at cognitive levels C4 (Analyzing), C5 (Evaluating), and C6 (Creating) of the Revised Bloom’s Taxonomy, covering Basic Competency (BC) 3.8 (Respiratory System) and BC 3.9 (Photosynthesis) topics. The instrument consists of 40 two-tier multiple-choice (TTMC) items and 10 essay items. Content validity was assessed by three experts using Aiken’s V formula, yielding a mean V of 0.87 (highly valid). Reliability was estimated using the KR-20 formula in a large-group trial (n = 64), producing a coefficient of 0.82 (reliable), and Cronbach’s alpha of 0.79 for essay items. Item analysis encompassed difficulty index, discriminating power, and distractor effectiveness. Results showed that 34 of 40 TTMC items (85%) met all quality criteria simultaneously: moderate difficulty, good discriminating power, and effective distractors. The final instrument (34 TTMC + 10 essay items) is valid and reliable for measuring higher-order thinking skills among SMP students. The TTMC format provides added diagnostic value by revealing specific conceptual misconceptions in addition to measuring HOTS proficiency, aligning with the formative assessment emphasis of the Merdeka Curriculum.