Development of A Science Literacy-Based Four-Tier Diagnostic Instrument to Identify Misconceptions in Chemical Bonding Material

Abstract

This study aimed to develop a science-literacy-based four-tier diagnostic instrument for identifying students’ misconceptions in chemical bonding and to establish evidence of its content validity. The instrument was developed using the Test Design and Development approach, focusing on the planning and development stages. An initial pool of 40 multiple-choice items integrated into nine contextual informational texts was evaluated by five expert validators, comprising three chemistry education lecturers and two high school chemistry teachers. Content validity was assessed using the Content Validity Ratio (CVR), Content Validity Index (CVI), Item-Level Content Validity Index (I-CVI), and Scale-Level Content Validity Index based on the average method (S-CVI/Ave). The results showed that 37 of the 40 items obtained a CVR of 1 and met the required criterion for five validators, while three items failed to satisfy the threshold and therefore required further revision or exclusion before pilot testing. Based on the I-CVI analysis, all 40 items met the acceptable criterion of ≥ 0.78. At the scale level, the instrument achieved an overall CVI of 0.9875 and an S-CVI/Ave of 0.99375, indicating a high level of agreement among experts regarding the relevance of the instrument content. These findings provide strong preliminary evidence for the content validity of the developed instrument but do not establish its overall psychometric quality or diagnostic accuracy. Further empirical testing involving students is therefore required to examine construct validity, reliability, item functioning, classification accuracy, and diagnostic performance before the instrument can be recommended for broader diagnostic use.