Contextual Constraints in Science Assessment: Effects of Item Sequencing and Item Features on Measures of Student Performance
Creators:Rector, Meghan A.
MetadataShow full item record
Series/Report no.:2012 Edward F. Hayes Graduate Research Forum. 26th
Innovations in automated scoring and related technologies are facilitating increased use of constructed-response (CR) assessment instruments in science education and other content areas. While a large body of work has explored a variety of issues relating to the effects of multiple-choice item sequencing on student performance, comparatively less work has explored these issues using constructed-response assessments. We report on the results from three studies of undergraduate students that examine the effects of item sequencing, difficulty, and surface features on student performance using constructed response items in biology education. Our first study examined the extent to which item sequencing was associated with student performance on isomorphic, constructed-response instruments. In our response corpus, verbosity (number of words) and use of accurate scientific ideas (key concepts) declined significantly from the first to the fourth items. In contrast, use of naïve ideas was not correlated with verbosity or the use of scientific ideas. Our second study examined the interaction between item sequencing and familiarity of items. Use of key concepts in this sample was significantly related to response verbosity and item familiarity, but not item sequencing. The use of naïve ideas was also correlated with familiarity, with more familiar items eliciting more naïve ideas. Our third study examined the interaction between item sequencing and the polarity of item features. In our collection of student responses, verbosity was significantly related to item sequencing but concept use was not. In contrast, the polarity of the item significantly affected the use of key concepts and naïve ideas, corroborating previous research on the gain and loss of traits. Together the results from our three studies identify multiple factors influencing student performance on CR items. Assessments containing items with similar surface features were subject to greater order effects relative to item sequencing that differed in surface features. Additionally, item sequencing had a greater effect on response verbosity, which corresponded with use of accurate scientific ideas. We conclude with a conceptual model accounting for factors influencing student success on CR assessments.
Education and Human Ecology: 2nd Place (The Ohio State University Edward F. Hayes Graduate Research Forum)
A three-year embargo was granted for this item.
Items in Knowledge Bank are protected by copyright, with all rights reserved, unless otherwise indicated.