Modeling Extreme Response Style in Psychological Data: A Cross-Cultural Investigation
Loading...
Date
2017-03
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Researchers routinely use item responses on multi-point response scales to estimate individuals’ standing on constructs of theoretical and practical interest. For example a researcher might ask participants to rate their overall happiness on a scale from 1 to 5, where 1 is very unhappy and 5 is very happy. However, in addition to focal constructs (such as happiness), item responses may be meaningfully influenced by response styles, which are characteristic ways in which participants use response scales that may reflect personality differences which are stable across measures and over time and may interfere with the accurate measurement of focal constructs. Research on response styles has consistently demonstrated that individuals of Asian descent are less likely to respond in the extremes (e.g., use the endpoints of a multi-point rating scale, such as 1 or 5 on a 5-point response scale) relative to individuals from the US. This could complicate the interpretation of group differences if individuals with the same underlying levels on a construct (like happiness) appear to differ because of these cultural differences in extreme response style, which may instead reflect differences in how it is socially acceptable to portray oneself in the US (which tends to promote bold individualism) and Asian cultures (which tend to emphasize the importance of group harmony). This finding has become a common motivating example for considering the potential impact of extreme response style in cross-cultural comparisons because ignoring its impact could negatively affect the validity of test score interpretations and uses. The present research focuses on statistical models for six correlated dimensions of subjective well being and a general extreme response style dimension for college students in the US (N = 407) and China (N = 398). Participants completed the Ryff Scales of Psychological Well-Being, rating nine items (e.g., “In general, I feel confident and positive about myself.”) for each well-being dimension on a six-point Likert-type scale (1 = strongly disagree, 6 = strongly agree). A six-factor correlated traits model was fit to the data (separately for each country) using categorical confirmatory factor models (psychometric models that are appropriate for item responses in ordered categories) to provide evidence that the six-dimensional structure was plausible in both countries, and the models provided a close fit to the data, suggesting that participants’ responses to the 54 items were reasonably well-described in terms of six dimensions: positive relationships with others, mastery over one’s environment, personal growth, purpose in life, a sense of autonomy, and self-acceptance. The model that is the focus of this investigation can be viewed as a multi-trait cross-cultural extension of the Two-Decision Model (TDM), which posits that two distinct processes underlie responses to items on Likert-type scales, as a person first chooses whether to respond positively, neutrally (if a middle option is available), or negatively, and then determines the extremity of the response. It represents a departure from the typical assumption that Likert-type scales result in ordinal- or interval-level measurement because different patterns of item responses are linked to distinct constructs. The results of this model were compared to a traditional sum score-based model (in which item responses for each dimension are added together to create a total score) in order to gauge the practical impact of using the TDM rather than traditional classical test theory (sum score-based) methods. Results based on the traditional method suggested that well being among the US students was higher than or comparable to well being among Chinese students across the six dimensions. On the other hand, TDM results, controlling for extremity (which was greater among US students), suggest that the Chinese students reported more positive relationships with others, mastery over their environments, and personal growth, comparable levels of purpose in life, and less autonomy and self-acceptance. This work highlights the importance of thinking critically about group differences that are identified based on traditional sum score models, and the potential usefulness of psychometric models that can take into account the influences of response styles in psychological data. The use of these models may lead to more scientifically sound inferences in cross-cultural comparisons.
Description
Social and Behavioral Sciences; Social Work; Law: 1st Place (The Ohio State University Edward F. Hayes Graduate Research Forum)
Keywords
multidimensional item response theory, cross-cultural measurement, psychological well-being, extreme response style