Ask: Research and Methods. Volume 32, Issue 1 (2023)

Permanent URI for this collection

https://kb.osu.edu/handle/1811/103630

Issue DOI: https://doi.org/10.18061/ask.v32i1

Full Issue
Description | Full Text PDF

Front Matter
Description | Full Text PDF

Articles

Attention checks and how to use them: Review and practical recommendations
Muszyński, Marek pp. 3-38
Description | Full Text PDF

Studying Hard-to-Reach Populations applications using river sampling, The case of foster families
Skalec-Ruczyńska, Agata pp. 39-66
Description | Full Text PDF

Stability and Determinants of Social Acceptance of Innovative Sustainable Technologies. Evidence from a Two-Wave Panel Study
Buntfuß, Nelly; Holz, Manuel; Mayerl, Jochen pp. 67-90
Description | Full Text PDF

The Many Faces of Cognitive Labs in Educational Measurement
Arieli-Attali, Meirav; Katz, Irvin R.; Cayton-Hodges, Gabrielle pp. 91-120
Description | Full Text PDF

How do local political elites respond to a survey? Evidence from a large-scale councilors survey in Poland
Kroszka, Jan; Żółtak, Tomasz; Gendźwiłł, Adam pp. 121-143
Description | Full Text PDF

Back Matter
Description | Full Text PDF

Browse

Now showing 1 - 8 of 8

Front Matter (Volume 32, Issue 1, 2023)
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2025)
Back Matter (Volume 32, Issue 1, 2023)
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2025)
Full Issue (Volume 32, Issue 1, 2023)
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2023) Muszyński, Marek; Skalec-Ruczyńska, Agata; Buntfuß, Nelly; Holz, Manuel; Mayerl, Jochen; Arieli-Attali, Meirav; Katz, Irvin R.; Cayton-Hodges, Gabrielle; Kroszka, Jan; Żółtak, Tomasz; Gendźwiłł, Adam
Stability and Determinants of Social Acceptance of Innovative Sustainable Technologies. Evidence from a Two-Wave Panel Study
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2023) Buntfuß, Nelly; Holz, Manuel; Mayerl, Jochen
This study investigates the role of attitudinal factors in shaping the acceptance of emerging sustainable technology. Using panel data from two waves of surveys conducted in two districts of the city of Cologne, Germany (N = 204), this paper discusses the impact of environmental, prosocial, and technology attitudes, as well as beliefs about the costs and benefits of the technology. The study is part of an interdisciplinary research project, VertiKKA, funded by the Federal Ministry of Education and Research (BMBF). Autoregressive and cross-lagged structural equation models (SEM) are applied to examine the stability of attitudinal factors and their impact on the acceptance of the sustainable technology. The results of the study suggest that while most of the attitudinal factors analyzed show stable autoregressive effects, the acceptance of sustainable technology is more volatile. We also find that general attitudes have a significant impact on beliefs about benefits, while only general environmental attitudes have an effect on beliefs about costs. Both beliefs function as total mediators between general and specific attitudes, with benefits having a greater influence on acceptance than cost aspects. Overall, the study highlights the importance of considering attitudinal factors in understanding the acceptance of sustainable technologies. The results provide insights that could inform the design and promotion of sustainable technologies, as well as help policymakers and practitioners to develop effective strategies to increase their adoption.
How do local political elites respond to a survey? Evidence from a large-scale councilors survey in Poland
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2023) Kroszka, Jan; Żółtak, Tomasz; Gendźwiłł, Adam
Researchers use various techniques to increase the number of respondents and the representativeness of a survey sample. This is particularly challenging in the surveys targeting elites and narrow groups of professionals. In this paper, we investigate which subgroups of local elected representatives respond more eagerly to the CAWI survey. We also examine how the effort of collecting councilors’ e-mail addresses and sending them personal invitations may increase the likelihood of their participation in the survey. For this purpose, we analyze the original dataset from a large-scale survey of municipal councilors conducted in 2023 in Poland. The study targeted more than 3 thousand local councilors from 198 municipalities. We find that response rates do not differ substantially due to the basic individual characteristics of the councilors, but they are significantly higher in bigger municipalities. However, this correlation is fully mediated by the public availability of individual e-mail addresses. This variable is a highly relevant predictor of participation with average marginal effects of 12 p.p. on opening the survey and 11 p.p. on completing the questionnaire. Our findings have potential practical implications for designing sampling schemes in future local elite surveys. We emphasize the benefits of sending personal email invitations in CAWI surveys.
Studying Hard-to-Reach Populations applications using river sampling, The case of foster families
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2023) Skalec-Ruczyńska, Agata
This paper presents the application of non-probability sampling – the rivers sample method to reach a group of professional foster parents in Poland – case of hard-to-reach population. This is a population whose public, nationwide register does not exist and additionally, it is kittle known about it in terms of the characteristics of its members, therefore quota sampling was not feasible. River sample method involves recruiting respondents by inviting them to the survey while they are doing some other activity on the Internet. The invitation to take part in the survey can be made through online banners, advertisements, information in forums or e-mails and on thematic websites. It provides a quick, low-cost, flexible way of reaching the group of respondents who are the target but it has its limitations- under-coverage and self-selection – which may affect the bias in the sample and thus the possibility to generalise to populations. Despite those limitations the presented study obtained a sample that reflects the proportions regarding the known characteristics in the population of professional foster carers in Poland: their provincial distribution and diversity of forms. In general, the river sample method works efficiently especially for exploratory studies and hard-to-reach groups.
The Many Faces of Cognitive Labs in Educational Measurement
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2023) Arieli-Attali, Meirav; Katz, Irvin R.; Cayton-Hodges, Gabrielle
Cognitive labs are becoming increasingly popular over the past decades as methods for gathering detailed data on the processes by which test‑takers understand and solve assessment items and tasks. Yet, there’s still misunderstandings and misconceptions about this method, and there is somewhat skepticism about the benefits of the method as well as lack of best practices for using it. This study’s purpose was to clear out some of the misconceptions about cognitive labs, and specifically to show through theory and examples of use, the concrete benefits and best practices of cognitive labs in different stages of assessment development, ranging from early stages of conceptualizing and designing the task or item to later stages of gathering validity evidence for it. Previous literature review on the topic revealed that even the term “cognitive labs” describes different techniques, originated in three different fields of study (Arieli-Attali, King, & Zaromb, 2011): 1) Cognitive Psychology and Artificial Intelligence research (“Think Aloud” studies, e.g., Ericsson and Simon, 1993); 2) Survey development studies (“Cognitive Interviews”, e.g., Willis, 2005); and 3) software development studies (“Usability Test”, e.g., Nielsen and Mack, 1994). While the latter two fields draw from the first original method, the different terminology and practices might have been the cause for skepticism and avoidance of use in educational measurement. This study maps the various ways of applying the method, shedding light on which variation can be used in which context of assessment development, in order to answer the research questions. We conclude that while it is evident that uninterrupted think aloud is needed for collecting response process validity, more flexible techniques may be used in contexts of usability or for assessment fairness or accessibility purposes.
Attention checks and how to use them: Review and practical recommendations
(The Ohio State University Libraries in partnership with the Institute of Philosophy and Sociology, Polish Academy of Sciences, 2023) Muszyński, Marek
Web surveys dominate contemporary data collection in numerous disciplines within the broadly understood social sciences. However, this mode of data collection comes with additional challenges, particularly related to careless or insufficient effort responding (C/IER), which can distort study results and poses a direct threat to the validity. One of the recommended approaches to address this problem is using attention checks, which are additional tasks or items with objective answers that indicate attentive responding. Despite the potential benefits of attention checks, recent evidence suggests that they are still not sufficiently researched to justify their uncritical use in screening out inattentive participants. This article provides an abridged review of the attention checks literature, offers evidence-based practical recommendations, and highlights crucial gaps in research regarding attention checks. Evidence-based recommendations concerning the type, number, and placement of attention checks in a survey are presented. Generally, including more than one attention check in a survey is advisable, especially for longer surveys. Long instructed manipulation checks should be avoided, instead, covert attention checks, which are difficult for participants to identify, are recommended to reduce negative side effects such as noncompliance. In addition to attention checks, other criteria, such as item-level response time analysis, should be used in combination to identify inattentive participants. It is crucial to carefully analyse all data before making decisions about participant elimination. Ethical considerations related to the use of attention checks are also discussed, recognizing the importance of maintaining participant trust and understanding the potential impact on survey completion rates and data quality. Overall, attention checks hold certain promise as a tool to enhance data quality, but further research and a thoughtful implementation are necessary to maximise their effectiveness.

Browse

Recent Submissions