Eisele, G, Hiekkaranta, A, Kunkels, YK, Rot, MAH, van Ballegooijen, W, Bartels, SL, Bastiaansen, JA, Beymer, PN, Bylsma, LM, Carpenter, RW, Ellison, WD, Fisher, AJ, Forkmann, T, Frumkin, MR, Fulford, D, Naragon-Gainey, K, Greene, T, Heininga, VE, Jones, A, Kalokerinos, EK et al, Kuppens, P, Modecki, KL, Müller, F, Neubauer, AB, Panaite, V, Schneider, M, Sun, J, Wilson, SJ, Zygar-Hoffmann, C, Myin-Germeys, I and Kirtley, OJ (2025) ESM-Q: A consensus-based quality assessment tool for experience sampling method items. Behavior Research Methods, 57 (4). ISSN 1554-351X

Text ESM-Q A consensus-based quality assessment tool for experience sampling method items.pdf - Accepted Version Restricted to Repository staff only until 31 March 2026. Download (375kB)

Abstract

The experience sampling method (ESM) is increasingly used by researchers from various disciplines to answer novel questions about individuals' daily lives. Measurement best practices have long been overlooked in ESM research, and recent reviews show that item quality is often not reported in ESM studies. The absence of information about item quality may be partly explained by the lack of consensus on how ESM item quality should be evaluated. As part of the ESM Item Repository project (esmitemrepository.com)-an international open science initiative that collects ESM items in an open item bank and evaluates their quality-we brought together 42 international ESM experts to develop an ESM item quality assessment tool. In four Delphi phases, experts suggested 57 item quality criteria, rated the criteria, provided arguments for and against the criteria, and rated the criteria again, considering reflections from other experts. The result of the Delphi process is ESM-Q: a quality assessment tool consisting of 10 core criteria, as well as an additional 15 supplementary criteria, to be used depending on the type of items being rated and the availability of supplementary information. The criteria cover topics ranging from construct validity to the optimal wording of items. ESM-Q can aid ESM researchers in selecting existing ESM items, developing new high-quality ESM items, and evaluating the quality of ESM items in systematic reviews. Expert reflections also highlight open research questions surrounding ESM item design that form a research agenda for ESM measurement.

Item Type:	Article
Additional Information:	This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://doi.org/10.3758/s13428-025-02626-1
Uncontrolled Keywords:	Humans; Consensus; Research Design; Delphi Technique; Ambulatory assessment; Delphi study; Ecological momentary assessment; Item quality criteria; Questionnaire development; Humans; Delphi Technique; Consensus; Research Design; 5202 Biological Psychology; 5204 Cognitive and Computational Psychology; 49 Mathematical Sciences; 4905 Statistics; 52 Psychology; 8.4 Research design and methodologies (health services); Generic health relevance; Humans; Delphi Technique; Consensus; Research Design; 0801 Artificial Intelligence and Image Processing; 1701 Psychology; 1702 Cognitive Sciences; Experimental Psychology; 4905 Statistics; 5202 Biological psychology; 5204 Cognitive and computational psychology
Subjects:	B Philosophy. Psychology. Religion > BF Psychology
Divisions:	Psychology (from Sep 2019)
Publisher:	Springer
SWORD Depositor:	A Symplectic
Date Deposited:	07 Apr 2025 10:32
Last Modified:	07 Apr 2025 10:32
DOI or ID number:	10.3758/s13428-025-02626-1
URI:	https://researchonline.ljmu.ac.uk/id/eprint/26046

View Item