BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.com//sips2026-dc//talk//YUA8FT
BEGIN:VTIMEZONE
TZID:EST
BEGIN:STANDARD
DTSTART:20001029T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10;UNTIL=20061029T070000Z
TZNAME:EST
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
END:STANDARD
BEGIN:STANDARD
DTSTART:20071104T030000
RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11
TZNAME:EST
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20000402T030000
RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4;UNTIL=20060402T080000Z
TZNAME:EDT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
END:DAYLIGHT
BEGIN:DAYLIGHT
DTSTART:20070311T030000
RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3
TZNAME:EDT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:pretalx-sips2026-dc-YUA8FT@pretalx.com
DTSTART;TZID=EST:20260609T140500
DTEND;TZID=EST:20260609T141000
DESCRIPTION:We conducted a study each year 2021-2024 comparing data quality
  across online platforms (e.g.\, Prolific\, CloudResearch) and university 
 laboratory pools (UC Berkeley\, UChicago). Participants completed identica
 l surveys with multiple quality assessments: “attention checks” (e.g.\
 , embedded instructions)\, “accuracy checks” (e.g.\, reading comprehen
 sion\, writing quality metrics)\, and self-reported attention and engageme
 nt measures. Three sets of results challenge common beliefs about data qua
 lity. First\, across years and platforms the quality measures correlated o
 nly weakly with each other (often r<.17)\, suggesting quality is a multi-d
 imensional construct. Second\, online participants outperformed lab partic
 ipants on traditional attention checks and self-reported attention\, but l
 ab participants performed higher on accuracy checks\, suggesting these poo
 ls have systematically different quality. Finally\, despite numerous chang
 es over the years we studied (e.g.\, AI)\, quality metrics remained relati
 vely stable. These findings suggest different platforms yield different ty
 pes of quality suited to different research needs.
DTSTAMP:20260524T193327Z
LOCATION:AUDITORIUM
SUMMARY:Assessment of Survey Data Quality: Four Years of Data Quality Measu
 res Collected in Online and Laboratory Participant Pools - Don Moore
URL:https://pretalx.com/sips2026-dc/talk/YUA8FT/
END:VEVENT
END:VCALENDAR