BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.com//euroscipy-2025//speaker//TVZK37
BEGIN:VTIMEZONE
TZID:Europe/Warsaw
BEGIN:DAYLIGHT
DTSTART:20240821T000000
TZNAME:CEST
TZOFFSETFROM:+0200
TZOFFSETTO:+0200
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:20241027T030000
RDATE:20251026T030000
TZNAME:CET
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20250330T030000
RDATE:20260329T030000
TZNAME:CEST
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
SUMMARY:Processing Cloud-optimized data in Python (Dataplug) - Universitat
  Rovira i Virgili (Pedro Garcia Lopez)\, Daniel Alejandro Coll Tejeda
DTSTART;TZID=Europe/Warsaw:20250821T140500
DTEND;TZID=Europe/Warsaw:20250821T143500
DTSTAMP:20260911T010826Z
UID:pretalx-euroscipy-2025-SVTWWE@pretalx.com
DESCRIPTION:The elasticity of the Cloud is very appealing for processing l
 arge scientific data. However\, enormous volumes of unstructured research 
 data\, totaling petabytes\, remain untapped in data repositories due to th
 e lack of efficient parallel data access. Even-sized partitioning of these
  data to enable its parallel processing requires a complete re-write to st
 orage\, becoming prohibitively expensive for high volumes. In this article
  we present Dataplug\, an extensible framework that enables fine-grained p
 arallel data access to unstructured scientific data in object storage. Dat
 aplug employs read-only\, format-aware indexing\, allowing to define dynam
 ically-sized partitions using various partitioning strategies. This approa
 ch avoids writing the partitioned dataset back to storage\, enabling distr
 ibuted workers to fetch data partitions on-the-fly directly from large dat
 a blobs\, efficiently leveraging the high bandwidth capability of object s
 torage. Validations on genomic (FASTQGZip) and geospatial (LiDAR) data for
 mats demonstrate that Dataplug considerably lowers pre-processing compute 
 costs (between 65.5% — 71.31% less) without imposing significant overhea
 ds.
LOCATION:Room 1.20 (Ground Floor\, Shannon)
URL:https://pretalx.com/euroscipy-2025/talk/SVTWWE/
END:VEVENT
END:VCALENDAR