BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.com//bbuzz22//speaker//LKGJ7S
BEGIN:VTIMEZONE
TZID:Europe/Berlin
BEGIN:DAYLIGHT
DTSTART:20210614T000000
TZNAME:CEST
TZOFFSETFROM:+0200
TZOFFSETTO:+0200
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:20211031T030000
RDATE:20221030T030000
TZNAME:CET
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20220327T030000
RDATE:20230326T030000
TZNAME:CEST
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
SUMMARY:Open Science: Building Models Like We Build Open-Source Software -
  Steven Kolawole
DTSTART;TZID=Europe/Berlin:20220614T122000
DTEND;TZID=Europe/Berlin:20220614T130000
DTSTAMP:20261009T015706Z
UID:pretalx-bbuzz22-R9WGGV@pretalx.com
DESCRIPTION:Elevator pitch\nThe use of transfer learning has begun a golde
 n era in applications of ML but the development of these models “democra
 tically” is still in the dark ages compared to best practices in SWE. I 
 describe how methods of open-source SWE can allow models to be built by a 
 distributed community of researchers.\n---\nOver the past few years\, it h
 as become increasingly common to use transfer learning when tackling machi
 ne learning problems (e.g. the BERT model on HuggingFace Hub has been down
 loaded tens of millions of times). However\, pre-training often involves t
 raining a large model on a large amount of data. This incurs substantial c
 omputational (and therefore financial) costs\; for example\, Lambda estima
 tes that training the GPT-3 language model would cost around $4.6 million.
  As a result\, the most popular pre-trained models are being created by sm
 all teams within large\, resource-rich corporations. This means that the m
 ajority of the research community is excluded from participating in the de
 sign and creation of these valuable resources.\n\nHere\, I elaborate on wh
 y we should develop tools that will allow us to build pre-trained models i
 n the same way that we build open-source software. Specifically\, models s
 hould be developed by a large community of stakeholders who continually up
 date and improve them. Realizing this goal will require porting many ideas
  from open-source software development to building and training models\, w
 hich motivates many threads of interesting research and opens up machine l
 earning research for much larger participation.
LOCATION:Palais Atelier
URL:https://pretalx.com/bbuzz22/talk/R9WGGV/
END:VEVENT
END:VCALENDAR