BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//pretalx//pretalx.com//pyconde-pydata-berlin-2023//speaker//N3BPWK
BEGIN:VTIMEZONE
TZID:CET
BEGIN:STANDARD
DTSTART:20001029T040000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
TZNAME:CET
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:20000326T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3
TZNAME:CEST
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
UID:pretalx-pyconde-pydata-berlin-2023-SJCEFG@pretalx.com
DTSTART;TZID=CET:20230418T114000
DTEND;TZID=CET:20230418T121000
DESCRIPTION:“Got an NLP problem nowadays? Use transformers! Just download
  a pretrained model from the hub!” - every blog article ever\n\nAs if it
 ’s that easy\, because nearly all pretrained models have a very annoying
  limitation: they can only process short input sequences. Not every NLP pr
 actitioner happens to work on tweets\, but instead many of us have to deal
  with longer input sequences. What started as a minor design choice for BE
 RT\, got cemented by the research community over the years and now turns o
 ut to be my biggest headache: the 512 tokens limit.\n\nIn this talk\, we
 ’ll ask a lot of dumb questions and get an equal number of unsatisfying 
 answers:\n\n1. How much text actually fits into 512 tokens? Spoiler: not e
 nough to solve my use case\, and I bet a lot of your use cases\, too.\n\n2
 . I can feed a sequence of any length into an RNN\, why do transformers ev
 en have a limit? We’ll look into the architecture in more detail to unde
 rstand that.\n\n3. Somebody smart must have thought about this sequence le
 ngth issue before\, or not? Prepare yourself for a rant about benchmarks i
 n NLP research.\n\n4. So what can we do to handle longer input sequences? 
 Enjoy my collection of mediocre workarounds.
DTSTAMP:20260415T092736Z
LOCATION:B05-B06
SUMMARY:Using transformers – a drama in 512 tokens - Marianne Stecklina
URL:https://pretalx.com/pyconde-pydata-berlin-2023/talk/SJCEFG/
END:VEVENT
END:VCALENDAR