Azamat Omuraliev
AI engineer at Typetone, where I'm taming LLMs to automate end-to-end marketing.
We help unburden SMEs and solopreneurs from doing their content marketing, and this task is surprisingly hard for LLMs to solve yet!
In past lives personalized marketing at ING as a data scientist and ran a non-profit in Kyrgyzstan.
Session
04-25
13:20
30min
Is your LLM any good at writing? Benchmarking on creative writing and editing tasks
Azamat Omuraliev
Many LLM benchmarks focus on reasoning and coding tasks. These are exciting tasks! But the majority of LLM usage is still in writing and editing related tasks, and there's a surprising lack of benchmarks on these.
In this talk you'll learn what it took to create a writing benchmark, and which model performs best!
PyData: Natural Language Processing & Audio (incl. Generative AI NLP)
Platinum3