Martin Iglesias Goyanes
I work in Applied Machine Learning at Adyen, where I focus on deep learning research and training infrastructure. I am passionate about bridging SOTA deep learning with the scale and complexity of the financial industry.
Session
Pretraining foundation models on tabular and sequential data presents challenges that differ fundamentally from NLP or vision. This talk covers the key design decisions involved in building a payments foundation model trained on billions of transactions and trillions tokens: tokenisation of heterogeneous features, sequence construction, masking strategies and pretraining objectives for sequential tabular data, and the architectural trade-offs between hierarchical and language-model framings. Attendees will leave with transferable techniques for adapting self-supervised pretraining to non-text sequential data.