Hi all,
I'm currently a software engineer on the big data platform team in Pinterest where our focus is on the workflow platform. We are migrating from old systems to this new one, which is built around airflow. Prior to here, I was on the data engineering team at Pandora where I first started exploring workflow systems and brought in a service built around Airflow there.
Outside of software, I help my family operate two pizza shops so coding and cooking are in my DNA!
- Airflow as the next Gen of workflow system at Pinterest
Adam Boscarino is a Data Engineer at Devoted Health working to improve healthcare in America. Previously, he worked at DigitalOcean with open-source data tools like Apache Spark, Kafka, and Airflow. And before that, he worked at Fitbit making sure all your steps were properly counted using tools like Luigi, Looker, and Snowflake.
- From cron to Airflow on Kubernetes: A Startup Story
Aishwarya Sankaravadivel is a senior technologist leading the Scheduling Platform at PayPal having deep expertise in managing Airflow & Elasticsearch for global customers. She is a passionate software engineer having intense hands-on coding experience in Scala/Python and Reactive Programming. She has grabbed several awards like CPI India Champion, Wonder Woman, Star of the month for her exemplary work at PayPal. She has spoken at several conferences in India and Asia Pacific regions and also a Women in Data Science Ambassador who has organized multiple meetups in India. Besides work, she is an avid reader and reviewer of many books and novels.
- Data flow with Airflow @ PayPal
- Diversity - Making Airflow more welcoming community (Keynote 3)
- Talk by Google - Platinum Sponsor of the Airflow Summit
- Airflow at Société Générale : An open source orchestration solution in a banking environment
Work in Data Engineering at Wrike since August 2016.
Migrated product data engineering ETLs between Spark clusters
Leading the migration from our own hardware to GCP and BigQuery
Make data available to engineers across the company
- How do we reason about the reliability of our data pipeline in Wrike?
Experienced Solutions Architect with a demonstrated history of working in a variety of domains such as High-Performance Computing, Big Data Analytics, Data Engineering, Software Engineering, Platform Integration. 12+ years of multi-national experience with a strong academic background.
- Autonomous Driving with Airflow
I am a first generation white-collar worker and Salvadoran immigrant. I am currently a Data Engineer working on automating ETL pipelines for Data Analysts/Scientists. My focus is reproducible data infrastructure as code that is easy to stand-up and troubleshoot.
- Data Engineering Hierarchy of Needs
- Migration to Airflow Backport Providers
- Future of Airflow (KeyNote 2)
Data engineer at GoDataDriven, Airflow trainer and committer, and co-author of the (currently in progress) Manning book Data Pipelines with Apache Airflow.
In his daily job he helps companies become more data driven by building data solutions, and wants to combine cool data products with scalable and solid software. In the past years he worked at various companies such as Booking, ING and Unilever.
- Testing Airflow workflows - ensuring your DAGs work before going into production
Bolke de Bruin is VP of Apache Airflow and CTO of Wholesale Banking Advanced Analytics. Bolke is passionate about embedding new ideas in the Wholesale Banking organization and strives to make Wholesale Banking more data driven. Before joining ING in 2008 Bolke worked at the 2004 summer and 2006 winter Olympic Games managing the technology, communication and data requirements for all news & media feeds at two large event locations. Bolke has also run his own start up commercializing multi-touch technology. In his spare time, Bolke is a guest lecturer at the University of Amsterdam, fun father to Mattia (6) and Timo (2) and can be found surfing, obstacle running (Ever done a 15km Mud Run? – www.obstakels.com) or taking in a museum when the opportunity arises.
- Airflow then and Now (Keynote 1)
- Data DAGs with lineage for fun and for profit
I am a software engineer at Airbnb's data platform
- Airflow In Airbnb
- Airflow In Airbnb
- How AirBnB/Twitter/Lyft use Airflow - Keynote 4
Daniel Imberman is a full-time Apache Airflow committer, a digital nomad, and constantly on a search for the perfect bowl of ramen. Daniel received his BS/MS from UC Santa Barbara in 2015 and has worked for data platform teams ranging from early-stage startups, to large corporations like Apple and Bloomberg LP.
- Machine Learning with Apache Airflow
- Future of Airflow (KeyNote 2)
Software Engineering at Pinterest. Working on workflow systems, including Airflow.
- Airflow as the next Gen of workflow system at Pinterest
Blaine is Sr Data Engineer at One Medical where he builds solutions for data pipelines and infrastructure. Blaine has 15+ years of experience within Data Engineering at various companies including MySpace, Chegg, Linkedin, and Microsoft.
- Using Airflow to speed up development of data intensive tools
My name is Emil Todorov. I am Software Engineer @ Financial Times and I am part of the Data Platform team for almost year and a half. Tackling data at scale became my passion and I believe this is the future.
- Democratised Data Workflows at scale
- Achieving Airflow Observability
Software engineer with special interest in machine learning and data engineering. Graduated with a MS in Computer Science on September 2019 from University of Colorado Boulder. For his master thesis he designed and implemented an ingestion system for social media messages produced during disasters. This work currently helps analysts at the EPIC laboratory to analyze and support the information needs by members of the public during times of mass emergency. In his free time, he likes to ski, swim, take pictures and read science fiction novels.
- AIP-31: Airflow functional DAG definition
- Diversity - Making Airflow more welcoming community (Keynote 3)
Over the past decade, Hendrik has served 15 out of the top 20 US technology firms, helping organizations capitalize on their data assets. Currently, Hendrik is employed as Director of Analytics at Optum, part of Unitedhealth Group. At Optum, Hendrik's team leads research and innovation to identify data solutions that help make the healthcare system work better for everyone.
- Airflow as an Elastic ETL Tool
Itai Yaffe is a big data tech lead at Nielsen Identity Engine, where he deals with big data challenges using tools like Spark, Druid, Kafka, and others. He is also a part of the Israeli chapter's core team of Women in Big Data. Itai is keen about sharing his knowledge and has presented his real-life experience in various forums in the past.
- Migrating Airflow-based Spark jobs to Kubernetes - the native way
Jake fell in love with open source process thanks to the inclusiveness and helpfulness of the Airflow Community. He's blessed to be a part of the Google Cloud Professional Services family which enables him in making GCP easier to use by building OSS tooling to help our customers.
- Airflow CI/CD: Github to Cloud Composer (safely)
After working in all the engineering/team/project management functions up to a CTO. As CTO he built software house 10-fold: from 6 to 60 people. After few years of being the CTO, he decided to go back to full-time engineering role and he works as a Principal Software Engineer in his company (and is super happy about it). Jarek has been working as an engineer in many industries - Telecoms, Mobile app development, Google, Robotics and Artificial Intelligence, Cloud and Open Source data processing. Jarek is currently PMC and active committer in the Apache Airflow project.
- Future of Airflow (KeyNote 2)
- Production Docker Image for Apache Airflow
- Airflow Observability using Databand
Kamil is a geek and programming enthusiast. In his free time, he realizes his passion - programming. In the past, he worked with an NGO dealing with FOI. Then he worked on a startup in the blockchain area. Now he enjoys a great Apache Airflow project every day.
- Future of Airflow (KeyNote 2)
- Future of Airflow (KeyNote 2)
Engineering manager of Workflow Orchestration team in Airbnb. Apache Airflow PMC.
- How AirBnB/Twitter/Lyft use Airflow - Keynote 4
- Airflow In Airbnb
Leah Cole is a developer programs engineer at Google, working on Composer, Google Cloud’s hosted version of Apache Airflow. Previously, she worked at GE for on multiple projects in the industrial IoT space. Leah is a graduate of Carleton College, where she studied computer science and also took enough German to have a semi-accidental minor. Outside of work, Leah likes playing piano, traveling, and crocheting.
- From S3 to BigQuery - How A First-Time Airflow User Successfully Implemented a Data Pipeline
I am a Data Engineer at QuintoAndar.
I firstly graduated in System Analysis and Development and then studied Big Data and Data Management Intelligence at Polytechnic School of the University of Sao Paulo. I started working as a Data Engineer at QuintoAndar, where I knew Apache Airflow and its vibrant community.
Our data team daily works towards making our data platform better to continue supporting QuintoAndar as a data-driven company.
I am passionate about learning and sharing ideas, so I will be glad to talk about data architecture or any Airflow related topics.
- Effective Cross-DAG Dependency
- Talk by Polidea - Gold Sponsor of the Airflow Summit
- Airflow then and Now (Keynote 1)
I graduated from Virginia Tech in Spring of 2019 as a Computer Science Major. I have been working at Nielsen for about a year as a Software engineer in their Emerging Technologies Program. I work with Nielsen Digitals Site Reliability Engineering team and Collections Platform team. Over the course of my time here I have deep dived into Kubernetes to enable us too more easily create, maintain, and deploy our workflows, while also having much more control of our resources to reduce cloud infrastructure costs. I am passionate about the movement to cloud native services on Kubernetes and am determined to contribute to it. I have actively contributed to Airflows open source stable helm chart and plan on contribute to more open source projects in the future.
- Airflow on Kubernetes: Containerizing your Workflows
Big Data Engineer @ DXC Technology. 5+ years of experience as data engineer with strong technical background.
- Autonomous Driving with Airflow
My name is Mihail Petkov and I have more than 9 years of experience in the software industry. I'm currently working as a Big Data Engineer at Financial Times. Being a Big Data Engineer is really exciting. I personally enjoy being challenged and love solving complex problems on a daily basis.
- Democratised Data Workflows at scale
Geek & DevOps by nature
- Airflow at Société Générale : An open source orchestration solution in a banking environment
Naresh has about 12+ years of hands-on experience in big data and data warehousing technologies. Currently working as a Lead Data Engineer at PlayStation based of Los Angeles, CA. Prior to that, he had opportunities to work for various organizations such as GRUBHUB, Comcast, Dell and AT&T.
He holds a Bachelors degree in Electrical Engineering and a Masters Degree in Power Electronics.
Doing DIY home projects on weekends, reading tech blogs, crawling through LinkedIn network are few of his hobbies
God Of War is his favorite 1st party PlayStation game.
- Airflow - A beast character in the gaming world
Nehil Jain is a Senior Software Engineer at SnapTravel , a leader in the conversational commerce space that allows millions of users around the world to book travel via messaging. SnapTravel has raised over $22M and driven over $150M in hotel bookings, and Nehil is responsible for leading and scaling the team that handles all data and infrastructure.
Prior to SnapTravel, Nehil completed his undergraduate and graduate degrees in Engineering, conducted research at McGill University, and was an early-stage member of a genetics startup providing DNA analysis for athletics, injuries and nutrition. In addition to tackling the complex problems in scaling data within a hyper-growth tech startup, Nehil is an avid runner and enjoys pushing himself to new limits through competitive dragon boating.
- Building Reuseable and Trustworthy ELT pipelines (A templated approach)
Software Engineer II at Electronic Arts
- Scheduler as a service - Apache Airflow at EA Digital Platform
I live in Tel-Aviv with my wife, baby-daughter and dog. I used to teach Python classes, do freelance work and organize Python community gatherings (PywebIL and Pycon Israel). These days I'm working as a data engineering team lead a Bluevine.
- From Zero to Airflow: bootstrapping a ML platform
- Airflow - A beast character in the gaming world
I have been working on Airflow since I joined in Airbnb in 2018. It is also the time when I came to the data world. Before Airbnb, I worked for Citadel to build risk analysis tools, and in Groupon, I was part of the Users Service Team to manage users lifecycle. I am interested in different tech areas, including SOA, infrastructure, data, DevOps.
- Airflow In Airbnb
Preethi Ganeshan is currently a Software Engineer in the Data Platform team at Electronic Arts. She has made several contributions to the central data platform and ad-hoc analytics at EA which includes leveraging engines such as Hadoop, Hive, and Presto in addition to tools like Airflow and Superset.
- Scheduler as a service - Apache Airflow at EA Digital Platform
Rafael Ribaldo is a Data Engineering Manager at QuintoAndar, where he leads a high-performance team towards building a data platform that can scale, has high performance and, most of all, is out of this world.
Before starting a career in Data Engineering, Ribaldo spent a couple of years as a Software Engineer writing Java code for companies in the payment and beverage businesses. After discovering a passion for Big Data projects, Ribaldo now helps QuintoAndar to continue to be a data driven company by making data available in a whole new level.
Ribaldo's available for discussing data architectures and the main goals to achieve data drivenness. You can reach him at rafael.ribaldo@quintoandar.com.br.
- Effective Cross-DAG Dependency
Roi Teveth is a big data engineer at Nielsen Identity Engine, where he specializes in research and development of solutions for big data infrastructure using cutting-edge technologies such as Spark, Kubernetes and Airflow. Roi has a vast system engineering background and is a CNCF certified Kubernetes administrator.
- Migrating Airflow-based Spark jobs to Kubernetes - the native way
Ry is a founder and CTO of Astronomer, which is dedicated to helping enterprises realize the value of Apache Airflow. Prior to Astronomer, he co-founded Differential — a high-growth, venture capital-backed startup studio. He served as Chief Experience Officer of RecruitMilitary and VP/Technology of The Devine Group (both acquired), and in 1995 he established one of the first web interactive agencies in the United States.
- Improving Airflow’s user experience
Born in Bogotá Colombia in 1990, studied Systems Engineering and then migrated to Germany in 2012. Worked as a Software Engineer and always loved to work with data. I did my Master of Science in Business Intelligence & Analytics and still continue taking online courses to learn more about Data Science. Currently working as Business Intelligence Architect for Lovoo (a The Meet Group Dating App)
- Airflow the perfect match in our Analytics Pipeline
Tao Feng is a staff software engineer on the data product team at Lyft. Tao is currently working on various data products including Amundsen(github.com/lyft/amundsen), an OSS metadata discovery platform. Tao is a committer and PMC member on Apache Airflow. Previously, Tao worked on data infrastructure, tooling, and performance at LinkedIn and Oracle.
- How AirBnB/Twitter/Lyft use Airflow - Keynote 4
Tomek is a software engineer at Polidea and Apache Airflow committer. He is an open-source enthusiast and chapter lead of ALC Warsaw. Tomek is a book and philosophy lover with a big interest in financial markets.
- Future of Airflow (KeyNote 2)
Placeholder biography
- Demo: Reducing the lines, a visual DAG editor
Vanessa is a research software engineer for the Stanford Research Computing Center. She received her PhD in Biomedical Informatics in 2016, and stayed at Stanford to focus on open source software development for scientific reproducibility. Her work includes development of container technologies, workflow software, and recipes for continuous integration. She is passionate about programming and system design, and continues to run the Singularity Hub container registry and maintain a large set of open source libraries. When not programming, Vanessa can be found eating avocados, recording podcast episodes or fun videos, making dinosaur noises, and running outside in the snow.
- Adding An Executor to Airflow: A Contributor Overflow Exception
- Airflow as an Elastic ETL Tool
Victor Shafran is cofounder of Databand, an APM and observability solution for data engineering teams. Victor brings 20 years of experience in enterprise software and data product development. In his last position as VP R&D at Equalum, a high growth startup in the data pipelining space, he led a team developing big data infrastructure for Fortune 100 companies. Before that, Victor was Director of Research at SAP and NICE Systems, where he led a team of data scientists on machine learning research. Victor holds an MBA from Tel Aviv University and an M.Sc (cum laude) in computer science.
- Pipelines on Pipelines: Agile CI/CD Workflows for Airflow DAGs
My name is Xiaoqin, I'm a Software Engineer at Electronic Arts from Austin, Texas. I started to work on Apache Airflow last summer, I'm excited to see and share how we far we have went and leveraging Apache Airflow at EA.
- Scheduler as a service - Apache Airflow at EA Digital Platform
Software Engineer at Airbnb, working on Airflow since 2018. Before joining Airbnb, I worked for Foxit Software, Inc. and GE Digital in Silicon Valley, being responsible for the development of website and back-end Management Information System. Interested in challenges from different tech areas.
- Airflow In Airbnb
I'm a Software Engineer @ Pinterest Workflow Team in the Data Org. I have been working on Airflow for almost a year and the team is in the progress of migrating from the old workflow platform to Airflow. Before joining Pinterest, I received my master degree from Carnegie Mellon University.
- Airflow as the next Gen of workflow system at Pinterest