2024-11-16 –, LT9
Language: English
For data engineers to set up reliable data pipelines, it is crucial to conduct validations at each step to ensure the high quality of the data. Great Expectations (GX) is an open source framework that provides an intuitive way -- Expectations -- to define and manage data quality.
This lighting talk will explore how to leverage Great Expectations to automate data quality checks and increase transparency in your data pipeline. To demonstrate GX's features, we will explore two examples of Expectations (basic and conditional).
Meixin Wang is a data engineering analyst in Bloomberg's Data department in Hong Kong. She specializes in the Python programming language and has experience developing and growing automated processes and quality controls by using technology solutions for the company's Fixed Income data products. She is passionate about staying up-to-date with the latest technologies and trends in the field and will continue developing her skills to make a positive impact. She holds a master's degree in software engineering from Fudan University.