The main obstacles to a truly open-source AI stack are fragmented and incompatible tooling, limited reproducibility, and the lack of mature pipelines for production environments. Our approach addresses these challenges through a multi-gateway architecture that decouples the model graph from the underlying execution environment. This allows a single logical model to run across different target systems while keeping data logic, storage formats, and runtime providers clearly separated.

Built on the DuckDB and SQLMesh frameworks, the platform can be reproduced and operated seamlessly both on a developer’s laptop and in the cloud—all from a single codebase. In the cloud, both compute jobs and the database are provided as serverless services, enabling automatic scaling without the overhead of server management. Compared with conventional commercial data transformation software, this toolset is particularly well suited to complex data transformations and machine learning workloads.

The framework equips organizations with a concrete architectural approach and a curated set of modular tools that can be integrated seamlessly. It enables teams to move quickly from local development and prototyping to production-ready, scalable pipelines. Analytical workloads are processed through clearly defined stages for data ingestion, transformation, and layering, ensuring robust governance throughout the entire data lifecycle.