2023-04-19 –, B07-B08
Working with python is fun.
Managing python packaging, linters, tests, CI, etc. is not as fun.
Every maintainer needs to worry about consistent styling, quality, speed of tests, etc as the project grows.
Monorepos have been successful in other communities - how does it work in Python ?
As a python project grows (within 2-3 years), you will go down either of these 2 paths:
- Create a monolith
- Modularize your code into smaller packages
In the current world, you will be affected by multiple other libraries you use. And modularity is a requirement for any good project.
But managing multiple modular packages becomes tough over time.
- How do you ensure coding standards (quality, styling, etc) is consistent across them ?
- How would we ensure all the pakages work correctly without spending hours and hours of CI time ?
- How can common logical pieces be modularized further and still be DRY ?
These are common issues I have faced by the 2-3 year mark in any active project. And if not solved quickly can easily cause your project to get messy very quickly.
This talk aims to discuss these common issues and how a monorepo structure which is widely popular in other communities like NodeJS can also be applied to python.
We also discuss how the crux of the issue:
- Making your code structure machine understandable
- How this structured information can then be used to optimize workloads
- How this structured information can be used to automate tasks
And also go into discussing how monorepo tools like pants, bazel, nx, etc. leverage this code structure information to simplify your life as a maintainer
None
Expected audience expertise: Python –Intermediate
Abstract as a tweet –Monorepos have been successful in other communities - how does it work in Python ?
Hi, I'm Abdeali Kothari - a.k.a Ali (if we're talking) or @AbdealiLoKo (if we're typing)
I graduated from IIT Madras and then worked with American Express, followed by Corridor Platforms where I am architecting a Decisioning platform for analytics in the Financial domain.
I've dabbled with Robotics, Operating System architectures, Machine Learning, Game Development, and Web Development a lot for a bunch of personal projects.
And worked mainly in Big Data, Machine Learning, and Analytics in the Financial Domain for enterprise-productional use-cases.
I'm a big fan of code hygiene and clean architecture. With a lot of Code Analytics experience under my belt.
And worked mainly in Python in all the above fields for about 13 years now (Back when the first blogpost telling us to stop using Python 2.x was written :D)
I'm extremely lazy - and hence an automation freak. And have created great automated test suites and CI/CD pipelines to help me remain lazy.