In this mostly high-level talk, attendees interested in source code archival and code generation training datasets will get to know CodeCommons (https://codecommons.org), an ambitious project by a collaboration of the Software Heritage team at the French Institute for Research in Computer Science and Automation (INRIA), the French Alternative Energies and Atomic Energy Commission (CEA) and Modus Create.
CodeCommons is being built on the foundation of Software Heritage (https://www.softwareheritage.org), the world's largest publicly accessible source code archive. While Software Heritage currently comprises over 350,000,000 source code projects, it lacks licensing and important metadata about the context in which code was written.
As AI code generators become increasingly widespread, concerns over legal and ethical use of training data based on open-source projects are growing. CodeCommons extends Software Heritage with licensing information, comprehensive contextual metadata and filtering capabilities to create a large-scale reference dataset for training AI on code. It emphasizes traceability, author's rights, and ethical reuse, reducing the need for repeated, unsustainable data collection.
In this talk, we will introduce the audience to the Software Heritage project, then explore the changes CodeCommons will introduce and finally present in more detail Modus Create's work on the CodeCommons distributed computing, storage and querying infrastructure.