Julie Hollek
Sessions
Are you a first time SciPy attendee? Confused about what the conference elements are or how to get the most out of your experience? Come join us for a casual chat, orientation, and suggestions for maximum fun and learning!
Generative AI seems like it’s everywhere and attendees of this very conference have built the technical foundations that have enabled its explosive growth. However, unlike the scientific computing software that we typically build, the rapid adoption of generative AI has not been met with the type of rigorous quality control that is required of powerful systems and expected of scientific endeavors. While there are many efforts around assessing the performance of generative AI systems, (e.g., benchmarks, human-in-the-loop AI red teaming, etc.) these methods often lack the rigor and context to make them truly scientific evaluations. In this BoF, we will host a community conversation to discuss the requirements to claim that a generative AI evaluation is scientifically sound while also maintaining subject matter expertise, relevance, and actionability. The SciPy Conference is an excellent forum for this discussion, bringing together scientists, developers, and practitioners.