Background

The growing integration of AI in research and everyday applications brings significant challenges related to ethics, compliance, and reproducibility, which are key pillars for maintaining our trust in AI's capabilities. Compliance challenges in AI applications arise from the need to navigate evolving regulatory standards, ensure fairness and transparency, and maintain data privacy and accountability across diverse and complex systems. Bias related issues , for example bias across demographic groups pose serious risks, while reproducibility issues, such as incomplete experiment documentation or unstable software environments, further erode confidence in AI-driven applications. To tackle these problems, we present a Retrieval-Augmented Generation (RAG) framework that combines vector-based retrieval methods (e.g., FAISS for fast data access) with transformer-based language models (e.g., Mistral) to deliver transparent, standards-compliant recommendations. We also familiarize participants with complementary tools like IBM’s AI Fairness 360. that support bias detection and mitigation through metrics such as disparate impact and reweighing techniques.

Significance

As AI becomes central to research across disciplines, maintaining ethical standards and reproducibility is essential. Compliance with frameworks like the EU Artificial Intelligence Act and FAIR principles helps prevent biased outcomes and irreproducible results that can damage scientific credibility. In fields such as biomedicine, social sciences, and environmental science, biased or unreliable models can have serious consequences. To address these challenges, this tutorial offers hands-on exercises with a Retrieval-Augmented Generation (RAG) compliance plugin and the IBM AI Fairness 360 toolkit. These tools help researchers detect bias, improve transparency, and ensure reproducibility, supporting the creation of trustworthy, accountable AI systems across diverse domains.

Objectives

The tutorial aims to teach participants about the concepts like ethical compliance, RAG capabilities and tools for estimating and mitigating bias in AI workflows. We will have focused demos using the python RAG plug-in and Fairness 360 that will show how these tools solve compliance issues, using a healthcare case study. Participants will gain skills in ethical AI checks and reproducible methods, preparing them to meet global standards in their research.

Tutorial Breakdown

• Introduction (5 min): Describe AI research challenges, like bias and reproducibility, introduce RAG, Fairness360, noting links to standards like the European Union Artificial Intelligence Act and FAIR principles.

• Setup and Data Preparation (15 min): Look at setting up a Jupyter environment (Python 3.8+, FAISS, Transformers, Fairness360, MLflow, Docker, Plotly, Gradio) and review a synthetic healthcare dataset (10,000+ records, World Health Organization-aligned, with age, gender, ethnicity, and ICD-10 codes) and a pre-built compliance knowledge base (EU AI Act, FAIR, GDPR).

• Demonstration 1: Retrieval-Augmented Generation (35 min): Explore RAG’s system, showing vector-based retrieval (e.g., FAISS searching EU AI Act rules on data use) and LLM generation (e.g., Mistral creating compliance tips). Demonstrate the plug-in’s process for finding standards and making checks in the healthcare case study, like ensuring GDPR-compliant data use for diagnostic models, with MLflow tracking results.

• Demonstration 2: Fairness360 (35 min): Study Fairness360’s bias-checking tools, calculating measures like disparate impact and statistical parity for the healthcare diagnostic model (e.g., comparing accuracy for gender and ethnicity groups). Show how to fix bias (e.g., reweighing to balance groups), using Plotly to display bias measures (e.g., bar charts of disparity) and Gradio to interact with audit reports, focusing on fair results.

• Q&A and Wrap-Up (5 min): Talk about uses in biomedicine (e.g., fair diagnostics), social sciences (e.g., unbiased data analysis), and environmental science (e.g., repeatable climate models), and answer questions to clarify concepts.

Outcomes

Participants will deeply understand Retrieval-Augmented Generation and Fairness360, and see how they work in a compliance plug-in. They will build skills in ethical AI checks and reproducible methods, ready to use in biomedicine, social sciences, and environmental science, meeting standards like the European Union Artificial Intelligence Act and FAIR principles.