Retrieval-Augmented Generation (RAG) chatbots are a key use case of GenAI in organizations, allowing users to conveniently access and query internal company data. A first RAG prototype can often be created in a matter of days. But why are the majority of prototypes still in the pilot stage? [1]

In this talk we share our insights from developing a production-grade chatbot at Merck. Our RAG chatbot for R&D experts accesses over 50,000 documents across numerous SharePoint sites and other sources. We identified three key success factors:
1. Developing a robust data pipeline that syncs documents from source systems and that handles enterprise features such as replicating user permissions.
2. Establishing a comprehensive evaluation framework with a clear optimization metric.
3. Driving adoption through an onboarding training and ongoing user engagement, such as regular office hours.

We think that many of these lessons are broadly applicable to RAG chatbots, making this talk valuable for practitioners aiming to implement GenAI solutions in business contexts.