In recent years, the explosion of data sources and data volumes available has made their management increasingly difficult for organizations. Companies are left with vast amounts of unstructured data such as text documents, images, or videos, which could now be unlocked thanks to the availability of high quality embedding models. However, this data is underutilized because its processing would require too much time, especially when its size reaches above the billions of records.
This talk will guide you through the domain of vector search, which are techniques that can be used to efficiently index and retrieve huge amounts of unstructured data. We will present classical and state of the art algorithms used in the most popular open-source vector search libraries (cuVS, FAISS, …), their tradeoffs, limitations, and how GPUs can accelerate those algorithms when the scale of data becomes challenging.
Additionally, we will present how those techniques can be applied on Milvus, an open-source vector database, to speed-up indexing workflows on billions of vectors. What used to take multiple weeks can be reduced to a few days, while keeping an exceptionally low search latency, suitable for recommendation, search, and RAG applications that operate over unstructured data embeddings.