Sonam Pankaj
Sonam is the creator of the open-source library called Embed-Anything, which helps to create local and multimodal embeddings and index them efficiently to vector databases, it’s built in rust and thus it’s more greener and efficient. She works as the GenerativeAI Evangelist at Articul8, spun-off of Interl, Articul8 is the go-to generativeAI platform for enterprise.
Session
Vector databases are everywhere, powering LLMs. But indexing embeddings, especially multivector embeddings like ColPali and Colbert, at a bulk is memory intensive. Vector streaming solves this problem by parallelizing the tasks of parsing, chunking, and embedding generation and indexing it continuously chunk by chunk instead of bulk. This not only increase the speed but also makes the whole task more optimized and memory efficient.
The library gives many vector database supports, like Pinecone, Weavaite, and Elastic.