PyCon APAC 2023

Your locale preferences have been saved. We like to think that we have excellent support for English in pretalx, but if you encounter issues or errors, please contact us!

Auto-evaluation of ranking model by LLM
2023-10-27 , track 1

Discover an innovative Python framework for automating offline search engine ranking evaluation using LLM. By enabling search engineers and data scientists to perform offline evaluations rapidly, our method minimizes human bias, accelerates evaluation, and optimizes ranking algorithms swiftly, leading to improved search performance and more relevant user results.


Explore our groundbreaking framework that utilizes Large Language Models to automate the offline evaluation of search engine ranking results efficiently. By significantly reducing reliance on costly human labelers and minimizing the risk of human-induced bias, this technology allows search engineers and data scientists to swiftly conduct offline evaluations, resulting in a more seamless and efficient process. Our innovative solution empowers you to optimize search engine algorithms and deliver accurate, contextually relevant results to users, ultimately enhancing the overall search experience. Harness the power of large language models to transform your search engine’s performance, streamline the evaluation process, and reshape the way users interact with search technology.

See also: slides

Dr. Petrie Wong is a Staff Machine Learning Engineer at LegalOn Technologies, a leading LegalTech company specializing in contract management systems and AI-driven contract risk detection.