Description

LitSearch is a benchmark for evaluating retrieval systems on realistic literature search queries that require deep understanding of ML and NLP research and reasoning across full articles. It comprises 597 queries created via (1) GPT-4-generated questions based on paragraphs containing inline citations from research papers and (2) questions manually written by authors about their recent papers, with all queries manually examined or edited by experts.