Description

CaselawQA is a benchmark for evaluating models on legal-text annotation and classification, comprising 260 legal annotation tasks, nearly all new to the machine learning community. It measures model accuracy across diverse empirical legal research tasks to compare prompting of commercial models versus fine-tuning open-source models.