ReviewArena
A large-scale cross-conference dataset and benchmark for LLM peer review
Overview
ReviewArena is a large-scale resource for studying automated scientific peer review. It brings together 51,529 papers and 196,099 reviews from seven OpenReview venues, preserving native review forms, scores, rebuttals, meta-reviews, decisions, and paper text.
ReviewArena-Eval
The project also introduces a 1,002-paper evaluation benchmark aligned to the scoring conventions of six conferences. Experiments with six open-weight language models show persistent problems with calibration, rating-scale compression, and accept/reject discrimination.
Recognition
Accepted as a Spotlight paper with a contributed talk at the ICML 2026 AI for Science Workshop in Seoul.
| OpenReview | Dataset |