💫 Generative AI Leaderboard

This { _FRONTIER / #1 / #2 / #3 / SUBPAR } SCORE in the leaderboard is based on the following benchmarks:

1. Chatbot Arena Elo - a crowdsourced, randomized battle platform for large language models (LLMs). We use 5M+ user votes to compute Elo ratings. Arena also provides category-specific benchmarks (Coding, Vision, etc.).
2. AAII - Artificial Analysis Intelligence Index v3 aggregating 10 challenging evaluations.
3. ARC-AGI - Artificial General Intelligence benchmark v2 to measure fluid intelligence.
4. MMLU - Massive Multitask Language Understanding benchmark covering 57 subjects including STEM, humanities, and social sciences.

HM (Harmonic Mean) combines normalized Arena Elo, AAII, and ARC-AGI scores. HM=0 means missing benchmark data (HM cannot be computed).

Use-Case Filters: Sort models by task-specific performance using weighted benchmarks:

• Coding - Arena Coding benchmark (practical programming capability)
• Scientific Analysis - ARC-AGI 40% + MMLU 30% + Arena Coding 30% (reasoning + knowledge + computational implementation)
• Creative Writing - Arena Elo (human preference for writing quality)
• Scientific Writing - MMLU 70% + Arena Elo 30% (accuracy + clarity)

Specialized Leaderboards: External benchmarks for specific AI capabilities:

• Literature Understanding - Scientific literature agents (AstaBench) (Source)
• Genomics - Variant effect prediction benchmarks (benchmarks.bio) (Source)
• Spatial/Omics - Spatial and omics AI benchmarks (benchmarks.bio) (Source)
• Therapeutics - Drug discovery / therapeutics benchmarks (benchmarks.bio) (Source)
• Computational Biology - Computational biology tasks (Genentech CompBioBench) (Source)
• Biosecurity - Biosecurity surveillance benchmarks (benchmarks.bio) (Source)
• MedHELM - Holistic clinical LLM evaluation (Stanford CRFM) (Source)
• Search - AI search model rankings (Source)
• Image-to-Video - Video generation from images (Source)
• Vision OCR - Optical character recognition (Source)
• Text-to-SQL - Database query generation (Source)
• Agent - Autonomous AI agent rankings (Source)