LLM benchmarks and leaderboards (November 2026): what each one measures, who runs it, and how to read a score
Quick answer
No single LLM benchmark ranks every model: each leaderboard measures one thing. Arena ranks models by people's blind votes, Artificial Analysis by its own Intelligence Index of 9 evaluations plus speed and price, Terminal-Bench and SWE-bench by coding tasks an agent resolves, Humanity's Last Exam by expert questions answered, and ARC-AGI by score against cost per task (each owner's page, read on November 18, 2026). To compare models, pick the leaderboard that matches your job, check which agent and effort setting produced the number, and run your own tasks with a tool such as Inspect, lm-evaluation-harness, or promptfoo.
Free plan: no credit card, 1 published page, 10,000 page views a month.
On this page
- LLM benchmarks and leaderboard websites: the comparison table
- Which LLM benchmark to read for which question
- AI model comparison charts: what the leaderboards showed on November 18, 2026
- LLM benchmarks for coding: SWE-bench, Terminal-Bench, and Aider
- Open-source LLM benchmarks and Hugging Face leaderboards
- How to read a benchmark score a vendor reports
- LLM benchmarking tools: run a benchmark yourself
- A benchmark of your own, in 4 steps
- Frequently asked questions
- Sources
You want to know which AI model is better, you open a leaderboard, and three sites give you three different orders. They aren't disagreeing: each one measures something different. This page lists the main public LLM benchmarks and leaderboard websites, says what each one measures and how, and shows how to read the score a vendor prints in a launch post.
Every description below comes from the benchmark's own website or repository, read on November 18, 2026 (Japan time). This page builds no ranking of its own: where a number appears, it is one leaderboard's number, with that leaderboard named.
LLM benchmarks and leaderboard websites: the comparison table
Public LLM benchmarks and leaderboards, read on November 18, 2026.
| Benchmark or site | What it measures | Who runs it | How it scores | Where to read it |
|---|---|---|---|---|
| Arena (formerly LMArena, Chatbot Arena) | Which answer people prefer | Arena | "In battle mode, you'll be served 2 anonymous models." You vote, and votes "shape the public AI leaderboards" | arena.ai/leaderboard |
| Artificial Analysis | Intelligence, speed, and price of models and API providers | Artificial Analysis: "Independent analysis of AI" | Intelligence Index v5.0.1: 9 evaluations, "a weighted average across four categories: Knowledge Work (25%), Coding (25%), Science (25%), and General (25%)" | artificialanalysis.ai |
| Terminal-Bench | Coding agents on tasks in a terminal | "Hosted by Stanford / Harbor / Laude Institute" | Resolution rate of Terminal-Bench 4.1 tasks, ± a 95% confidence interval, with the agent, tokens, and cost of each run | tbench.ai |
| SWE-bench Verified | Fixing real GitHub issues | The SWE-bench team | % resolved on "a human-filtered subset of 500 instances". The default board is bash-only, run with mini-SWE-agent | swebench.com |
| SWE-Bench Pro | "challenging long-horizon software engineering tasks" | Scale AI | Resolve Rate: "the percentage of tasks an agent successfully resolves", on public, held-out, and private sets | labs.scale.com |
| Aider polyglot | Editing code in 6 languages | Aider | Percent correct on "225 challenging Exercism coding exercises across C++, Go, Java, JavaScript, Python, and Rust" | aider.chat |
| Humanity's Last Exam (HLE) | Expert-level academic questions, text and images | Center for AI Safety and Scale AI | Accuracy and calibration error on 2,500 questions | lastexam.ai |
| ARC-AGI | Solving new puzzles; ARC-AGI-3 uses "novel interactive environments" | ARC Prize | Score, plotted against cost per task | arcprize.org/leaderboard |
| LiveBench 2.0 | Long real work: "decks, research reports, bug fixes, apps that run for hours" | Abacus AI | A weighted mean of 7 suites, and cost per successful task | livebench.ai |
| Vals AI | Industry tasks: "legal, finance, healthcare, coding, math, and more" | Vals AI | 42 benchmarks, each comparing "accuracy, cost, and latency"; the Vals Index combines them | vals.ai/benchmarks |
| OSWorld | Agents using a real computer desktop | The OSWorld team (xlang.ai) | Task success in "a first-of-its-kind scalable, real computer environment" | os-world.github.io |
| METR time horizons | How long a task an agent can finish | METR | The 50%-time horizon: the task length, in human expert time, "at which an agent is predicted to succeed half the time" | metr.org/time-horizons |
| HELM | Many scenarios and metrics per model | Stanford CRFM | Separate leaderboards: Capabilities, HELM Lite, MMLU, and more | crfm.stanford.edu/helm |
| Epoch AI benchmarks | Results over time, across benchmarks | Epoch AI | A database of results "administered internally by Epoch AI as well as data collected from external sources" | epoch.ai/benchmarks |
| OpenRouter rankings | Which models developers use | OpenRouter | "Models are ranked by tokens processed through the OpenRouter API" | openrouter.ai/rankings |
| MLPerf Inference | How fast hardware runs a model | MLCommons | It "measures how fast systems can process inputs and produce results using a trained model" | mlcommons.org |
Two dated notes. METR's page says: "This page is no longer actively updated." ARC Prize's says: "ARC Prize 2026 has finished", with results due December 4, 2026.
Which LLM benchmark to read for which question
| Your question | Read | Because |
|---|---|---|
| Which model do people like talking to? | Arena | It is the one built on human votes between two hidden models |
| Which model is best for coding agents? | Terminal-Bench, SWE-Bench Pro, SWE-bench Verified | Each runs an agent on real tasks and counts the ones it finishes |
| Which model edits code well in my language? | Aider polyglot | 225 exercises across 6 languages |
| Which model knows the most? | Humanity's Last Exam | 2,500 expert questions, scored on accuracy |
| Which model is the best value? | Artificial Analysis, LiveBench 2.0, Terminal-Bench | Each prints cost beside the score |
| Which model fits legal, finance, or healthcare work? | Vals AI | Benchmarks per industry |
| Which model can drive a computer? | OSWorld | Tasks in a real desktop environment |
| Which API provider is fastest for one model? | Artificial Analysis | It compares "API providers on speed, latency, price and endpoint accuracy for each model" |
| What are developers using? | OpenRouter rankings | Usage, by tokens processed |
| How fast is progress? | Epoch AI, METR | Results plotted over time |
| How fast is this GPU or server? | MLPerf Inference | It measures the system, with the model fixed |
AI model comparison charts: what the leaderboards showed on November 18, 2026
Each line is one site's own chart on the day this page was written. They rank different things, so they don't add up to one order.
- Terminal-Bench 4.1: Copilot Fusion (max) in GitHub Copilot leads at 71.2% ± 2.9%. Claude Opus 5.5 and Claude Fable 5.2, both in Claude Code, are tied at 65.8%, then Gemini 4 Carbon in Antigravity CLI at 64.5% and GPT-6.1 Sol in Codex at 60.9%. The board has 50 rows.
- Artificial Analysis Intelligence Index, highest effort: Claude Fable 5.2 (max) is listed first, then GPT-6.2 Astra (max), Gemini 4 Carbon (high), Claude Opus 5.5 (max), and GPT-6.2 Sol (max).
- Vals Index: "As of November 16, 2026, Claude Fable 5.2 ranks first on the Vals Index with 71.31%, followed by Claude Sonnet 5.5 (70.85%) and Gemini 4 Carbon (70.45%)."
- OpenRouter, weekly tokens: DeepSeek V4.1 Flash 1017 is first, with 61.2T tokens.
The same model can sit first on one chart and twentieth on another. DeepSeek V4.1 Flash 1017 is the most-used model on OpenRouter and 20th on Terminal-Bench 4.1, where its full run cost $0.2k against $16.5k for Claude Fable 5.2. For the models themselves, see the best AI models right now and Chinese AI models.
LLM benchmarks for coding: SWE-bench, Terminal-Bench, and Aider
Coding boards score an agent, not a bare model. The same model gets a different number in a different agent, so each row names both.
- Terminal-Bench 4.1 lists the agent beside every model: Claude Code, Codex, Antigravity CLI, GitHub Copilot, Grok Build, mini-swe-agent. It also prints the tokens and dollars each run used.
- SWE-bench has five boards: Verified, Multimodal, Multilingual, Lite, and Full. Verified "Defaults to bash-only setting (run with mini-SWE-agent)", which holds the agent fixed so the model is what changes.
- SWE-Bench Pro was built to be harder and cleaner. Scale's page says its public and held-out sets "use strong copyleft licenses (e.g., GPL)" to reduce the chance the code was in training data, and a private set comes from "proprietary codebases from startup partners".
- Aider polyglot reports the percent of 225 Exercism exercises a model gets correct, across 6 languages.
Vendors also report coding benchmarks that have no public leaderboard of their own, such as DeepSWE v1.1. Read those in the vendor's model card, with the harness note beside the number. DeepSeek vs Qwen vs Kimi for coding lays three labs' tables side by side.
Open-source LLM benchmarks and Hugging Face leaderboards
Hugging Face's docs describe the Hub as holding "leaderboards and evaluations for machine learning models, including LLMs, chatbots, and more", and the Open LLM Leaderboard runs there as a Space. Three more ways to compare open-weight models:
- LiveBench 2.0 has an "Open weights" filter on its leaderboard.
- Terminal-Bench 4.1 lists open-weight models in the same table as closed ones, such as Kimi K3.1 Pro at 59.7% and GLM-5.3 at 41.2%.
- Model cards on Hugging Face carry each lab's own table. Run the same benchmark yourself with the tools below to check it.
How to read a benchmark score a vendor reports
A launch post says "77.0 on DeepSWE" or "65.7% on Terminal-Bench". Before you compare it with another vendor's number, check five things:
- Which version. Terminal-Bench 4.0 and 4.1 are different task sets. HLE "has been finalized with 2,500 questions." A score on one version says nothing about another.
- Which agent, or harness. DeepSeek's model card: "We use the Claude Code harness for Terminal-Bench 4.0 and the Mini-SWE-V2 harness for DeepSWE v1.1." Moonshot's card for Kimi K3.1 Pro says it "is evaluated with the Claude Code harness". Two vendors using two harnesses have run two different tests.
- Which effort setting. Leaderboards print it: "(max)", "(xhigh)", "(high)". Artificial Analysis lets you switch its chart between "Highest" and "API Default".
- Whose run. A vendor's table often takes rivals' numbers from their own posts. Moonshot's card says other scores "are the maximum values reported in model blog posts". Its own blog reports 65.7% for Kimi K3.1 Pro on Terminal-Bench 4.0; the Terminal-Bench 4.1 leaderboard shows 59.7% ± 4.2%.
- The error bar and the cost. Terminal-Bench prints a 95% confidence interval of about 3 points, so a 1-point lead is a tie. It also prints cost: GPT-6.1 Sol's 60.9% run cost $0.6k, and GPT-6 Astra's 60.3% run cost $3.3k.
Artificial Analysis states its own rule in one line: "We generally use pass@1 scoring across our evaluations, where a model must produce the correct answer on its first attempt."
LLM benchmarking tools: run a benchmark yourself
Tools that score answers and agents.
| Tool | What it does, in its own words | Who makes it | Start with |
|---|---|---|---|
| Inspect | "a framework for frontier AI evaluations", covering "coding, agentic tasks, reasoning, knowledge, behavior, and multi-modal understanding" | UK AI Security Institute and Meridian Labs | inspect.aisi.org.uk |
| lm-evaluation-harness | "a unified framework to test generative language models on a large number of different evaluation tasks" | EleutherAI | pip install lm_eval[hf], or lm_eval[vllm] |
| Harbor | "a framework for specifying sandboxed agent tasks for evaluation and optimization", "from the makers of terminal-bench" | The Terminal-Bench team | uv tool install harbor |
| mini-SWE-agent | The agent SWE-bench Verified's default board runs | The SWE-bench team | swebench.com |
| promptfoo | "an open-source CLI and library for evaluating and red-teaming LLM apps" | promptfoo | promptfoo.dev |
| DeepEval | "similar to Pytest but specialized for unit testing LLM apps" | Confident AI | github.com/confident-ai/deepeval |
| Lighteval | "your all-in-one toolkit for evaluating LLMs across multiple backends" | Hugging Face | github.com/huggingface/lighteval |
| OpenCompass | "an LLM evaluation platform" with "100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety" | OpenCompass | github.com/open-compass/opencompass |
OpenAI's simple-evals carries a deprecation notice: since July 2025 it "will no longer be updated for new models or benchmark results."
Tools that measure speed (LLM performance benchmarking, local and GPU).
| Tool | Measures | Who makes it |
|---|---|---|
| llama-bench | "Performance testing tool for llama.cpp": prompt processing and text generation speed for a local model | llama.cpp |
| vllm bench serve | Throughput and latency of a vLLM server, including time to first token. vLLM's docs recommend GuideLLM for production servers | vLLM |
| MLPerf Inference | Standard load patterns across four scenarios, for comparing systems | MLCommons |
A benchmark of your own, in 4 steps
Public boards tell you which 3 models to try. Your own tasks tell you which one to keep.
- Collect 10 to 20 real tasks from your work, each with a way to check the answer: a test that passes, a field that matches, a reviewer's yes or no.
- Write them as a promptfoo config or an Inspect task, so every model gets the same prompt and the same check.
- Run each model at the effort setting you'll pay for, and record the pass rate and the cost.
- Rerun when a new model ships. The file is your leaderboard.
To send the results to your team, ask the model for one HTML file with the table and charts, drop it into Convika's publish tool, and share the link it gives you. Published from a free account, the page keeps one address you update after each rerun.
Frequently asked questions
What is the best LLM benchmark website?
It depends on the question. Arena ranks models by people's blind votes. Artificial Analysis compares intelligence, speed, and price. Terminal-Bench and SWE-bench score coding agents. Vals AI covers legal, finance, and healthcare tasks. OpenRouter's rankings show which models developers use most.
Is there an AI model comparison chart?
Yes, several. Artificial Analysis charts its Intelligence Index with speed and price beside it. Terminal-Bench has table, Pareto, and waffle views, with the cost of each run. LiveBench 2.0 plots its overall score against cost per successful task. ARC Prize plots score against cost per task.
Which LLM benchmark is best for coding?
Terminal-Bench, SWE-Bench Pro, and SWE-bench Verified score a coding agent on real tasks, and Aider polyglot scores code edits in 6 languages. On Terminal-Bench 4.1 on November 18, 2026, Copilot Fusion led at 71.2%, with Claude Opus 5.5 and Claude Fable 5.2 at 65.8%.
What tools can I use to benchmark an LLM myself?
Inspect, from the UK AI Security Institute and Meridian Labs, and EleutherAI's lm-evaluation-harness run standard evaluations. promptfoo and DeepEval test your own prompts and apps. Harbor runs agent tasks in sandboxes. For speed, llama-bench measures a local llama.cpp model and vllm bench serve measures a vLLM server.
How do I benchmark a local LLM or a GPU?
For a model on your own machine, llama-bench reports prompt processing and text generation speed. For a served model, vllm bench serve reports throughput and time to first token. To compare hardware, MLCommons' MLPerf Inference "measures how fast systems can process inputs and produce results using a trained model."
Why do vendors' benchmark numbers differ from the leaderboards?
They often ran a different version, agent, or effort setting. Moonshot's blog reports 65.7% for Kimi K3.1 Pro on Terminal-Bench 4.0, and the Terminal-Bench 4.1 leaderboard shows 59.7% ± 4.2%. Check the version, the harness, the effort, who ran it, and the confidence interval before comparing.
Is there an LLM leaderboard on Hugging Face?
Yes. Hugging Face's docs say the Hub holds "leaderboards and evaluations for machine learning models, including LLMs, chatbots, and more", and the Open LLM Leaderboard runs there as a Space. Hugging Face's Lighteval is the toolkit for running evaluations yourself.
How have LLM benchmark scores changed over time?
Epoch AI's benchmarks page keeps a database of results over time, from its own runs and outside sources. METR's time-horizon page tracks how long a task agents can finish, measured in human expert time; that page says it is no longer actively updated.
Sources
Each page was read on November 18, 2026, between 03:35 and 04:00 Japan time.
- Leaderboards: Arena: how it works, Artificial Analysis and its Intelligence Index methodology, Terminal-Bench, SWE-bench, SWE-Bench Pro, Aider LLM leaderboards, Humanity's Last Exam, ARC Prize leaderboard, LiveBench, Vals AI benchmarks, OSWorld, METR time horizons, HELM, Epoch AI benchmarks, OpenRouter rankings, MLPerf Inference: Datacenter, Hugging Face leaderboards docs, Open LLM Leaderboard
- Tools: Inspect, lm-evaluation-harness, Harbor, promptfoo, DeepEval, Lighteval, OpenCompass, simple-evals, llama-bench, vLLM benchmark CLI
- Vendor notes quoted: DeepSeek-V4.1-Flash-1017 model card, Kimi K3.1 Pro model card and blog post
Next step
How to password protect an HTML page or website
A JavaScript password script leaves the password in the page source. To password protect an HTML page or link, encrypt the file or let a server check it.
Related articles
- The best AI models right now (November 2026): GPT-6, Claude, Gemini, and Grok comparedGPT-6 Astra, Claude Fable 5.2 and Opus 5.5, Gemini 4 Carbon, and Grok 4.7 compared: release dates, context, API prices, plans, and each lab's own benchmarks.
- Chinese AI models in November 2026: the latest from DeepSeek, Qwen, Kimi, GLM, and MiniMaxEvery major Chinese AI model, from DeepSeek-V4.1 and Qwen4-Max to Kimi K3.1 Pro, GLM-5.3, and MiniMax M3.1: open weights, license, API price, and where to use them.
- What is the Riemann Hypothesis? A plain-English explanationThe Riemann Hypothesis in plain English: what the zeta function and its zeros are, why they control the primes, what is proven, and the November 2026 proof claim.
- DeepSeek vs Qwen vs Kimi for coding (November 2026): latest models, benchmarks, API prices, and where to run themDeepSeek V4.1 Flash, Qwen4-Max, and Kimi K3.1 Pro for coding: each lab's own benchmarks, API and OpenRouter prices, Claude Code, Cursor, Cline, and Ollama.
- GPT-6 vs Claude Opus 5.5 (November 2026): which is better for coding, and what each costsGPT-6 Astra vs Claude Opus 5.5 for coding: each lab's own benchmarks, API prices with long-prompt rates, plans, GPT-6 vs GPT-5.6, and Astra vs Fable 5.2.
- We gave 9 AI models the same landing page prompt: here's what they builtOne landing page prompt, sent once to GPT-6 Astra, Claude, Gemini 4, Grok 4.7, DeepSeek, Qwen, Kimi and GLM on Nov 16, 2026. Cost, time, phone test.
Skipbox
Cheap & fast GitHub Actions runners, unlimited for $39 a month
Your own machine keeps Docker layers and package caches between jobs. Switch by changing one runs-on line.
Start a free Skipbox trial