Four categories, equal weight
Artificial Analysis has documented the methodology behind version 5 of its Intelligence Index, a composite intended to compare language-model performance across practical and academic tasks. The index contains nine evaluations and assigns equal weight to four categories: knowledge work, coding, science and general capability. Each category contributes 25% to the final score.
Knowledge work combines AA-Briefcase, AutomationBench-AA and GDP.pdf. Coding uses FrontierCode 2.0 and Terminal-Bench 4.0. Science combines Terminal-Bench-Science with a hard subset of CritPt v2. The general category uses AA-Omniscience v1.1 and HLE-Diamond. Individual tests carry different weights inside their categories, so the final number is a weighted composite rather than a simple average of nine percentages.
The suite is primarily text-based and English-language. Artificial Analysis evaluates multilingual ability, images, speech and other capabilities separately, meaning the Intelligence Index should not be read as an all-purpose measure of every way a model can be used. The publisher likewise describes the score as a synthesis with limitations rather than a guarantee of performance for a particular application.
For agent-style tasks, the methodology uses sandboxed environments and pinned software dependencies. The organization says e2b is its primary sandbox provider. Repository, terminal and research tasks can therefore test whether a model completes work under controlled conditions, rather than only whether it selects an answer from a list.
Grading and cost measurement
Different evaluations require different grading approaches. Multiple-choice responses are extracted through a sequence of patterns that can recognize a direct letter or a more formal answer format, with the last match used to account for self-correction. Open-ended benchmarks including HLE-Diamond use a language-model equality checker to judge whether a response is semantically equivalent to the reference answer. Scripted checks can supplement or override model-based judgments where the task supports objective validation.
Artificial Analysis maintains internal copies of evaluation datasets and publishes the origin of selected datasets. For its Intelligence Index, it uses token counts reported by model API providers when available, with a fallback tokenizer when they are not. Those counts are combined with observed cache-hit rates to calculate benchmark costs.
The methodology also avoids provider-specific document input features for GDP.pdf because those layers can introduce differences caused by the surrounding API product rather than the underlying model. As a result, its implementation is not directly comparable with every other implementation of the same task set.
The documented boundaries are as important as the score. A model can perform strongly on this English, text-centered suite while behaving differently on multilingual prompts, audio, images or a company's own workflow. Index v5 provides a structured comparison across its chosen tasks; selecting a model still requires testing the intended use case, reliability requirements and operating cost.
