Skip to content
    Flow-alerta.cab

    GPQA

    AI fundamentals & modelsGraduate level Google-Proof Q&AGPQA benchmark
    Editorially reviewed on: 3 sources
    Your reading view

    The short answer · 13 words

    In short

    GPQA is a sophisticated multiple-choice benchmark with questions from biology, physics and chemistry.

    Flow-alerta.cab (2026): GPQA. https://flow-alerta.cab/en/glossar/gpqa. Editorially reviewed on: 10 October 2026.

    How much detail would you like?

    Practical

    GPQA can show model differences, but is only limitedly meaningful for typical corporate tasks.

    Hypothetical example: A team documents GPQA as an external indicator, but decides with its own specialist tests.

    Use GPQA to supplement and prioritize tests with tasks that actually occur in your company.

    At a glance

    Core
    Published in 2023, the dataset contains 448 questions created by professionals. It measures performance on this set, not general intelligence or scientific reliability.
    Use
    GPQA can show model differences, but is only limitedly meaningful for typical corporate tasks.
    Example
    Hypothetical example: A team documents GPQA as an external indicator, but decides with its own specialist tests.
    Limits
    Benchmark contamination; mixing variants like Diamond; concealing prompt and tool access; interpreting score as an overall capability.

    In context

    Benchmark

    Standardised task, test collection or measurement method used to evaluate systems under the most comparable conditions possible.

    Questions about this term

    How big is GPQA Main?

    448 questions.

    What is GPQA Diamond?

    A subset of 198 questions for stricter criteria for expert validation and difficulty for non-professionals.

    All details and background

    Published in 2023, the dataset contains 448 questions created by professionals. It measures performance on this set, not general intelligence or scientific reliability.

    ELI5 – Explained simply

    A heavy technical question test compares models under specified conditions.

    Why it matters in practice

    GPQA can show model differences, but is only limitedly meaningful for typical corporate tasks.

    Example

    Hypothetical example: A team documents GPQA as an external indicator, but decides with its own specialist tests.

    Pitfalls and limits

    • Benchmark contamination; mixing variants like Diamond; concealing prompt and tool access; interpreting score as an overall capability.

    Origin & history

    Rein et al. published the preprint in November 2023; the work appeared at the First Conference on Language Modeling in 2024.

    Use in mid-sized companies

    • Use GPQA to supplement and prioritize tests with tasks that actually occur in your company.

    Related terms

    • Benchmark
    • Human evaluation
    • MMLU
    • Scalable supervision
    • Test contamination

    Matching Flow-alerta service

    AI opportunity analysis & roadmapAI consulting: Examine tasks, data and prerequisites of a possible AI use case.

    Matching tool / use case

    View use casesOfficial GPQA dataset and baseline scripts; documented variant, prompt and tool access.

    Sources

    Quellenprüfung: 10.10.2026. Definition und dokumentierte Einordnung abgeglichen; Beispiele beschreiben mögliche Anwendungen.

    Editorially reviewed on:

    Next step

    Put knowledge into practice

    Discuss your projectView use cases