HellaSwag
Definition
HellaSwag is a benchmark for commonsense reasoning and plausible continuation of situations.
The dataset introduced in 2019 uses multiple-choice continuations and adversarial filtering to create examples that are difficult for models but usually easy for humans.
Simply explained
Like a test where a model must choose which continuation of an everyday scene is the most plausible.
Why it matters in practice
Relevant for standardized model comparisons, but only for a limited subset of language understanding and commonsense.
Example from business
A team uses HellaSwag as one of several benchmarks and supplements it with their own service dialogues and document tasks.
Limits and pitfalls
- Equating benchmark scores with real-world reliability; ignoring possible training data contamination; using only one score for model decisions.
Origin & history
Zellers et al. published HellaSwag in 2019 as an advancement of adversarially filtered commonsense tasks.
Use in mid-sized companies
- Possible application: technical model comparison within a broader evaluation.
Frequently asked questions
What does HellaSwag measure?
Plausible continuations in commonsense scenarios. Is a high score proof of general intelligence? No.
Sources
- HellaSwag · Zellers et al. (2019), HellaSwag
Sources checked on 10.10.2026.
Editorially reviewed on: