k-Armed Bandit
Definition
A k-armed bandit is a decision problem with k possible actions whose unknown rewards are learned through trial and error.
The core problem is the trade-off between exploring new options and exploiting the best option so far. Unlike full reinforcement learning, the classic bandit has no complex state transitions.
Simply explained
How to choose among multiple slot machines while simultaneously figuring out which pays off the most in the long run.
Why it matters in practice
Relevant for online experiments, recommendations, and adaptive selection when results are quickly observable.
Example from business
A system adaptively distributes traffic across multiple subject variants while maintaining controlled exploration.
Limits and pitfalls
- Equating bandit with A/B testing; ignoring delayed or biased rewards; stopping exploration too early; optimizing unsuitable objectives.
Origin & history
Bandit problems are a classic model of sequential decision theory and are extensively covered in reinforcement learning literature.
Use in mid-sized companies
- Possible applications: adaptive experiments, recommendation selection, and dynamic resource allocation.
Frequently asked questions
What does k mean?
Number of available actions. What is exploration? Deliberate testing of uncertain options.
Sources
- Reinforcement Learning: An Introduction, Chapter 2 · Sutton & Barto, Reinforcement Learning: An Introduction, Chapter 2
- Bandit Algorithms, Kapitel 4 und 6 · Lattimore & Szepesvári, Bandit Algorithms, Kapitel 4 und 6
Source check: 10./11.10.2026.
Editorially reviewed on: