Kahneman–Tversky Optimization (KTO)
Definition
KTO is an alignment method for language models that can use binary desired or undesired examples instead of pairwise preference data.
The method formulates a prospect-theory-inspired utility function and optimizes desired versus undesired responses. It is a post-training approach and not a general inference method.
Simply explained
How to fine-tune a model with "good" and "bad" rated answers without always needing answer pairs.
Why it matters in practice
Relevant for specialized model post-training; usually unnecessary for normal AI usage.
Example from business
A research team compares KTO and DPO on a domain-specific preference dataset.
Limits and pitfalls
- Portraying KTO as universally better; using weak labels; equating offline benchmarks with production behavior.
Origin & history
Ethayarajh et al. published KTO in 2024 as "Model Alignment as Prospect Theoretic Optimization."
Use in mid-sized companies
- Possible application in medium-sized businesses: Only consider when adapting your own model with reliable data and an evaluation pipeline.
Frequently asked questions
Does KTO require preference pairs?
No, binary desirability signals can suffice. Is KTO prompting? No.
Sources
- KTO · Ethayarajh et al. (2024), KTO
Sources checked on 10.10.2026.
Editorially reviewed on: