Low-Rank Adaptation (LoRA)
Definition
LoRA adapts large models by training small low-rank matrices while keeping the original model weights frozen.
The method reduces trainable parameters and often memory requirements compared to full fine-tuning. Quality depends on rank, target modules, data, and task.
Simply explained
How to adapt a large model with small additional modules instead of rebuilding all weights.
Why it matters in practice
Relevant for more cost-effective fine-tuning of open-weight models.
Example from business
A company trains LoRA adapters for a clearly limited text classification domain and compares against prompting.
Limits and pitfalls
- Do not automatically equate LoRA with better quality or RAG; check license and data rights. Base version and adapter must match. Choose rank and training data based on independent evaluation; do not include test data in training.
Origin & history
Hu et al. published LoRA in 2021.
Use in mid-sized companies
- Possible application in SMEs: Use only if prompting/RAG is insufficient and sufficient high-quality training data is available.
Frequently asked questions
Does LoRA change base weights?
In the classical approach, they remain frozen. Is LoRA without training? No.
Sources
- LoRA · Hu et al. (2021), LoRA
Sources checked on 10.10.2026.
Editorially reviewed on: