RMSprop
Definition
RMSprop adjusts learning steps based on previous gradient magnitudes.
An adaptive gradient method that scales the learning step of each parameter based on an exponentially weighted moving average of its squared gradients.
Simply explained
Parameters with consistently large update values receive smaller effective learning steps.
Why it matters in practice
The term helps to select, develop, evaluate AI systems in a well-founded manner and realistically assess their technical risks.
Example from business
Parameters with consistently large gradients receive smaller effective learning steps.
Limits and pitfalls
- Confusing RMSprop with RMSNorm, failing to document variant and hyperparameters, or guaranteeing convergence and better generalization.
Origin & history
RMSprop was introduced in Geoffrey Hinton's 2012 lecture and later widely adopted.
Use in mid-sized companies
- Possible application: Parameters with consistently large gradients receive smaller effective learning steps.
Frequently asked questions
What does RMSprop mean?
An adaptive gradient method that scales the learning step of each parameter based on an exponentially weighted moving average of its squared gradients. What mistakes should be avoided? Confusing RMSprop with RMSNorm, failing to document variant and hyperparameters, or guaranteeing convergence and better generalization.
Sources
Source checked on 10.10.2026. Editorial application examples are hypothetical; no evidence of real customer results.
Editorially reviewed on: