Unintended Memorization
Definition
Unintentional memorization occurs when a model stores individual training contents instead of only general patterns.
As a result, rare or sensitive text sequences can be reproduced under certain conditions. The risk depends, among other things, on data, repetitions, and training.
Simply explained
Like accidentally memorizing a confidential sentence during practice.
Why it matters in practice
Confidential training data can pose a disclosure risk even after training.
Example from business
Hypothetical example: A team checks with artificial test markers whether a model repeats training sequences verbatim.
Limits and pitfalls
- A single inconspicuous test does not prove that nothing was stored; test secrets must not be real personal data.
Origin & history
Research investigates memorization, among other things, through targeted extraction attempts on language models.
Use in mid-sized companies
- Minimize training data and test models for undesired reproduction.
Frequently asked questions
Is every learned pattern already memorization?
No, here it specifically means storing concrete contents.
Sources
- The Secret Sharer · Carlini et al.: The Secret Sharer –
Editorially reviewed on: