Data Layout
Definition
Data layout describes how values are arranged in memory or in a file.
A data layout defines the physical arrangement of data, for example row-wise or column-wise, as well as with specific spacing and data types. It influences how efficiently programs read and process values.
Simply explained
In a toolbox, it makes a difference whether all parts of a set are together or all similar parts are next to each other.
Why it matters in practice
Appropriate arrangements can reduce unnecessary memory accesses and facilitate cooperation between different programs.
Example from business
Hypothetical example: An analysis reads only two of twenty table columns and uses a column-oriented format for this.
Limits and pitfalls
- Logical table structure and physical storage are different. A favorable layout for one query can be unfavorable for another.
Origin & history
Data layouts are part of the fundamentals of memory management, file formats, and database systems.
Use in mid-sized companies
- For large analytical data, select file format and access pattern together.
Frequently asked questions
Is a layout the same as a schema?
No, a schema describes meaning and structure; the layout concerns their concrete arrangement.
Sources
- Columnar Format · Apache Arrow: Columnar Format
Editorially reviewed on: