Dictionary, Run-Length, Delta and Bit Packing

• Dictionary encoding replaces each value with an index into a table of distinct values. The column becomes a small dictionary plus an array of narrow integers. Its effectiveness is governed by one number: distinct values relative to total values. A country column is nearly free; a session_id column is worse than plain, because the dictionary is as large as the data and the indices are pure overhead (Cardinality: The Label That Took Down Monitoring).
• Run-length encoding replaces a run of identical adjacent values with the value and a count. It is governed by *adjacency*, not by cardinality — five distinct statuses in random order have no runs, and the same five sorted have five runs. This is why sorting is the single highest-leverage encoding decision available (Partitioning).
• Delta encoding stores the difference between consecutive values instead of the values themselves. It is for ordered numeric and temporal columns: an ascending id or a sequence of timestamps becomes a series of small numbers, which then bit-pack tightly. It does nothing for an unordered column and nothing for a string.
• Bit packing stores integers in the minimum number of bits their observed range requires rather than in a fixed 32 or 64. It is the finisher applied on top of the others: dictionary indices, run lengths and deltas are all small integers, and bit packing is what turns "small integer" into "few bits" (Bits, Bytes and Words — and Why "Word" Is Not a Fixed Size).
• These compose. The common case in a real Parquet page is dictionary encoding producing indices, run-length-plus-bit-packing applied to those indices, and a general-purpose codec applied to the result. Three layers, each exploiting a different kind of redundancy, and only the last one is what most people mean by "compression".
• The property that makes these more than size reductions is that some of them are evaluable in place. A predicate country = 'DE' can be answered by finding DE in the dictionary once and then comparing narrow integers, and a run-length-encoded run can be skipped or accepted wholesale. That is a CPU saving on top of the I/O saving, and it is why the encoding layer is not interchangeable with the codec layer (Vectorized Execution).

1-- Run this against the source before arguing about codecs.
2-- The ratio, not the row count, decides what the encoder can do.
3 
4SELECT
5  'country'                                       AS column_name,
6  COUNT(*)                                        AS n_rows,
7  COUNT(DISTINCT country)                         AS n_distinct,
8  COUNT(DISTINCT country) * 1.0 / COUNT(*)        AS distinct_ratio
9FROM events
10WHERE event_date = DATE '2026-08-25'
11 
12UNION ALL
13 
14SELECT 'session_id', COUNT(*), COUNT(DISTINCT session_id),
15       COUNT(DISTINCT session_id) * 1.0 / COUNT(*)
16FROM events
17WHERE event_date = DATE '2026-08-25';
18 
19-- distinct_ratio near 0  -> dictionary is close to free
20-- distinct_ratio near 1  -> dictionary is overhead; the writer will fall back
21--                            to plain, and no codec setting will change that

country column, 12 values, 3 distinct

  PLAIN
    "DE" "DE" "FR" "DE" "IT" "FR" "DE" "DE" "DE" "IT" "FR" "DE"
    -> 12 variable-length strings, each stored in full

  DICTIONARY
    dict : [0]="DE"  [1]="FR"  [2]="IT"          <- stored once per chunk
    data :  0 0 1 0 2 1 0 0 0 2 1 0              <- one small int per row
    -> then BIT PACKED: 3 distinct values need 2 bits, not 32

  DICTIONARY + SORTED BY country
    dict : [0]="DE"  [1]="FR"  [2]="IT"
    data :  0 0 0 0 0 0 1 1 1 2 2 2
    -> then RUN-LENGTH: (0 x6) (1 x3) (2 x3)     <- three pairs, not twelve ints

  Same values. The third arrangement is the second one plus a sort,
  and the sort is the part that costs a shuffle at write time.

• Give columns their real types. A date stored as a string, a number stored as a string, a boolean stored as 'Y'/'N' — each of these removes the encoding that was designed for it, and the fix is a cast at the staging boundary rather than a codec setting (SQL Transformations).
• Sort or cluster on the low-cardinality column you filter on. That single choice converts dictionary encoding into run-length encoding and simultaneously makes chunk statistics selective — one decision, two independent wins (Clustering and Sort Order).
• Keep cardinality in mind when you add a column. A high-cardinality identifier added to a wide fact table is a permanent, uncompressible tax on every scan of that table, and if it is only needed occasionally it belongs in a side table.
• Reduce precision where the business does not need it. Truncating an ingested_at to the second rather than the microsecond creates runs where there were none, and nobody downstream can tell the difference — but confirm that with the consumer rather than assuming it (Who Actually Consumes This Data).
• Read the footer before tuning anything. Every Parquet and ORC writer records the encodings it used and the compressed and uncompressed size per column chunk. That is a measurement of your data rather than a benchmark of someone else's (Parquet Internals).

• All four encodings are lossless and fully reversible. A reader reconstructs the exact original values, including the ones a human would consider redundant.
• The writer chooses encodings; the reader must support all of them. This means encoding choice is never a compatibility contract with consumers — but it *is* a compatibility contract with reader library versions, since a newly-added encoding is unreadable by older readers.
• No encoding guarantees a size reduction. Dictionary encoding on a unique column reliably produces more bytes than plain, which is why writers implement a fallback rather than trusting the encoding.
• Predicate evaluation against encoded values is an engine capability, not a format guarantee. Two engines reading the same file can differ completely in whether they decode before filtering (Query Engines).