[jeffchen006]

This is the dictionary follow-up suggested at the end of #4165:

the current patch applies the new strategy to the noDict variants of fast and dfast only. It seems interesting to also test and eventually adopt this strategy for dictionary compression too.

ZSTD_compressBlock_doubleFast_dictMatchState_generic() tests matchIndex >= prefixLowestIndex before it reads a prefix candidate. With an attached dictionary, the input is usually small and the prefix is short, so many candidate indexes come from before the prefix. The branch is then hard to predict.

This PR applies the method of #4144 / #4165 to the three prefix checks in this function: the long match at ip, the short match at ip, and the long match at ip+1.

• The candidate is read through ZSTD_selectAddr(), which returns a local dummy[] buffer when the index is invalid. This is the same dummy[] as in the noDict function.
• The index is tested only after the bytes compare equal. Equal bytes are rare, so both branches become well predicted.

The index test still decides the result, so the compressed output is byte-identical.

Unlike #4165, this does not need a separate branch-based variant for long inputs. With an attached dictionary, this function runs only while the dictionary is in the window, and long streamed inputs are faster as well (see below).

Results

silesia.tar compressed as independent blocks (zstd -b<level> -i2 -B<size> -D <dict>), with a 16 KB raw-content dictionary: the first 16,384 bytes of dickens (head -c 16384 dickens > raw.dict). Blocks of 16 KB or less attach the dictionary, so they use dictMatchState. Each cell is the median of 10 alternating dev/PR pairs (12 for the 16 KB rows) on one pinned core. "Change" is the median of the per-pair ratios.

Compiler	Level	Block	dev MB/s	PR MB/s	Change	Faster pairs
clang 14.0.0	3	1 KB	106.1	121.2	+14.36%	10/10
clang 14.0.0	3	4 KB	163.7	191.8	+17.34%	10/10
clang 14.0.0	3	16 KB	203.9	236.4	+15.94%	12/12
clang 14.0.0	4	4 KB	157.8	184.6	+16.79%	10/10
clang 14.0.0	4	16 KB	195.1	222.4	+13.95%	12/12
gcc 11.4.0	3	1 KB	114.1	118.0	+3.28%	10/10
gcc 11.4.0	3	4 KB	176.5	182.4	+3.52%	10/10
gcc 11.4.0	3	16 KB	218.8	226.8	+3.47%	12/12
gcc 11.4.0	4	4 KB	170.7	177.1	+4.01%	10/10
gcc 11.4.0	4	16 KB	208.3	214.0	+2.60%	12/12

Compressed sizes are identical to dev in every cell. A/A on the same core (-B16K -D, level 3): gcc +0.33%, clang +0.41%. Decompression is not changed, and all its rows are within −0.4% to +0.7%.

The results are similar with trained dictionaries (64 KB and 112 KB, trained on 4 KB slices of Silesia), level 3:

Compiler	Dictionary	4 KB blocks	16 KB blocks
clang 14.0.0	trained 64 KB	+16.09% (10/10)	+15.08% (10/10)
clang 14.0.0	trained 112 KB	+15.71% (10/10)	+14.18% (10/10)
gcc 11.4.0	trained 64 KB	+3.44% (10/10)	+3.22% (10/10)
gcc 11.4.0	trained 112 KB	+3.16% (10/10)	+2.63% (10/10)

Mechanism (perf stat, per pass over 12 × 1 MB Silesia files, level 3, 16 KB chunks with ZSTD_compress_usingCDict()):

Compiler	Branch misses	Instructions	Cycles
clang 14.0.0	3.30 M → 1.16 M (−65%)	+7.3%	−16.5%
gcc 11.4.0	3.36 M → 2.19 M (−35%)	+13.7%	−3.5%

Long inputs with an attached dictionary. #4165 found that the cmov method makes long noDict inputs slower. Here, the matching case is a stream with unknown size: the CDict stays attached, and the prefix grows up to the window size. I measured it with ZSTD_compressStream2(), ZSTD_CCtx_refCDict(), 128 KB ZSTD_e_continue steps, the same 16 KB dictionary, and the 1 MB prefixes of the 12 Silesia files. perf confirms that ZSTD_compressBlock_doubleFast_dictMatchState_* takes 79% of the time. 12 pairs:

Compiler	Level	dev MB/s	PR MB/s	Change	Faster pairs
clang 14.0.0	3	208.5	216.6	+3.42%	11/12
clang 14.0.0	4	206.5	215.2	+4.71%	12/12
gcc 11.4.0	3	216.4	221.2	+2.44%	10/12
gcc 11.4.0	4	210.8	217.2	+2.52%	11/12

A/A in this mode: gcc +0.17% (6/12), clang +0.34% (9/12).

Unchanged paths (this function is not used):

Case	gcc 11.4.0	clang 14.0.0
whole silesia.tar, no dictionary, level 3	+0.26% (7/12)	−0.74% (3/12)
whole silesia.tar, no dictionary, level 4	−0.62% (2/10)	−0.02% (5/10)
64 KB blocks + dictionary (copied, not attached), level 3	−0.06% (5/10)	−0.29% (3/10)
128 KB blocks + dictionary, level 3	−0.06% (4/10)	−0.10% (3/10)

These rows are within the A/A spread.

Cost

• Code size. size text:

  Build	gcc 11.4.0	clang 14.0.0
  zstd CLI	1,042,034 → 1,043,354 (+1,320, +0.13%)	967,602 → 970,546 (+2,944, +0.30%)
  zstd_double_fast.o, -O3	+1,288	+2,944
  zstd_double_fast.o, -O2	+704	+752

• Adversarial input. The code compares 8 or 4 bytes from dummy[] with the input when the index is invalid. An input made of the dummy[] byte pattern therefore causes many false-equal reads, and every false-equal read costs an index test. On such a crafted 300 KB input (-B4K, level 3), the PR is −1.6% (gcc) and −4.4% (clang). noDict has the same property.

Correctness

• Long checks: ZSTD_selectAddr(idx, low, match, dummy) returns match if and only if idx >= low. The two operands of && have no side effects, so the swapped order gives the same boolean. For a valid index, the read address is the same as before. For an invalid index, the result is false whatever dummy[] contains.
• Short check: the control flow is the same as before. ZSTD_selectAddr() uses >=, but the check uses >. At idx == prefixLowestIndex, the read goes to prefixLowest, which is valid memory, and idx > prefixLowestIndex is false. The dictionary branch is reached under the same condition as before (dictTagsMatchS && idx <= prefixLowestIndex).
• Memory: the code reads 8 bytes from the 10-byte local array through MEM_read64, and it never reads below the prefix.
• Byte identity, with cross-decoding:
  • About 150,000 library configurations per build had 0 differences. They cover 23 inputs × 6 dictionaries × 14 parameter sets (minMatch 4–7, wlog 10–17, hlog/clog), chunk sizes from 1 B to 128 KB, attach modes 0/1/2, CDict and loadDictionary, and compress2 and streaming with and without a pledged size. The builds were gcc, clang, -m32 and the forced C fallback of ZSTD_selectAddr().
  • The attach cutoff (16,000–32,769 B) is covered by 58,752 configurations.
  • 23,520 CLI cases had 0 differences.
  • A coverage build hit both sides of every changed condition, including idx == prefixLowestIndex and dummy bytes equal to the input.
• Tests: fuzzer -T180s, zstreamtest -T180s, an ASan+UBSan fuzzer -T120s, and playTests.sh (also with --test-large-data) all passed. ASan+UBSan and DEBUGLEVEL=1 library builds passed 66,168 configurations each.

Method

• Machine: AMD EPYC 9654 (Zen 4), Ubuntu 22.04, Linux 5.15. The machine is shared, so each benchmark ran pinned to one core.
• Builds: make zstd and make -C lib libzstd.a with default flags (-O3) and no -march.
• Input: silesia.tar, 211,957,760 bytes, SHA-256 0b5ec70c8048321525f820bc1d76429840bd22ee2c7a116175253a0fec2c724d.
• Dictionary: head -c 16384 dickens > raw.dict (SHA-256 626287fc08506a3f5c970220e11a187237abd7a24c86e907699a781b1e6ef88d). The trained dictionaries were made with zstd --train on 4 KB slices of Silesia, so they favor this data.
• CLI rows: zstd -b<level> -i2 -B<size> -D <dict> silesia.tar. Each pair runs dev and the PR back to back, and the order alternates between pairs.
• Streaming and perf rows: a small C program, statically linked with each libzstd.a. In stream mode it uses ZSTD_CCtx_refCDict() + ZSTD_compressStream2() with 128 KB ZSTD_e_continue steps, then ZSTD_e_end. In chunk mode it calls ZSTD_compress_usingCDict() on each 16 KB chunk. Each run does 1 second of complete passes. The program checks that dev and the PR produce the same total compressed size. I can add it to the PR if it helps.

Limits

• Platforms: measured only on x86-64 (Zen 4) with gcc 11.4 and clang 14.0. I have no aarch64, Apple M1 or newer-compiler results. On aarch64, ZSTD_selectAddr() is a plain ternary, so the effect depends on whether the compiler emits csel. Improve compression speed on small blocks #4165 saw a good result on M1 for noDict.
• Scope: only the dfast dictMatchState path (levels 3–4 with an attached dictionary) is changed. The extDict path and the fast dictMatchState path are not changed.

Checklist

• I searched open pull requests and issues: this problem is not already being addressed.
• Bug fix: it comes with a reproducer or a test that fails without the fix. (Not applicable: this is a performance change with identical output.)
• Performance change: numbers from repeated runs are included, with CPU, compiler, flags and data.
  A gain in one configuration is not a general gain: see the Performance section of CONTRIBUTING.md.
• I stated below whether AI or other automated tools were used.

Tools used

Claude Code (Claude Opus) found this optimization, wrote the patch, ran the correctness checks (byte-identity harnesses, fuzzer, zstreamtest, ASan/UBSan, playTests) and the benchmarks, and drafted this description. A separate Claude Code agent reviewed the patch independently and repeated the measurements. I reviewed the diff and the benchmark data before submitting.