Repository navigation
compress: support high dictionary IDs on 32-bit builds - #4864
trestoncuzzort wants to merge 1 commit into
Conversation
ZSTD_loadZstdDictionary() and ZSTD_compress_insertDictionary() returned both dictionary IDs and errors through size_t. On 32-bit builds, valid private dictionary IDs near UINT32_MAX overlap the encoded error range and are rejected by their callers. Return a status and write the ID through an output parameter instead. Raw or absent dictionaries still produce ID zero, and failed loads still return their error before the caller stores the ID. Add round trips and frame-ID checks for low and high IDs through direct dictionaries, prepared CDicts, and CCtx-loaded dictionaries.
|
Hi @trestoncuzzort! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
What and why
Fixes #4580.
On 32-bit builds, dictionary IDs near
UINT32_MAXoverlap the encodedsize_terror range. The compression-side dictionary loaders returned either an ID or an error, so a valid dictionary could fail withError (generic)or makeZSTD_createCDict()returnNULL. These high IDs are permitted for private use.The internal loaders now return status and write the
U32ID separately. The public API stays the same, and the ID is preserved in the compressed frame.Validation
Local static analysis is not clean. Clang 23 reports the same 15 diagnostic/source signatures on base and fix; a Clang 18 comparison reports the same nine on both. Neither comparison passes. I am submitting this as a draft with that baseline limitation explicit.
Checklist
Tools used
OpenAI Codex (GPT-6) assisted with investigation, implementation and tests. dawnr checked a mathematical model of status encoding; that model does not prove the C implementation.