Repository navigation
Fix MS-ZIP blocks that end with short codes or copy from 32 KiB back - #7
Merged
Merged
Conversation
InflateCodes waited for as many bits as its lookup table uses before it decoded a literal, length or distance code. When the input ended with fewer bits than that, the last codes were not decoded, even though they were complete. An MS-ZIP block is a deflate stream on its own, so its last match was lost and the block came out short (SabreTools#1). When no more input is there, the code is now looked up with the bits that are left and used if it is not longer than those. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SetDictionary kept only the last 32767 bytes of a 32 KiB dictionary, because the window needs one free byte between its read and write positions. MS-ZIP sets the previous 32 KiB block as the dictionary, and a match at the start of a block can copy from 32768 bytes back, which then read a byte that was not in the window. A dictionary that fills the window now starts with the read and write positions at 0, so the first byte written replaces the oldest byte of the dictionary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mnadareski
approved these changes
Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1.
Two problems in the inflater made MS-ZIP blocks decompress wrong. Each commit fixes one and adds a test that fails without it.
Short codes at the end of the input.
InflateCodeswaited for as many bits as its lookup table uses before it decoded a code. An MS-ZIP block is a deflate stream on its own, so when a block ended with fewer bits than that, its last match was not decoded and the block came out short. This is what #1 reports: block 43495 of AgainstRome1.cab decodes to 32638 bytes and block 7 of folder 177 of Data2.cab to 32743 bytes, the same numbers as in the issue. Since 4dc1378 the missing end is filled with zeros, which only gives the right data when the block ends in zeros. The short blocks of AgainstRome1.cab and Group2.cab do, so main extracts those two correctly. The one in Data2.cab does not, so_86261BAE7A6B4C45A5D691D3320D5993from the issue still comes out wrong on main. When no more input is there, the code is now looked up with the bits that are left and used if it is not longer than those.A full 32 KiB history.
SetDictionarykept only the last 32767 bytes of the previous block, so a match at the start of a block that copies from 32768 bytes back read a byte that was not in the window. None of the three disc cabinets below has such a match. Generated blocks with them decode the same in zlib, cabextract and 7-Zip, and wrong on main.Measured on Linux with the cabinets from the issue, taken from the dumps of GameStar 07-2008 (Germany) (DVD-XL), Test Drive Unlimited (Germany) (En,Fr,De,Es,It) (Rerelease) and GameStar 09-2008 (Germany) (DVD-XL):
_86261BAE7A6B4C45A5D691D3320D5993differs.The split block in the spanned Data.cab set, which threw the exception in the issue, is already joined correctly on SabreTools.Serialization main.
Prepared with AI assistance (Claude Opus 5.5) and reviewed before submission.