Skip to content

docs: recommend checksums with dictionaries #4846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
FOWEPJF255 wants to merge 1 commit into
base: dev
Choose a base branch
from
+19 −0
Open

docs: recommend checksums with dictionaries #4846

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions doc/zstd_compression_format.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,14 @@ in which case the frame may or may not need a dictionary to be decoded,
and the ID of such a dictionary is not specified.
The decoder must know this information by other means.

`Dictionary_ID` identifies a dictionary but does not verify its contents.
Different dictionaries can share the same ID, and decoding with the wrong
dictionary can produce incorrect output. Producers SHOULD include the optional
`Content_Checksum` in dictionary-compressed frames to help decoders that verify
it detect incorrect output. Checksums are optional and decoders may ignore them,
so this is only a partial integrity check; it does not authenticate a dictionary
or replace an authenticated dictionary-distribution mechanism.

#### `Frame_Content_Size`

This is the original (uncompressed) size. This information is optional.
Expand Down
6 changes: 6 additions & 0 deletions programs/zstd.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,12 @@ Note: If \fBwindowLog\fR is set to larger than 27, \fB\-\-long=windowLog\fR or \
.IP "\(bu" 4
\fB\-D DICT\fR: use \fBDICT\fR as Dictionary to compress or decompress FILE(s)
.
.IP
When compressing with a dictionary, keep the content checksum enabled
(\fB\-\-check\fR, the default) to help decoders detect a mismatched dictionary.
This only works when the decoder verifies the checksum and is not
cryptographic.
.
.IP "\(bu" 4
\fB\-\-patch\-from FILE\fR: Specify the file to be used as a reference point for zstd\'s diff engine\. This is effectively dictionary compression with some convenient parameter selection, namely that \fIwindowSize\fR > \fIsrcSize\fR\.
.
Expand Down
5 changes: 5 additions & 0 deletions programs/zstd.1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,11 @@ the last one takes effect.
`--memory=windowSize` needs to be passed to the decompressor.
* `-D DICT`:
use `DICT` as Dictionary to compress or decompress FILE(s)

When compressing with a dictionary, keep the content checksum enabled
(`--check`, the default) to help decoders detect a mismatched dictionary.
This only works when the decoder verifies the checksum and is not
cryptographic.
* `--patch-from FILE`:
Specify the file to be used as a reference point for zstd's diff engine.
This is effectively dictionary compression with some convenient parameter
Expand Down