Skip to content

fix(archiver): give generated filenames the charset the post-id gate accepts - #35

Closed
sayed1145 wants to merge 1 commit into
du82:mainfrom
sayed1145:fix/archiver-filename-reader-charset
Closed

sayed1145 wants to merge 1 commit into
du82:mainfrom
sayed1145:fix/archiver-filename-reader-charset

Conversation

@sayed1145

Copy link
Copy Markdown

Closes #34

What

generate_filename() derived a name from the Telegraph API's page.path and replaced only a
shell-safety set (/ \ : ? * " < > |). The matching read goes through is_valid_post_id(),
which accepts only ASCII alphanumerics, - and _. Any other character therefore produced a file
that was written successfully and then never served:

remote path filename written readable?
a/../../etc/passwd a-..-..-etc-passwd.md no (. rejected)
..\..\win --..-win.md no
.env .env.md no (hidden)
"", ".", ".." .md, ..md no (hidden, empty id)
400-character path 400-char stem no (over MAX_POST_ID_LEN)

is_valid_post_id()'s doc comment asserted the invariant that was violated -- that "the Telegraph
archiver yields [A-Za-z0-9_-] slugs". It did not.

How

Apply the reader's character set on the write side: map everything outside [A-Za-z0-9_-] to -,
collapse runs, trim the ends, fall back to untitled for a degenerate path, and bound the stem to
250 bytes so MAX_POST_ID_LEN (255) still holds once .md is appended. Written and servable are
now the same set by construction, which is the property the new tests assert.

Also corrected the is_valid_post_id() doc comment and noted why the write side has to satisfy the
same gate.

Deliberate behaviour change: a path with non-ASCII segments loses them --
unicode-日本語-ページ becomes unicode.md. That is the price of a name the reader accepts, and it
replaces a name the reader could never accept at all; the test comment says so explicitly.

Verification

cargo test on the patched tree:

test result: ok. 149 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.42s

archiver module only:

test archiver::tests::test_generate_filename ... ok
test archiver::tests::test_generated_filenames_are_servable ... ok
test archiver::tests::test_degenerate_paths_fall_back_to_untitled ... ok
test archiver::tests::test_separators_and_dots_do_not_survive_into_the_filename ... ok
test archiver::tests::test_long_path_is_bounded_to_a_readable_id ... ok
test archiver::tests::test_distinct_ordinary_paths_do_not_collide ... ok
test archiver::tests::test_extract_path_from_url ... ok
test result: ok. 7 passed; 0 failed

The same suite against the unpatched implementation (tests kept, implementation reverted):

---- test_separators_and_dots_do_not_survive_into_the_filename ----
panicked: path "a/../../etc/passwd"
  left:  "a-..-..-etc-passwd.md"
  right: "a-etc-passwd.md"

---- test_long_path_is_bounded_to_a_readable_id ----
panicked: "aaaa...aaaa.md" is not readable

test result: FAILED. 3 passed; 4 failed

The tests assert the contract, not a character list:

let stem = filename.strip_suffix(".md").expect("filename must end in .md");
assert!(crate::is_valid_post_id(stem),
        "path {path:?} produced {filename:?}, whose id {stem:?} is not readable");
assert!(!filename.starts_with('.'),
        "path {path:?} produced the hidden filename {filename:?}");

is_valid_post_id becomes pub(crate) so the archiver's tests can assert against the real gate
rather than a copy of it. I also verified the mapping by exhaustive and randomised enumeration
outside the suite (all 1,464 combinations up to length 3 of ./\: -_a0?*, then 20,000 random
printable-ASCII strings): no output was ever rejected by the gate or started with ., and the stem
stayed within the bound for inputs up to 5,000 characters.

Related

Adjacent to but not the same as #27 / #28: that was the read route interpolating an unvalidated id
into a path. This is the write route producing a name the read gate rejects. The existing
separator replacement already prevents traversal, and the new mapping keeps that true.

I did not add a Monero address to any comment; if a bounty applies, any address convenient to you is
fine, and no payment is expected if this does not meet the bar.

…accepts

generate_filename() derived a name from the Telegraph API's page.path and replaced
only a shell-safety set (/ \ : ? * " < > |). The read side goes through
is_valid_post_id(), which accepts only [A-Za-z0-9_-], so a path containing any
other character produced a file that was written successfully and then never
served:

  "a/../../etc/passwd" -> "a-..-..-etc-passwd.md"   (unreadable)
  "..\\..\\win"          -> "--..-win.md"            (unreadable)
  ".env"               -> ".env.md"                (hidden, unreadable)
  "" / "." / ".."      -> ".md" / "..md"           (hidden, empty id)
  400-char path        -> 400-char stem          (over MAX_POST_ID_LEN)

is_valid_post_id()'s doc comment asserted that "the Telegraph archiver yields
[A-Za-z0-9_-] slugs"; it did not.

Apply the reader's charset on the write side: map anything outside [A-Za-z0-9_-]
to '-', collapse runs, trim the ends, fall back to a fixed name for a degenerate
path, and bound the stem to leave room for the extension. Written and servable
are now the same set by construction.

The cost is that a non-ASCII path segment is dropped ("unicode-日本語-ページ"
becomes "unicode.md"); it replaces a name the reader could never accept at all.

Verified: cargo test is 149 passed / 0 failed. Against the unpatched source the
new suite fails 4 of 7, including the contract assertion that every produced
filename satisfies is_valid_post_id.

Closes #34
@sayed1145

Copy link
Copy Markdown
Author

One note on CI: the Rust Tests workflow is action_required, which I read as GitHub
waiting for a maintainer to approve the run for a first-time contributor, rather than a failure.

I ran the suite locally on this branch before opening the PR, and again after reverting only the
implementation to confirm the new tests actually catch the defect:

$ cargo test
test result: ok. 149 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.42s

$ cargo test archiver::
test result: ok. 7 passed; 0 failed; 142 filtered out

# tests kept, implementation reverted to main:
test result: FAILED. 3 passed; 4 failed
  path "a/../../etc/passwd" -> left "a-..-..-etc-passwd.md", right "a-etc-passwd.md"
  "aaaa...aaaa.md" is not readable

cargo build and cargo test both produced no errors; the only warning is the pre-existing
fields url, author_url, and views are never read.

Happy to adjust if the fallback name or the length bound should be different.

@du82 du82 closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telegraph archiver writes filenames its own post-id gate rejects, so the archived page is never served

2 participants