Skip to content

Close the TIFF on writeTIFF() error paths to prevent a later crash - #15

Open
billdenney wants to merge 1 commit into
s-u:masterfrom
billdenney:fix-writetiff-error-path-leak
Open

billdenney wants to merge 1 commit into
s-u:masterfrom
billdenney:fix-writetiff-error-path-leak

Conversation

@billdenney

Copy link
Copy Markdown

Fixes #13

A failed writeTIFF() leaves the session corrupted, so the next readTIFF()
that itself has to take an error path crashes R instead of raising its normal
condition.

Reproducible example

bad  <- tempfile(fileext = ".tif")
good <- tempfile(fileext = ".tif")
file.create(good)                    # 0-byte, unreadable .tif
try(tiff::writeTIFF(array(runif(80), dim = c(4L, 4L, 5L)), bad), silent = TRUE)
tiff::readTIFF(good)
#> *** caught segfault ***

Expected Error: pkg:tiff: Cannot read TIFF header. Observed, varying run to
run: caught segfault, free(): invalid size, or
Fatal error: glibc detected an invalid stdio handle — the signature of a
use-after-free rather than a null dereference.

Environment: tiff 0.1-12, R 4.6.1, x86_64-pc-linux-gnu, Ubuntu 24.04.4,
libtiff 4.5.1+git230720-4ubuntu2.5.

Cause

write_tiff() opens the TIFF at src/write.c:66 with a pointer to the
stack-allocated tiff_job_t declared at src/write.c:31:

 31:    tiff_job_t rj;                    /* stack-allocated */
 61:    f = fopen(fn, "w+b");
 66:    tiff = TIFF_Open("wm", &rj);      /* libtiff now holds &rj */
...
101:    if (planes < 1 || planes > 4)
102:        Rf_error("image must have either 1 ... or 4 (RGBA) planes");

TIFFClose() is reached only on the success path (lines 235 and 238). Every
Rf_error() sited after TIFF_Open() longjmps back to R leaving the TIFF*
open, the FILE* unclosed, and — the part that actually crashes —
common.c's static last_tiff pointing at a TIFF whose client data is a
stack frame that no longer exists.

The stale pointer is dereferenced because TIFF_Open() assigns last_tiff
only after TIFFClientOpen() returns:

/* common.c:169 */
return (last_tiff = TIFFClientOpen("pkg:tiff", mode, (thandle_t) rj, ...));

libtiff diagnoses an unreadable header inside TIFFClientOpen(), so
TIFFErrorHandler_() runs while last_tiff still refers to the dead write
job, and its first action is TIFFClose(last_tiff) (common.c:30).

This also explains why only a failing read crashes: a successful read never
invokes the error handler, so it merely overwrites last_tiff.

Fix

Route the five Rf_error() calls sited after TIFF_Open() through a helper
that closes the TIFF first. TIFFClose() runs TIFFCloseProc_(), which
fcloses the output file or frees the in-memory buffer and resets
last_tiff — so one call is the complete cleanup for both output targets.

read.c already uses this same TIFFClose()-then-Rf_error() pattern at
lines 380 and 385; this applies it to write.c. Error messages and all
success paths are unchanged. +23/−5 in src/write.c, no other file touched.

Moving the planes check above the fopen would fix only the example above:
planes is derived per-iteration from VECTOR_ELT(img_list, ...) inside the
write loop, so it cannot be hoisted for the image-list case, and the other
four sites crash identically (measured below).

Verification

No tests/ directory exists in this package, so no regression test is added
here rather than introducing a test framework; the reprex above is the
regression case. Each check below ran in its own Rscript process with stdin
closed, since a segfault cannot be caught by tryCatch().

Check Sequence Before After
A failed read alone clean error clean error
B two failed reads clean ×2 clean ×2
C unrelated R error → failed read clean clean
D failed write → valid read works works
E failed write → failed read hung/crashed clean error
W84 error at line 84 → failed read hung/crashed clean error
W88 error at line 88 → failed read hung/crashed clean error
W115 error at line 115 → failed read hung/crashed clean error
MEM in-memory write error → failed read hung/crashed clean error
RT normal write+read round trip OK OK

Repeating the reprex, since the symptom is nondeterministic:

  • Before: 0/10 clean — 7 × caught segfault (exit 139), 2 ×
    invalid stdio handle, 1 × free(): invalid size (exit 134), all core-dumped.
  • After: 20/20 clean, every run Error: pkg:tiff: Cannot read TIFF header.

R CMD build + R CMD check --no-manual: Status: OK. Compiles with no
new warnings.

Not changed here

Deliberately out of scope for a minimal crash fix, but the same class:

  • read.c:543 and read.c:546 are post-TIFF_Open() Rf_error() calls with
    no TIFFClose(), so a failed tiled-image read should prime the identical
    crash. (read.c:380/385 do close, so these two look like omissions.)
  • read.c:202 and write.c:67-71 leak the FILE* when TIFF_Open() itself
    fails. No dangling pointer there — tiff is NULL — so no crash.
  • More robustly, clearing last_tiff before calling TIFFClientOpen() in
    TIFF_Open() would make this class unreachable regardless of caller
    discipline, at the cost of leaking a genuinely-live previous TIFF instead of
    closing it. That seemed like your call rather than something to fold into a
    bug fix.

Fixes s-u#13

write_tiff() opens the TIFF at src/write.c:66 with a pointer to the
stack-allocated tiff_job_t declared at src/write.c:31, but TIFFClose() is
only reached on the success path. Every Rf_error() raised after TIFF_Open()
therefore longjmps back to R leaving the TIFF open, the output FILE*
unclosed, and common.c's static last_tiff pointing at a TIFF whose client
data is a stack frame that no longer exists.

That stale pointer is what crashes. TIFF_Open() assigns last_tiff only
after TIFFClientOpen() returns, so when a later readTIFF() hits an
unreadable header, libtiff raises the error from inside TIFFClientOpen()
while last_tiff still refers to the dead write job. TIFFErrorHandler_()
then calls TIFFClose(last_tiff) on it and dereferences the unwound frame.

This also explains why only a failing read crashes: a successful read never
invokes the error handler, so it merely overwrites last_tiff.

Reproducible example (crashed 10/10 runs before this change, with the
symptom alternating between "caught segfault", "free(): invalid size" and
"glibc detected an invalid stdio handle"; clean 20/20 after):

    bad  <- tempfile(fileext = ".tif")
    good <- tempfile(fileext = ".tif")
    file.create(good)
    try(tiff::writeTIFF(array(runif(80), dim = c(4L, 4L, 5L)), bad),
        silent = TRUE)
    tiff::readTIFF(good)

Route the five Rf_error() calls sited after TIFF_Open() through a helper
that closes the TIFF first. TIFFClose() runs TIFFCloseProc_(), which
fcloses the file or frees the in-memory buffer and resets last_tiff, so it
is the complete cleanup for both output targets. read.c already uses this
same TIFFClose()-then-Rf_error() pattern at lines 380 and 385; this applies
it to write.c.

Error messages and all success paths are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tiff: a failed writeTIFF() leaks its TIFF*/FILE*, crashing the next failing readTIFF()

1 participant