pack: allow exact-fit JSON buffers while reserving the terminator

[UgurTheG]

flb_msgpack_to_json() rejects a buffer containing exactly enough space for the JSON output plus its NUL terminator: converting MessagePack nil to null currently needs six bytes instead of five. The wrapper already reserves one byte, but try_to_write() rejects equality against the remaining payload capacity, effectively reserving another byte.

Allow literal writes to fill that payload capacity while keeping json_size - 1 for every writer. Reject zero-sized buffers before the subtraction and terminator write. Passing the full buffer capacity instead would allow a string writer to consume the terminator slot and cause the wrapper to write one byte past the buffer on failure.

Regression coverage sweeps capacities from zero through one byte beyond exact fit for 16 MessagePack inputs in both Unicode modes, including literals, escaped and long strings, binary/extension values, and nested containers. It checks return values, exact output, termination, and canaries on both sides. Restoring the old comparison fails the exact-fit assertions; substituting the full-capacity proposal fails the boundary canaries.

Related finding: #12313 (comment). This PR is based directly on upstream master and is independent of the iterative traversal fix in #12313.

---

Testing

• [N/A] Example configuration file for the change (core serialization API).
• Debug log output from testing the change (summaries below).
• Valgrind output shows no leaks or memory corruption (summary below).
• [N/A] Local packaging test (no packaging changes).
• [N/A] ok-package-test label (no packaging changes).

Built with internal/runtime tests enabled and the tail, dummy, lib, storage_backlog, Loki, and null plugins. Validation on Linux:

ctest --test-dir /tmp/fluent-bit-buffer-build -R '^flb-it-(pack|utils)$' --output-on-failure
100% tests passed, 0 tests failed out of 2

ctest --test-dir /tmp/fluent-bit-buffer-build -R '^flb-rt-out_loki$' --output-on-failure
100% tests passed, 0 tests failed out of 1

valgrind --leak-check=full --show-leak-kinds=all --errors-for-leak-kinds=all --error-exitcode=99 /tmp/fluent-bit-buffer-build/bin/flb-it-pack
SUCCESS: All unit tests have passed.
in use at exit: 0 bytes in 0 blocks
All heap blocks were freed -- no leaks are possible
ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

FLUENT_BIT_BINARY=/tmp/fluent-bit-buffer-build/bin/fluent-bit /tmp/fluent-bit-test-venv/bin/python -m pytest tests/integration/scenarios/out_loki -q
1 passed

VALGRIND=1 VALGRIND_STRICT=1 FLUENT_BIT_BINARY=/tmp/fluent-bit-buffer-build/bin/fluent-bit /tmp/fluent-bit-test-venv/bin/python -m pytest tests/integration/scenarios/out_loki -q
1 passed

Documentation

• [N/A] User documentation (no configuration change; API return documentation updated inline).

Backporting

• Backport to latest stable release (not included in this PR).

---

Fluent Bit is licensed under Apache 2.0; this contribution is submitted under that license.

Summary by CodeRabbit

• Bug Fixes

  • Improved JSON conversion buffer handling for more reliable output.
  • Buffers that exactly match the required capacity are now accepted.
  • Zero-sized output buffers are correctly rejected.
  • Output reserves space for a null terminator, helping prevent truncated or improperly terminated results.

• Tests

  • Added comprehensive coverage for buffer boundaries, escaping, Unicode, binary data, and nested structures.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 73f5cd10-b7f7-493d-87d7-ed99cf7a5015

📥 Commits

Reviewing files that changed from the base of the PR and between a48daa6 and 3faf785.

📒 Files selected for processing (2)

• src/flb_pack.c
• tests/internal/pack.c

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The JSON serializer now rejects zero-sized buffers, permits exact fits, reserves space for a null terminator, and has tests for capacity boundaries, output integrity, and sentinel preservation.

Changes

JSON buffer capacity handling

Layer / File(s)	Summary
Serializer capacity validation and tests src/flb_pack.c, tests/internal/pack.c	try_to_write permits exact boundary writes. flb_msgpack_to_json rejects zero-sized buffers and reserves one byte for the terminator. Tests cover multiple MessagePack values, escape modes, capacities, return values, output contents, and buffer boundaries.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: cosmo0920

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: support for exact-fit JSON buffers while reserving space for the terminator.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}