Skip to content

feat: add flatPaths optimization for JSON parsing - #76

Merged
kazeburo merged 2 commits into
mainfrom
feat/flatjson-optimization
Oct 5, 2026
Merged

kazeburo merged 2 commits into
mainfrom
feat/flatjson-optimization

Conversation

@kazeburo

@kazeburo kazeburo commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

PR Type

Enhancement, Bug fix, Tests


Description

  • Optimize parsing of small root-level JSON path sets.

  • Preserve generic parsing for unsupported or malformed input.

  • Propagate percentile append errors.

  • Add parity tests and comparative benchmarks.


Diagram Walkthrough

flowchart LR
  A["Configured JSON paths"]
  B["Flat root-level parser"]
  C["Generic EachKey parser"]
  D["Aggregators"]
  A -- "1–64 root-level paths" --> B
  A -- "other paths" --> C
  B -- "matched values" --> D
  B -- "parse error fallback" --> C
  C -- "remaining values" --> D
Loading

File Walkthrough

Relevant files
Enhancement
main.go
Track eligibility for flat JSON path parsing                         

cmd/mackerel-plugin-jsonl/main.go

  • Add flatPaths to parser options.
+1/-0     
parser.go
Parse flat JSON paths with early termination                         

cmd/mackerel-plugin-jsonl/parser.go

  • Parse eligible root-level keys with ObjectEach, stopping once all
    requested paths are found.
  • Fall back to EachKey on parsing errors without delivering already
    matched values twice.
+39/-1   
reader.go
Select flat paths and propagate append errors                       

cmd/mackerel-plugin-jsonl/reader.go

  • Enable flat parsing only for one to 64 single-segment paths.
  • Return errors from percentile append operations.
+9/-1     
Tests
main_test.go
Benchmark flat parsing against generic key parsing             

cmd/mackerel-plugin-jsonl/main_test.go

  • Add a benchmark using the generic EachKey parser for comparison.
  • Use require assertions in benchmark setup and validation.
+17/-21 
parser_test.go
Test flat parser parity and path eligibility                         

cmd/mackerel-plugin-jsonl/parser_test.go

  • Compare flat parsing with EachKey across escaped, duplicate, missing,
    compound, and array cases.
  • Verify flat parsing is selected only for one to 64 root-level paths.
+104/-0 

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit f28aee7)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Benchmark Comparison

Base: main
PR: feat/flatjson-optimization

BenchmarkMainParse results
goos: linux
goarch: amd64
pkg: github.com/monitoring-forge/mackerel-plugin-jsonl/cmd/mackerel-plugin-jsonl
cpu: AMD EPYC 7763 64-Core Processor                
                             │ base-bench.txt │            pr-bench.txt             │
                             │     sec/op     │   sec/op     vs base                │
MainParse_jsonl-4                42.97m ± 10%   34.14m ± 1%  -20.53% (p=0.000 n=10)
MainParse_parse_and_output-4     44.67m ±  2%   35.93m ± 3%  -19.57% (p=0.000 n=10)
MainParse_jsonl_eachkey-4                       42.47m ± 1%
geomean                          43.81m         37.35m       -20.06%

                             │ base-bench.txt │            pr-bench.txt             │
                             │      B/op      │     B/op      vs base               │
MainParse_jsonl-4                4.642Mi ± 0%   4.642Mi ± 0%       ~ (p=0.315 n=10)
MainParse_parse_and_output-4     5.412Mi ± 0%   5.412Mi ± 0%  +0.00% (p=0.000 n=10)
MainParse_jsonl_eachkey-4                       4.642Mi ± 0%
geomean                          5.012Mi        4.885Mi       +0.00%

                             │ base-bench.txt │            pr-bench.txt            │
                             │   allocs/op    │  allocs/op   vs base               │
MainParse_jsonl-4                 100.4k ± 0%   100.4k ± 0%       ~ (p=0.345 n=10)
MainParse_parse_and_output-4      100.5k ± 0%   100.5k ± 0%  +0.00% (p=0.000 n=10)
MainParse_jsonl_eachkey-4                       100.4k ± 0%
geomean                           100.5k        100.4k       -0.00%

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

No code suggestions found for the PR.

@sonarqubecloud

sonarqubecloud Bot commented Oct 5, 2026

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit f28aee7

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

No code suggestions found for the PR.

@kazeburo
kazeburo merged commit 163ea07 into main Oct 5, 2026
4 checks passed
@kazeburo
kazeburo deleted the feat/flatjson-optimization branch October 5, 2026 13:31
@github-actions github-actions Bot mentioned this pull request Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant