[fix] Fix streaming dataset chunking logic

[dodatboii]

Problem

chunk_batch_fn slices payloads by micro_batch_size, but splits metadata evenly using BatchMeta.chunk(num_splits). These boundaries can differ when the batch size is not divisible by the micro-batch size.

For example, 10 samples with micro_batch_size=4 produce payload batches of [4, 4, 2], while metadata is split into [4, 3, 3]. This misaligns sample data and metadata.

Change

Slice BatchMeta with the same [start:end] boundaries used for the payload.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-14T08:26:35.058023Z	8f595fe	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.