Skip to content

feat(scripts): add csvdt, CSV timestamp normalisation - #240

Open
YardQuit wants to merge 1 commit into
teamdfir:masterfrom
YardQuit:add-csvdt
Open

YardQuit wants to merge 1 commit into
teamdfir:masterfrom
YardQuit:add-csvdt

Conversation

@YardQuit

@YardQuit YardQuit commented Sep 6, 2026

Copy link
Copy Markdown

In support of teamdfir/sift#684, opened at Rob Lee's request that csvdt be added to the SIFT Workstation project.

Adds csvdt, a command-line CSV parser whose subject is timestamps: it reads epoch, ISO 8601 and Windows FILETIME in a column and writes any of them back, localised to a named zone. The use it is meant for on SIFT is bringing a timeline assembled from several parsers onto one clock before the rows are correlated.

Disclosure: I am the author of csvdt. Happy to change any of the below, and equally happy to close this if it turns out not to be something SIFT wants to carry.

  • Installed as the upstream musl-static binary, from a pinned immutable release tag with an explicit SHA256, to /usr/local/bin/csvdt. The rolling release tag is deliberately not used: its artefacts are rebuilt monthly against a newer IANA tzdata, so its checksum moves and a pin to it would fail source_hash on every install after the next rebuild.
  • Guarded to amd64 — upstream publishes no Linux arm64 binary yet. I would rather add aarch64-unknown-linux-musl upstream and drop the guard than have SIFT carry an arch gap on my account; say the word and I will do that before this is merged.
  • Renovate annotation included: one GitHub-hosted asset with a single digest, which is the case the custom manager in .github/renovate.json handles. Checked against that regex as renovate applies it (dotAll, whole file) — it captures currentValue and currentDigest together, so a bump arrives as a matching pair rather than a new version against a stale hash.
  • The manual page is rendered by the binary itself at install time (--generate-man), from the same definitions that parse its options, so the page on the image cannot describe an option the installed build lacks. mandb is tolerated as absent.
  • Registered in sift/scripts/init.sls in both the include: list and the test.nop require: list.

Applied in the saltstack-tester image on 22.04/3007 and 24.04/3006: three states, none failed, and a second run reports all three Clean. The installed binary reports csvdt 1.0.0+2026.08.8609888 (IANA tzdata 2025b) and writes a 68 KB man page. man -w does not resolve it inside the tester image, but only because that image is a minimized Ubuntu with man-db removed — which is the case the (mandb -q || true) tolerance is there for.

License: GPL-3.0-or-later. Source: https://github.com/YardQuit/csvdt

A CSV parser for the command line whose subject is timestamps: it reads
epoch, ISO 8601 and Windows FILETIME in a column and writes any of them
back, localised to a named zone, so a timeline assembled from several
tools can be brought onto one clock before analysis.

Installed from the upstream release binary, statically linked against
musl. Pinned to the version's own tag rather than the rolling 'release'
tag, whose artefacts are rebuilt monthly against a newer IANA database
and whose checksum therefore moves; a pin to it would fail source_hash
on every install after the next rebuild.

Linux binaries are published for x86-64 only, so the state carries the
amd64 grain guard rather than installing a binary that cannot run.

The manual page is generated by the binary itself at install time, from
the same definitions that parse its options, so the page on the image
cannot describe an option this build lacks.

Renovate can keep the pin current: one GitHub-hosted asset with a single
digest is the case the custom manager handles, capturing the version and
the hash together so a bump arrives as a matching pair.
@YardQuit YardQuit changed the title Add csvdt feat(scripts): add csvdt, CSV timestamp normalisation Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant