Repository navigation
fix: move to scalo 2.14.1 and turn PGO and BOLT on for GA - #220
Merged
Merged
Conversation
scalo 2.14.0 ships no vendor, licence, copyright or registry defaults, so the contract now names HYPERI PTY LIMITED, the io.hyperi label namespace, BUSL-1.1, the copyright line and ghcr.io/hyperi-io itself. The DLQ common topic is now an Option. A config file with no dlq: key now takes the fleet DLQ standard: common routing to dfe_fetcher_dlq, spooled under /var/spool/dfe/dlq. It took scalo's per-table defaults before, because a field-level serde default skips the app's own Default. config/defaults.yaml names the chart repo and the dfe namespace, so generate-artefacts writes the ArgoCD Application. build.skip_optimize is gone, so a release build runs PGO and BOLT on scripts/pgo-workload.sh. CloudWatch metric rows carry the timestamp as integer epoch milliseconds. They carried CloudWatch's float epoch seconds, which a DateTime64(3) column reads as milliseconds, so every row landed in January 1970. A cursor read that fails now refuses the tick under either on_missing_cursor setting, so the next tick reads again from the stored cursor. It used to fetch the default lookback window and write the cursor past everything that sat behind it. The capability catalogue and README describe vault specs the way scalo parses them: the first segment is the mount, and a data segment after it is optional. The Dockerfile, chart and config schema are regenerated: the new header comment, a checksum/secret pod annotation, the projected service-account volume and the x-scalo-secret marker. The schema check in the tests reads x-scalo-secret. Closes #138 Refs #218 Refs #183
catinspace-au
marked this pull request as draft
October 6, 2026 00:55
Every crate moves to its newest in-major release, and the manifest floors follow the lock: tokio 1.53.2, uuid 1.27.0, thiserror 2.0.21, smallvec 1.16.2, reqsign 0.20.7, async-compression 0.4.50, odbc-api 29.2.0 and tokio-test 0.4.6. arrow-odbc moves to 26.0.0. Its one breaking change quotes column names in generated insert statements, and the fetcher only reads through OdbcReaderBuilder. arrow stays on 59, because arrow-odbc 26.0.0 still depends on arrow 59 and arrow 60 would put two arrow majors in the graph. cel and opentelemetry-proto stay on the lines scalo pins, and testcontainers on the one testcontainers-modules 0.15 accepts. The workspace csv entry goes: no member inherits it, only csv-core. GHSA-w9wp-h8wv-79jx stays an accepted, unreachable risk on opentelemetry_sdk 0.31.0. The osv-scanner reason and the deny.toml note now name hyperi-io/scalo-rs#281 and metrics-exporter-opentelemetry, the crate that holds scalo on the 0.31 line. The Kafka test broker moves to apache/kafka 4.3.1, the version the current stack deploys, and it and ClickHouse 26.3.32.14 are pinned by digest. The PGO workload broker moves to Redpanda v26.2.2, pinned by digest. A manual dispatch no longer cuts a release by default: from-head is an empty string unless set, and the inputs match the other apps, optimize-tier included.
This was referenced Oct 6, 2026
scalo 2.14.1 evaluates CEL on cel 0.14, so the workspace cel range moves to >=0.14.5, <0.15 and one cel resolves. The filter and template code builds unchanged, and no shipped profile or example calls contains() on a list or map. The ClickHouse test fixture moves to 26.3.42.3 and the PGO workload broker to Redpanda v26.2.3, both pinned by digest.
catinspace-au
marked this pull request as ready for review
October 6, 2026 04:08
catinspace-au
changed the title
|
Released in v1.4.32 -- https://github.com/hyperi-io/dfe-fetcher/releases/tag/v1.4.32 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GA rebuild of dfe-fetcher on scalo 2.14.1, with PGO and BOLT back on and two GA fixes.
2.14.1in the workspace. 2.14.1 evaluates CEL on cel 0.14, so the workspace cel range is>=0.14.5, <0.15and one cel resolves. The filter and template code builds unchanged and all 77 filter, template and predicate tests pass. No shipped profile or example callscontains()on a list or map.HYPERI PTY LIMITED,io.hyperi,BUSL-1.1, the copyright line andghcr.io/hyperi-ioitself.common_topicis anOption. A config file with nodlq:key now gets the fleet standard (common routing,dfe_fetcher_dlq,/var/spool/dfe/dlq). It got scalo's per-table defaults before, and the old schema default shows it.config/defaults.yamlnames the chart repo and thedfenamespace, sogenerate-artefactswrites the ArgoCD Application again.build.skip_optimizeis gone: a release build runs PGO and BOLT onscripts/pgo-workload.sh.timestampas integer epoch milliseconds, not float epoch seconds. Pinned by a test on a real-shapedGetMetricDataanswer with fractional seconds. The loader end-to-end check the issue asks for is not done here, so this refs it.on_missing_cursorsetting, so the next tick reads again from the stored cursor rather than fetching the lookback and writing the cursor past the gap. Pinned by a scheduler test with a store that fails one read.credential_secrettext and the README describe vault specs the way scalo parses them.crates/core/src/secret.rs:11still carries the old wording.Regenerated, expected diffs:
Dockerfile: the new regenerate header line.chart/templates/deployment.yaml: thechecksum/secretpod annotation.docs/config-schema.*:x-scalo-secretbesidex-dfe-secret, the fixed DLQ defaults, scalo 2.14.0's own doc text.docs/capability-catalog.*: thecredential_secrettext.Second commit, the GA deps pass:
from-headis an empty string, inputs match the other apps.csventry removed.Local proof:
default: 1331 passed, 73 skipped.jemalloc: 1331 passed, 73 skipped.db-clickhouse,db-mongodb,file-tail: 1362 passed, 73 skipped. Docker was unreachable, so the testcontainers tests returned early; CI runs them, including the new Kafka and ClickHouse pins.cargo clippy --all-targets -- -D warningson rustc 1.99.0 is clean for all three feature sets, andcargo check -p dfe-fetcher-db --features odbc --all-targetscompiles arrow-odbc 26.checked_in_chart_matches_generated,checked_in_dockerfile_matches_generated,test_config_artifacts_do_not_drift.generate-artefactswithconfig/defaults.yamlwritesargocd-application.yamlwithrepoURL: https://github.com/hyperi-io/dfe-fetcherandnamespace: dfe.skip_optimize resolved=False,pgo=on, bolt=on.hyperi-ci check --quick:Detected language: rust,quality complete, markdownlint 30 files clean, clippy src and tests on all three feature sets passed.