Repository navigation
fix: move to scalo 2.14.1 and turn PGO and BOLT on for GA - #106
Merged
Merged
Conversation
scalo 2.14.0 writes no vendor, licence, copyright or group prefix of its own, so the app names them. The OCI labels carry the HyperI vendor, the io.hyperi namespace, BUSL-1.1 and the copyright line, as before. Derived consumer groups keep their dfe- prefix through KafkaSource::with_group_prefix. The integration tests pinning dfe-transform-vector-syslog and dfe-transform-vector-syslog-enriched fail on 2.14.0 without it and pass with it. PGO and BOLT are back on: build.skip_optimize is gone. config/defaults.yaml gives generate-artefacts the ArgoCD repo URL and the dfe namespace. The hand-fixed chart Deployment takes the generator's checksum/secret annotation and its projected service-account volume with token automount off. The Dockerfile header and config schema are regenerated. ci.yml passes optimize-tier through on dispatch, as the other apps do. The osv-scanner reason names the real blocker and the live scalo-rs tracker. RESEARCH.md goes, with its two links: it described an internal architecture this repo no longer has. DESIGN.md stops promising a dlq config the app never had, and the README secret-marker row says how the marker gets onto a field. The 25 markdownlint warnings left in DESIGN.md, MIGRATION.md, README.md and SECURITY.md are cleared.
scalo moves to 2.14.1, which brings cel 0.14.5, the one cel line the suite now shares. Its generated Dockerfile, chart, config schema and generate-artefacts output are byte-identical to 2.14.0's. Cargo.lock takes the newest compatible release of every crate. The floors move to what the lock now holds for tokio 1.53.2, clap 4.6.7, thiserror 2.0.21 and tokio-test 0.4.6, and the thiserror 1.x copy leaves the graph. testcontainers stays on 0.27: testcontainers-modules 0.15.0, the newest, requires it. figment, dotenvy, serde_yaml_ng and prometheus-parse have had no release in over a year. scalo depends on the first three as well, and no maintained drop-in replaces prometheus-parse, so each stays. The Kafka test broker is already apache/kafka 4.3.1 on the JVM image, pinned by digest. No open Dependabot alert. Code scanning is not enabled on this repo.
1 task
|
Released in v1.0.45 -- https://github.com/hyperi-io/dfe-transform-vector/releases/tag/v1.0.45 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GA rebuild of dfe-transform-vector on scalo 2.14.1, with PGO and BOLT back on and a final dependency pass. Two commits: the migration (proved on 2.14.0), then the deps pass, which takes scalo 2.14.1 and cel 0.14.5. 2.14.1's generated Dockerfile, chart, schema and generate-artefacts output are byte-identical to 2.14.0's.
What moved
io.hyperilabel namespace, BUSL-1.1, the copyright line.dfe-throughKafkaSource::with_group_prefix("dfe-").integration::config::dfe_source_derives_topics_and_cganddfe_source_with_pipeline_name_uses_pipeline_in_cgpindfe-transform-vector-sysloganddfe-transform-vector-syslog-enriched. On 2.14.0 without the prefix they fail withtransform-vector-syslog/transform-vector-syslog-enriched. With it they pass.build.skip_optimizeis gone, so the release channel runs PGO and BOLT again (workload_cmd: bash scripts/pgo-workload.sh, 300 s).config/defaults.yamlgivesgenerate-artefactsthe ArgoCD repo URL and namespacedfe.ci.ymlpassesoptimize-tierthrough on dispatch, matching dfe-receiver.GA fixes
RESEARCH.mddeleted, with its links in README.md and docs/architecture.md. It described an internal architecture this repo no longer has.dlqconfig. There is none: a record no retry gets through is dropped and counted inpipeline_dead_letters_dropped_total. The DLQ feature itself stays open as DLQ via Vector-native reroute_dropped sink: the DESIGN.md dlq config was never built #46 (post-ga).passwordwas schema'd asSensitiveString. The row now says that is how a field getsx-scalo-secret.Regenerated
io.hyperi.*labels are byte-identical.x-scalo-secretbesidex-dfe-secret.checksum/secret,automountServiceAccountToken: false, the projected service-account volume. chart/values.yaml takes two generator comment updates. Every non-hand-fixed chart file already matched the generator.Dependencies
cargo tree -d: one cel. Every remaining duplicate (base64, hashbrown, rand, reqwest 0.12/0.13, syn, toml_edit, ...) is transitive through scalo or testcontainers.Left as is
config.source.group_id: dfe-transform-vector-default. Dropping it changes the stock group todfe-transform-vector, and renders the KEDA trigger with an emptyconsumerGroup. Stock instances would still share one group, under a new name.Proof (local, rustc 1.99.0, docker unreachable)
cargo nextest run: 346 passed, 9 skipped (all#[ignore], Vector binary or live Kafka). Three Docker tests return early locally and run in CI.cargo clippy --all-targets --all-features -- -D warnings: clean.hyperi-ci check --quick: "Detected language: rust", markdownlint "9 file(s) clean", osv-scanner passed, "quality complete"..hyperi-ci.yamlignore is what passes it.build.skip_optimize resolved=False,channel=release, allocator=jemalloc, lto=fat, pgo=on, bolt=on.generate-artefacts: argocd-application.yaml repoURLhttps://github.com/hyperi-io/dfe-transform-vector,path: chart, namespacedfe.