diff --git a/crates/perturbation-sim/BANGLADESH_EGGLOG_ORACLE.md b/crates/perturbation-sim/BANGLADESH_EGGLOG_ORACLE.md new file mode 100644 index 000000000..152e4e113 --- /dev/null +++ b/crates/perturbation-sim/BANGLADESH_EGGLOG_ORACLE.md @@ -0,0 +1,69 @@ +# Bangladesh outage: dual-mechanism reference oracle and cold mathematical compiler + +**Status:** bounded offline scalar oracle plus compile-contract proposal. No egglog runtime integration, R2IL executable, measured accuracy, or Bangladesh operational claim is implied by this note. It is deliberately not a second physical engine. + +## Source and provenance + +- [Bangladesh-outage](https://github.com/AdaWorldAPI/Bangladesh-outage) is a historical telecom tower power-outage predictor, with synthetic A10H outage-generation / telemetry scripts, data preparation, FNN/LSTM/TCN training, and a Word project plan. The executable Python generators and pipeline are **reference behaviour for what the legacy system actually did**, not an independent causal ground-truth dataset. The Word project plan was not machine-inspected in this change. +- [`OutageOracle/MVP1/A10H/010_Generate_Raw_Outage_Logs.py`](https://github.com/AdaWorldAPI/Bangladesh-outage/blob/main/OutageOracle/MVP1/A10H/010_Generate_Raw_Outage_Logs.py): synthetic events `grid_failure | voltage_drop | maintenance`; tower `power_source` metadata and grid groups; shared weather and synthetic grid cascades. No independent telecom-path failure mechanism is generated. +- [`020_Generate_Raw_Telemetry.py`](https://github.com/AdaWorldAPI/Bangladesh-outage/blob/main/OutageOracle/MVP1/A10H/020_Generate_Raw_Telemetry.py): sets `grid_status = 'off' if is_off else 'on'` where `is_off` means a tower was inside a *synthetic outage interval*. **It is not an independent measurement of the electricity grid.** Likewise 0 V/0 A are generated from that same label. Never train/validate a causal classifier on these as independent evidence of the root cause. +- [`040_Merge_And_Aggregate.py`](https://github.com/AdaWorldAPI/Bangladesh-outage/blob/main/OutageOracle/MVP1/A10H/040_Merge_And_Aggregate.py): aggregates tower-level rows into region/day outage counts and durations, losing individual site/incident/root-cause identity. +- [`050_ExtendFeatures.py`](https://github.com/AdaWorldAPI/Bangladesh-outage/blob/main/OutageOracle/MVP1/A10H/050_ExtendFeatures.py): creates `target_outage_15d` before training; [`070_Train.py`](https://github.com/AdaWorldAPI/Bangladesh-outage/blob/main/OutageOracle/MVP1/A10H/070_Train.py) selects all numeric features except four basic columns, **not** `target_outage_15d`. That permits target leakage into its historical sequences. [`060_PreProcessScaling.py`](https://github.com/AdaWorldAPI/Bangladesh-outage/blob/main/OutageOracle/MVP1/A10H/060_PreProcessScaling.py) fits the scaler before the chronological train/validation split; region-level historical averages also use the full series. `070_Train.py` has sigmoid occurrence heads passed to `BCEWithLogitsLoss`, and fits its logistic calibrator on the same validation set later scored. These all need an independently verified walk-forward baseline before reported prediction accuracy has evidentiary force. +- [PR #1689](https://github.com/AdaWorldAPI/lance-graph/pull/1689): egglog cold functional database/semi-naive/equality saturation, eggcc effect-safe extraction, 3LA source-to-target validation. The PR is research-only, not an integrated compiler. +- [`perturbation-sim`](README.md): `simulate_outage` computes DC power-flow line-trip cascades and spectral perturbation **on a bus/line electrical network**. It does not model electrical distribution to a tower, telecom backhaul failures, battery runtime, or measured tower availability. Its `node_field`, `tripped` and `islanded` outputs are **grid risk witnesses**, not telecom labels. + +## Two axes, not one Boolean + +1. **Grid failure:** physical grid supply is absent at the tower connection. Independent recorded grid supply is distinct from generator/battery supply. A grid line trip or high Laplacian perturbation alone does not establish the site's grid outage. +2. **Telecom-path failure:** radio hardware, tower subsystem, upstream backhaul or other explicitly observed non-power path is unavailable. Even perfect grid power need not restore service. +3. **Coupling:** under the expressly simplified Boolean availability contract, `site_power = grid_supply OR backup_supply`, `tower_service = site_power AND telecom_path`. An upstream backhaul supply dependency, delayed battery depletion, maintenance, multi-feed switching and uncertain observations require more state or refusal, **not** a Boolean reinterpretation of the historical `grid_status` column. + +A grid incident may leave the tower online (backup works). An independent telecom-path failure may leave the electricity grid fully healthy. Both can occur at once. Neither can be established from a tower-down observation alone. + +The executable independent **scalar reference** is [`src/outage_oracle.rs`](src/outage_oracle.rs): `Observed::{Up,Down,Unknown}` with strong-Kleene `and/or/not`, `assess_site` with separate `grid_supply_outage`, `power_unavailable`, `telecom_path_failure`, predicted service, non-exclusive mechanism and contradiction/refusal. `assess_bound` checks globally stable tower GUID, common window and independently pinned source generations. It deliberately has no simulation dependency so compiler and reference cannot share an implementation bug. + +## Compiler split (proposal, NOT shipped) + +```text +historical Bangladesh data (synthetic labels stay marked synthetic) + + separately measured grid/tower/backup observations + -> time align, resolve tower↔electrical bus mapping and version-pin + -> physical perturbation-sim grid risk / line-trip evidence [separate] + -> typed site-availability equation + witness provenance + -> egglog: COLD pure guarded algebra / shared-DAG search + -> eggcc-style extractor refuses cross-version or effect reordering + -> typed R2IL FoldDialect procedure [when actually admitted] + -> Quack -> mask-risc execute_into [existing hot executor] + -> **independent** outage_oracle::assess_bound differential + -> source-age, authorization and actual write ack [separate effects] +``` + +**Possible legal pure rewrites:** `x AND x → x`; `x OR x → x`; De Morgan; guarded distributivity; common-subexpression reuse for the *same* focus, generation, truth domain and source read. Scalar `outage_oracle` tests enumerate all 27 K3 triples. A rewrite certificate only proves an expression identity in its declared truth domain, not a root-cause claim. + +**Forbidden simplifications:** `grid_down == tower_down`; `tower_down ⇒ grid_failure`; `line_tripped ⇒ site_unpowered`; `Unknown == Down`; `grid_synthetic_status == independently_observed_supply`; `bag_COUNT == boolean_EXISTS`; `cross-row ordinal match == tower identity`; `eq_saturation == source freshness/committed write`. No new hot executor or positional cross-table gathers. + +For each admitted extraction pin `source_revision`, `tower_guid`, `time_window`, source-generation tuple, input truth domain, output shape, multiplicity, effect-order witness and reference-case ID. `BIND`, external `READ`, `EMIT`, and `COMMIT` are effect-bearing. Numeric/spectral outputs must keep units and floating-point tolerance; K3 Boolean truth-tables do not justify numeric reassociation. Cost measurements must include compilation, binding, reads, execution, replay/validation and dirty/commit, amortized over repeated frames. + +## Minimum acceptance battery + +| ID | Input | Mandatory invariant | +|---|---|---| +| BD1 | grid down; backup up; telecom up | grid outage is observed, tower remains up; **not a telecom outage** | +| BD2 | grid down; backup down; telecom healthy | `GridPower` mechanism, tower down | +| BD3 | grid up; telecom down | `TelecomPath` mechanism; **not grid outage** | +| BD4 | grid and telecom down | `Both`; do not merge causes or erase either lineage | +| BD5 | grid down; backup unknown; telecom healthy | tower status and mechanism `Unresolved`, never default to DOWN | +| BD6 | healthy inputs; separately observed tower down | independent oracle returns `Contradiction`, not a fitted explanation | +| BD7 | same source-local ordinal, different tower GUID; reused stale generation | bind refuses `CrossFocus` or `StaleSource`, even if expression is algebraically legal | +| BD8 | grid `simulate_outage` changes spectral/line-trip fields only | cannot assert tower service, and may not change a direct observed telecom cause | +| BD9 | legacy A10H label reused as grid evidence | reject as circular; oracle requires independent input or an explicitly marked synthetic scenario | +| BD10 | compare optimized R2IL on separate held-out windows | final per-tower K3 tuple, mechanism, duration and site/day multiplicity agree with independent scalar; no source leakage | + +`BD1`–`BD7` and K3 algebra have Rust unit tests in `outage_oracle.rs`. `BD8`–`BD10` are **unimplemented integration gates**, not claimed passing tests. No physical-mechanism learning, site↔bus registry, egglog extraction, observed Bangladesh calibration or R2IL differential is present in this patch. + +## Sequenced next changes + +1. **Fix the original data boundary**, preferably in a separate Bangladesh-outage PR: add independent `grid_supply_status`, `backup_status`, `telecom_path_status`, `observed_tower_service`, cause evidence and source timestamps. Preserve legacy generated `grid_status` under an unmistakably synthetic label. Use stable tower IDs and event IDs. Maintain grid-group to feeder/bus mappings separately; do not infer them from region adjacency. +2. Separate grid-event/tower-event simulation and apply generator/battery behaviour before generating service outages. Enumerate mutually independent faults and a shared-weather counterfactual. Remove `random.choices(..., k=...)` duplicate tower incidents unless explicitly modeled as separate overlapping events, and handle empty same-grid cascade candidate sets. +3. Rebuild label-safe, time-safe training: *no* `target_outage_15d` or other future-derived columns in features; historical rolling stats/encoders/scaling fitted on training windows only; walk-forward temporal evaluation and separated calibration. Use raw logits with `BCEWithLogitsLoss` or sigmoid with `BCELoss`, not both; enforce event and tower-group holdouts and class-specific diagnostics. +4. Only then attempt one bounded egglog 3LA-style offline extraction with independent scalar oracle on the first pure K3 site equation. Profile end-to-end against unoptimized Quack/mask-risc. Keep grid spectral numerics as separate operands with their own bounds and units, not interchangeable Boolean evidence. \ No newline at end of file diff --git a/crates/perturbation-sim/src/lib.rs b/crates/perturbation-sim/src/lib.rs index 5b416d6b2..9d2394bf4 100644 --- a/crates/perturbation-sim/src/lib.rs +++ b/crates/perturbation-sim/src/lib.rs @@ -68,6 +68,7 @@ pub mod hhtl; pub mod inertia_data; pub mod ingest; pub mod model; +pub mod outage_oracle; pub mod perturbation; pub mod place_buffer; pub mod resilience; @@ -103,6 +104,10 @@ pub use model::{ apply_aging, assess_capability, edge_age_factors, scale_susceptance, with_uniform_derate, AgeModel, Capability, DataLevel, }; +pub use outage_oracle::{ + assess_bound, assess_site, BoundObservation, BoundSiteFrame, Mechanism, Observed, + OracleCheck, Refusal, SiteAssessment, SiteFocus, SiteFrame, SourcePins, +}; pub use perturbation::{ spectral_perturbation, SpectralPerturbation, FRAGMENTATION_SENTINEL, SPECTRAL_GAP_FLOOR, }; diff --git a/crates/perturbation-sim/src/outage_oracle.rs b/crates/perturbation-sim/src/outage_oracle.rs new file mode 100644 index 000000000..8b1cdbd85 --- /dev/null +++ b/crates/perturbation-sim/src/outage_oracle.rs @@ -0,0 +1,326 @@ +//! Independent, scalar reference for grid-supply versus telecom-path failures. +//! +//! Intended for offline Bangladesh-outage fixtures and later compiler differential +//! tests, not an estimate of real Bangladesh grid/tower failure probabilities. +//! A `CascadeResult` is a grid stress/line-trip witness, **not** an observation +//! that a tower lost grid supply or service. + +/// Three-valued physical observation: `Unknown` means *not established*, not down. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Observed { + Up, + Down, + Unknown, +} + +impl Observed { + /// Strong-Kleene conjunction (conservatively evaluate missing observations). + pub const fn and(self, rhs: Self) -> Self { + match (self, rhs) { + (Self::Down, _) | (_, Self::Down) => Self::Down, + (Self::Up, Self::Up) => Self::Up, + _ => Self::Unknown, + } + } + + /// Strong-Kleene disjunction. + pub const fn or(self, rhs: Self) -> Self { + match (self, rhs) { + (Self::Up, _) | (_, Self::Up) => Self::Up, + (Self::Down, Self::Down) => Self::Down, + _ => Self::Unknown, + } + } + + pub const fn not(self) -> Self { + match self { + Self::Up => Self::Down, + Self::Down => Self::Up, + Self::Unknown => Self::Unknown, + } + } +} + +/// All input signals refer to the **same tower**, time window and pinned source +/// generation. `telecom_path` covers radio equipment and upstream backhaul; +/// `backup_supply` covers battery/generator/solar availability. Do not substitute +/// the Bangladesh A10H synthetic `grid_status` for independent `grid_supply`: +/// that script derives its value from the tower outage label itself. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SiteFrame { + pub grid_supply: Observed, + pub backup_supply: Observed, + pub telecom_path: Observed, + /// Independently observed end-user tower service, when available. + pub observed_tower_service: Observed, +} + +/// Descriptive mechanism attribution based ONLY on fully resolved *inputs*. +/// Not a statistically inferred cause and not a certification of grid physics. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Mechanism { + Neither, + GridPower, + TelecomPath, + Both, + Unresolved, +} + +/// Independent end-to-end service observation versus the model prediction. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum OracleCheck { + Agrees, + Contradiction, + InsufficientEvidence, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SiteAssessment { + /// Grid-source failure, irrespective of backup success or telecom status. + pub grid_supply_outage: Observed, + /// Failure of usable site power (grid OR backup). + pub power_unavailable: Observed, + /// Independent non-power telecom-path failure. + pub telecom_path_failure: Observed, + pub predicted_tower_service: Observed, + pub mechanism: Mechanism, + pub check: OracleCheck, +} + +/// Mathematical reference: `service = (grid_supply OR backup_supply) +/// AND telecom_path`. Keep this scalar implementation independent of any +/// egglog rewrite selection or R2IL/Quack/mask-risc lowering. +/// +/// The model is intentionally minimal and only applies when the observed +/// signals have the defined meaning. An upstream shared backhaul power outage +/// must be expressed as a telecom-path failure or modeled separately, never +/// silently inferred from a grid line-trip or a tower's offline label. +pub fn assess_site(frame: SiteFrame) -> SiteAssessment { + let power = frame.grid_supply.or(frame.backup_supply); + let predicted = power.and(frame.telecom_path); + let power_failure = power.not(); + let telecom_failure = frame.telecom_path.not(); + let mechanism = match (power_failure, telecom_failure) { + (Observed::Down, Observed::Down) => Mechanism::Neither, + (Observed::Up, Observed::Down) => Mechanism::GridPower, + (Observed::Down, Observed::Up) => Mechanism::TelecomPath, + (Observed::Up, Observed::Up) => Mechanism::Both, + _ => Mechanism::Unresolved, + }; + let check = match (predicted, frame.observed_tower_service) { + (Observed::Unknown, _) | (_, Observed::Unknown) => OracleCheck::InsufficientEvidence, + (lhs, rhs) if lhs == rhs => OracleCheck::Agrees, + _ => OracleCheck::Contradiction, + }; + SiteAssessment { + grid_supply_outage: frame.grid_supply.not(), + power_unavailable: power_failure, + telecom_path_failure: telecom_failure, + predicted_tower_service: predicted, + mechanism, + check, + } +} + +/// A stable, already-resolved tower identity and a common time window. It is +/// NOT a local ordinal from two unrelated tables; resolving foreign identities +/// belongs to an authorized, version-pinned binding stage before compilation. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SiteFocus { + pub tower_guid: u128, + pub window_start: i64, + pub window_end: i64, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct BoundObservation { + pub focus: SiteFocus, + pub source_generation: u64, + pub value: Observed, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct BoundSiteFrame { + pub grid_supply: BoundObservation, + pub backup_supply: BoundObservation, + pub telecom_path: BoundObservation, + pub observed_tower_service: BoundObservation, +} + +/// Independent generation pins: different source streams need not have the +/// same numeric generation, but each must match its own attested snapshot. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SourcePins { + pub grid: u64, + pub backup: u64, + pub telecom: u64, + pub service: u64, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Refusal { + InvalidWindow, + CrossFocus, + StaleSource, +} + +/// A cold compiler may only optimize after this check; arithmetic on local +/// positions cannot ever substitute for matching tower identities and time. +/// The returned oracle assessment is *not* a commit acknowledgement. +pub fn assess_bound(frame: BoundSiteFrame, pins: SourcePins) -> Result { + let focus = frame.grid_supply.focus; + if focus.window_start >= focus.window_end { + return Err(Refusal::InvalidWindow); + } + if frame.backup_supply.focus != focus + || frame.telecom_path.focus != focus + || frame.observed_tower_service.focus != focus + { + return Err(Refusal::CrossFocus); + } + if frame.grid_supply.source_generation != pins.grid + || frame.backup_supply.source_generation != pins.backup + || frame.telecom_path.source_generation != pins.telecom + || frame.observed_tower_service.source_generation != pins.service + { + return Err(Refusal::StaleSource); + } + Ok(assess_site(SiteFrame { + grid_supply: frame.grid_supply.value, + backup_supply: frame.backup_supply.value, + telecom_path: frame.telecom_path.value, + observed_tower_service: frame.observed_tower_service.value, + })) +} + +#[cfg(test)] +mod tests { + use super::*; + + const fn f(grid: Observed, backup: Observed, telecom: Observed, service: Observed) -> SiteFrame { + SiteFrame { + grid_supply: grid, + backup_supply: backup, + telecom_path: telecom, + observed_tower_service: service, + } + } + + #[test] + fn grid_trips_but_backup_keeps_telecom_up() { + let a = assess_site(f(Observed::Down, Observed::Up, Observed::Up, Observed::Up)); + assert_eq!(a.grid_supply_outage, Observed::Up); + assert_eq!(a.power_unavailable, Observed::Down); + assert_eq!(a.predicted_tower_service, Observed::Up); + assert_eq!(a.mechanism, Mechanism::Neither); + assert_eq!(a.check, OracleCheck::Agrees); + } + + #[test] + fn grid_failure_causes_tower_service_loss_without_backup() { + let a = assess_site(f(Observed::Down, Observed::Down, Observed::Up, Observed::Down)); + assert_eq!(a.mechanism, Mechanism::GridPower); + assert_eq!(a.check, OracleCheck::Agrees); + } + + #[test] + fn healthy_grid_with_telecom_path_failure_is_not_grid_failure() { + let a = assess_site(f(Observed::Up, Observed::Unknown, Observed::Down, Observed::Down)); + assert_eq!(a.grid_supply_outage, Observed::Down); + assert_eq!(a.mechanism, Mechanism::TelecomPath); + assert_eq!(a.check, OracleCheck::Agrees); + } + + #[test] + fn concurrent_grid_and_telecom_faults_keep_both_causes() { + let a = assess_site(f(Observed::Down, Observed::Down, Observed::Down, Observed::Down)); + assert_eq!(a.mechanism, Mechanism::Both); + } + + #[test] + fn unknown_backup_is_not_false_and_cannot_prove_grid_caused_service_loss() { + let a = assess_site(f(Observed::Down, Observed::Unknown, Observed::Up, Observed::Down)); + assert_eq!(a.predicted_tower_service, Observed::Unknown); + assert_eq!(a.mechanism, Mechanism::Unresolved); + assert_eq!(a.check, OracleCheck::InsufficientEvidence); + } + + #[test] + fn observed_outage_contradicting_healthy_inputs_is_refused() { + let a = assess_site(f(Observed::Up, Observed::Up, Observed::Up, Observed::Down)); + assert_eq!(a.check, OracleCheck::Contradiction); + } + + #[test] + fn missing_telecom_status_is_not_a_proven_hardware_fault() { + let a = assess_site(f(Observed::Up, Observed::Down, Observed::Unknown, Observed::Down)); + assert_eq!(a.mechanism, Mechanism::Unresolved); + assert_eq!(a.check, OracleCheck::InsufficientEvidence); + } + + #[test] + fn total_truth_table_kleene_and_commutes_with_pure_operand_order() { + let vals = [Observed::Up, Observed::Down, Observed::Unknown]; + for &a in &vals { + assert_eq!(a.and(a), a); + assert_eq!(a.or(a), a); + assert_eq!(a.not().not(), a); + for &b in &vals { + assert_eq!(a.and(b), b.and(a)); + assert_eq!(a.or(b), b.or(a)); + assert_eq!(a.and(b), a.not().or(b.not()).not()); + for &c in &vals { + assert_eq!(a.and(b).and(c), a.and(b.and(c))); + assert_eq!(a.or(b).or(c), a.or(b.or(c))); + assert_eq!(a.and(b.or(c)), a.and(b).or(a.and(c))); + } + } + } + } + + #[test] + fn stale_source_refuses_even_when_local_site_and_values_agree() { + let focus = SiteFocus { tower_guid: 7001, window_start: 0, window_end: 60 }; + let a = BoundObservation { focus, source_generation: 17, value: Observed::Up }; + let b = BoundObservation { source_generation: 18, ..a }; + let f = BoundSiteFrame { + grid_supply: a, + backup_supply: b, + telecom_path: a, + observed_tower_service: a, + }; + let pins = SourcePins { grid: 17, backup: 18, telecom: 17, service: 17 }; + assert_eq!(assess_bound(f, pins).unwrap().mechanism, Mechanism::Neither); + assert_eq!(assess_bound(f, SourcePins { backup: 19, ..pins }), Err(Refusal::StaleSource)); + } + + #[test] + fn same_row_ordinal_different_tower_guid_refuses_join() { + let focus = SiteFocus { tower_guid: 7001, window_start: 0, window_end: 60 }; + let site_2 = SiteFocus { tower_guid: 7002, ..focus }; + let a = BoundObservation { focus, source_generation: 1, value: Observed::Up }; + let f = BoundSiteFrame { + grid_supply: a, + backup_supply: BoundObservation { focus: site_2, ..a }, + telecom_path: a, + observed_tower_service: a, + }; + let pins = SourcePins { grid: 1, backup: 1, telecom: 1, service: 1 }; + assert_eq!(assess_bound(f, pins), Err(Refusal::CrossFocus)); + assert_eq!(assess_bound(BoundSiteFrame { + backup_supply: a, + grid_supply: BoundObservation { focus: SiteFocus { window_end: 0, ..focus }, ..a }, + ..f + }, pins), Err(Refusal::InvalidWindow)); + } + + #[test] + fn known_grid_status_never_equates_to_tower_availability() { + // The two are different observations. Generator is entirely independent + // of a tower's on/off status (the historical Bangladesh bug). + let generator_protected = assess_site(f(Observed::Down, Observed::Up, Observed::Up, Observed::Up)); + let independently_failed = assess_site(f(Observed::Up, Observed::Up, Observed::Down, Observed::Down)); + assert_ne!(generator_protected.grid_supply_outage, independently_failed.grid_supply_outage); + assert_ne!(generator_protected.predicted_tower_service, independently_failed.predicted_tower_service); + } +} \ No newline at end of file