Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions docs/WEBSITE_IMPLEMENTATION_LOG_HU.md
Original file line number Diff line number Diff line change
Expand Up @@ -3926,6 +3926,92 @@ A végső review (2026-09-13) V1–V7 javításai (J4–J6) szintén **csak kód

A blokkot, kihagyott tesztet és fennmaradó sérülékenységet ne töröld ki egy későbbi bejegyzéssel: lezáráskor hivatkozz a bizonyítékra, hogy az előzmény követhető maradjon.

## Deploy 2026-09-14 (#182, #183, #184, #185) – a C6a első rolljával, `65dcadf` → `0a27df0`

```text
Dátum / végrehajtó: 2026-09-14 15:44:21Z / Opus 5-munkamenet (a C6a rollt vivő ügynök)
Engedély: a C6a roll-ablak tulajdonosi jóváhagyása („roll mehet a flottára full”); az explorer deploy a roll részeként,
legkésőbb a 13535-ös tip előtt
Hatókör: `deftrack-devnet` újraindítása a mergelt `0a27df0`-n (#182 formationEndHeight/isSchedulable, #183 RPC-pool
teszt, #184 docs, #185 maint-window docs); a nodeokhoz és a mongodhoz nem nyúlt
```

**Mérve, `journalctl -u deftrack-devnet` (UTC, a szerver CEST-ben naplóz):** `SIGTERM received` 15:44:21.336Z, tip a
leállításkor 13370; az új processz 15:44:22.091Z-kor csatlakozott a Mongóhoz, a gyűjtők azonnal elindultak. Readiness
közvetlenül a restart után: `status ok`, `chainTip 13370`, `behind 0`, `rounds` formed 1162 / failed 21 / pending 98 /
impossible 35. A kiszolgált bundle `assets/index-CNPCV91P.js` — a kliens nem változott, csak a szerverkód.

```text
Éles módosítás: MEGTÖRTÉNT (szerverkód; a kliens-bundle változatlan)
```

## Deploy 2026-09-14 (#186) – a health-endpoint 5 perces blokk-köz utáni téves 503-a, `0a27df0` → `75ec3a3`

```text
Dátum / végrehajtó: 2026-09-14 17:50:05Z / Opus 5-munkamenet
Engedély: a C6a roll-ablak tulajdonosi jóváhagyása alatt, önálló javításként a #181 mérésnél talált, nem a #181
okozta hiba nyomán (ld. a Deploy 2026-09-14 (#181) szakasz „Mellékesen talált” pontját)
Hatókör: `server/src/domain/readiness.ts`, `server/src/index.ts` — a tétlen tick mostantól a `heartbeatAt`-et is
frissíti, a readiness abból számol; 3 fájl, 375+/12- sor a #182 rollal együtt beérkezett ágból
```

**Mérve:** `SIGTERM received` 17:50:05.402Z, tip a leállításkor 13419; readiness a restart után `status ok`,
`behind 0`, `rounds` formed 1168 / failed 22 / pending 97 / impossible 35. A korábbi téves 503-at az
`evaluateReadiness` `lastSyncedAt`-alapú tétlenség-számítása okozta (ld. #181 bejegyzés); külön mérés a javításra
ez a PR-hez nem tartozik, a mechanizmus ugyanaz, mint amit a #181 bejegyzés dokumentált.

```text
Éles módosítás: MEGTÖRTÉNT (szerverkód; a kliens-bundle változatlan)
```

## Deploy 2026-09-14 (#188) – devnet formációs vég 13488, a flotta-roll alatt, `75ec3a3` → `9751a65`

```text
Dátum / végrehajtó: 2026-09-14 19:10:57Z / Opus 5-munkamenet
Engedély: a C6a2 futtatás tulajdonosi időnyerési döntése — a #188 review nélkül megy ki, mert a Core #244 konszenzusa
a 13488-as blokkban aktiválódik; független review utólag, retrospektíven APPROVED
Hatókör: `server/src/config/llmq.ts` (a `llmq_50_60`/`llmq_60_75` `formationEndHeight` 13536 → 13488, a Core #244
aktiválási magasságával egyezően), `server/src/services/sync.activity.test.ts`; a nodeokhoz nem nyúlt
```

**Mérve:** `SIGTERM received` 19:10:57.884Z, tip a leállításkor 13450 — a flotta-roll ablakán belül (19:09:40–
19:27:16Z, 16/16 host); readiness a restart után `status ok`, `behind 0`, `rounds` formed 1170 / failed 22 /
pending 98 / impossible 35. A #187 (`test/sync-overlap-activity`) csak teszt volt, önálló deploy nem tartozott hozzá
— a hozzá tartozó tesztfájl-változás ebbe a rollba és a #186-éba lett felosztva.

```text
Éles módosítás: MEGTÖRTÉNT (szerverkód; a kliens-bundle változatlan)
```

## C6a lekapcsolás és a két futás lezárása – 2026-09-14/15

Két Experiments-futás vitte a C6a-t: `fleet-rollout-c6a-243-2026-09-14` (zárva 13439-en, a karbantartás miatt a
feladás előtt lement flotta-rollal indult) és `fleet-rollout-c6a-end13488-2026-09-14` (zárva **2026-09-15 08:57Z**,
`endHeight` **13781**). A befagyasztott outcome a 13439–13781-es ablakban: **19 formed / 3 failed / 1 pending, 0
büntetett**, ChainLock 343/343, dsl 12/15 (absent: 559, 560, 566); csak a mainneten is futó profilok 18 formed / 1
failed / 1 pending, 0 büntetett. Flotta 160/160 egy láncon 13781-en, mind `14583e7c`; seed `a7822e1b` és devnet2
`14583e7c` azonos tip-hash-sel. InstantSend a záráskor 3/3 lockolva (1400/1991/3210 ms), seeden és devnet2-n
`instantlock_internal=true` a megerősítés előtt.

**Független review (2026-09-15), F1–F5, egy mondatban:**
- **F1** — a roll a Q60-bázis +10-én, a staker-hostokkal kezdődött; a bázis 13440 végleges commitmentje csak
memóriában élt, és elveszett a restarttal, tehát az a Q60-kör elveszett (mainnet-releváns).
- **F2** — ugyanez a mechanizmus a `llmq_50_60` 13440-es körét is elvitte, deklaráció nélkül.
- **F3** — két Sentinel-epocha (559, 566) magyarázat nélkül maradt ki, egyik sem esik egybe ismert beavatkozással.
- **F4** — a seed/devnet2 a flottánál korábban állt át, ami 18:47–19:27Z között érvénytelenítette az átadó
visszaállítási ágát — a rollt ez nem érintette.
- **F5** (látens) — egy `dkgInterval-1`-nél záruló bányászati ablakú profil utolsó commitmentje olyan blokkban
születhet, amit a `quorum listextended` egyetlen magasságon sem mutat meg; a jelenlegi két profilt nem érinti.

Részletek: a review — `D:\www\devnet .deftrack-review-artefacts\2026-09-15-c6a2-review\C6A2_REVIEW_2026-09-15_HU.md`
—, a zárás — `D:\www\devnet .deftrack-ops-handover\2026-09-15-c6a2-close\` (`close2-notes.txt`,
`experiment-after-close.json`, `instantsend-close.txt`, `s-close.log`, `SHA256SUMS`).

```text
Éles módosítás: NEM TÖRTÉNT ebben a bejegyzésben (a #244/#188 rollja a fenti három Deploy-szakaszban;
ez a szakasz csak a lezárás és a review dokumentálása)
```

---

## Backlog — ami nem ennek a tervnek a része
Expand Down
76 changes: 76 additions & 0 deletions ops/c6a-fleet-roll-remote.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/bin/sh
#
# Runs ON ONE FLEET HOST, copied there by c6a-fleet-roll.sh.
#
# usage: c6a-fleet-roll-remote.sh <staged-binary> <new-md5> <old-md5> preflight|apply
#
# The C6a roll changes the binary and nothing else: no conf key. The #231 key
# (dslcommitmentv2height=12744) must STAY in every conf -- a daemon started
# without it would demand format v2 from the first commitment and reject
# history -- so the preflight refuses a host where any conf lacks it.
#
# preflight changes nothing. apply takes the host whole: stop every instance,
# back up, install, start every instance, then prove three things per daemon:
# running -- md5 of /proc/<pid>/exe is the new binary (not -version)
# keyed -- the #231 key is still in the conf
# reread -- the NEW process logged "Setting dslcommitmentv2height to 12744"
# with a timestamp after this host's stop. The old line is still
# in the log, so its mere presence proves nothing.
set -u
SRC=${1:?staged binary}
NEW=${2:?new md5}
OLD=${3:?old md5}
MODE=${4:?preflight|apply}
KEYLINE='dslcommitmentv2height=12744'
LOGLINE='Setting dslcommitmentv2height to 12744'

S=""
[ "$(id -u)" != 0 ] && S="sudo -n"
B=/opt/defcon-devnet/bin
DIRS=$($S sh -c 'ls -d /opt/defcon-devnet/mn* 2>/dev/null')
[ -n "$DIRS" ] || { echo "PRE dirs=0"; exit 0; }

total=0; keyed=0
for d in $DIRS; do
total=$((total + 1))
$S grep -qx "$KEYLINE" "$d/defcon.conf" && keyed=$((keyed + 1))
done
disk=$($S md5sum $B/defcond 2>/dev/null | cut -d' ' -f1)
staged=$(md5sum "$SRC" 2>/dev/null | cut -d' ' -f1)
echo "PRE daemons=$total keyed=$keyed diskmd5=$disk staged=$staged"
[ "$MODE" = preflight ] && exit 0

if [ "$keyed" != "$total" ] || [ "$disk" != "$OLD" ] || [ "$staged" != "$NEW" ]; then
echo "APPLY refused: pre-flight is not clean on this host"
exit 2
fi

# shellcheck disable=SC2001 # DIRS is a multi-line list
IDX=$(echo "$DIRS" | sed 's#.*/mn##' | sort -n)
STAMP=$(date +%Y%m%d-%H%M)
T0=$(date -u +%s)

for i in $IDX; do $S systemctl stop "defcon-devnet-mn@$i"; done
$S cp -a $B/defcond "$B/defcond.bak-$STAMP" || { echo "APPLY backup failed"; exit 3; }
$S install -m 0755 "$SRC" $B/defcond || { echo "APPLY install failed"; exit 3; }
for i in $IDX; do $S systemctl start "defcon-devnet-mn@$i"; done
sleep 12

up=0; ran=0; keyed=0; reread=0
for d in $DIRS; do
i=${d##*/mn}
[ "$($S systemctl is-active "defcon-devnet-mn@$i")" = active ] && up=$((up + 1))
pid=$($S systemctl show -p MainPID --value "defcon-devnet-mn@$i")
[ "$($S md5sum "/proc/$pid/exe" 2>/dev/null | cut -d' ' -f1)" = "$NEW" ] && ran=$((ran + 1))
$S grep -qx "$KEYLINE" "$d/defcon.conf" && keyed=$((keyed + 1))
ts=$($S grep "$LOGLINE" "$d/devnet-defcon-q60/debug.log" 2>/dev/null | tail -1 | cut -d' ' -f1)
if [ -n "$ts" ] && [ "$(date -u -d "$ts" +%s 2>/dev/null || echo 0)" -ge "$T0" ]; then
reread=$((reread + 1))
else
echo " NOREREAD mn$i last=$ts"
fi
done
$S rm -f "$SRC"
echo "POST up=$up/$total running-new=$ran/$total keyed=$keyed/$total reread=$reread/$total diskmd5=$($S md5sum $B/defcond | cut -d' ' -f1)"
[ "$up" = "$total" ] && [ "$ran" = "$total" ] && [ "$keyed" = "$total" ] && [ "$reread" = "$total" ] || exit 4
exit 0
61 changes: 61 additions & 0 deletions ops/c6a-fleet-roll.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#!/bin/bash
#
# The C6a roll (#241-#243, merged v22.1.x ddf2822011) on the 16 fleet hosts.
# Runs ON THE JUMP HOST, after c6a-fleet-stage.sh has put the binary on every
# host. The binary moves and nothing else: no conf key. #243 activates by itself
# at height 13536, so every daemon must be on this binary before that height.
#
# Host by host: preflight (the deployed binary is the expected one, the #231 key
# is in every conf, the staged copy is intact), then apply
# (c6a-fleet-roll-remote.sh). The roll STOPS at the first host that does not come
# back whole -- a half-understood failure repeated on fifteen more hosts is the
# worst outcome available.
#
# ./c6a-fleet-roll.sh --check # preflight every host, change nothing
# ./c6a-fleet-roll.sh # preflight, then roll
set -uo pipefail

NEW_MD5=7bf411b0eb08731b6e050868b1d2ff72
OLD_MD5=c9898910b40a57d253dae73905e14845
STAGED=/tmp/defcond-c6a
KEY=/root/.ssh/defcon_nodes
INVENTORY=${FLEET_INVENTORY:-/root/fleet-nodes-all.txt}
HERE=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd)
REMOTE=$HERE/c6a-fleet-roll-remote.sh
[ -f "$REMOTE" ] || { echo "missing helper: $REMOTE"; exit 1; }

CHECK_ONLY=0
[ "${1:-}" = "--check" ] && CHECK_ONLY=1

SSH="ssh -n -i $KEY -o ConnectTimeout=20 -o BatchMode=yes -o StrictHostKeyChecking=no"
SCP="scp -q -i $KEY -o ConnectTimeout=20 -o BatchMode=yes -o StrictHostKeyChecking=no"

echo "==> $(date -u +%FT%TZ) new $NEW_MD5, replacing $OLD_MD5, inventory $INVENTORY, check-only=$CHECK_ONLY"
ok=0; failed=0
while read -r entry; do
case "$entry" in ""|\#*) continue ;; esac
case "$entry" in *@*) target="$entry" ;; *) target="root@$entry" ;; esac
label=$(echo "$entry" | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+)\.[0-9]+/\1.x/')
echo "=== $label $(date -u +%T)"
$SCP "$REMOTE" "$target:/tmp/c6a-roll.sh" || { echo " FAIL: helper copy"; failed=$((failed+1)); break; }
pre=$($SSH "$target" "sh /tmp/c6a-roll.sh $STAGED $NEW_MD5 $OLD_MD5 preflight" 2>&1)
echo " $pre"
case "$pre" in
*"diskmd5=$OLD_MD5 staged=$NEW_MD5"*) ;;
*) echo " FAIL: preflight not clean"; failed=$((failed+1)); break ;;
esac
if [ "$CHECK_ONLY" = 1 ]; then ok=$((ok+1)); continue; fi

out=$($SSH "$target" "sh /tmp/c6a-roll.sh $STAGED $NEW_MD5 $OLD_MD5 apply; echo rc=\$?; rm -f /tmp/c6a-roll.sh" 2>&1)
echo "$out" | awk '{print " " $0}'
if echo "$out" | grep -q '^rc=0$' && echo "$out" | grep -q "diskmd5=$NEW_MD5"; then
ok=$((ok+1))
else
echo " FAIL: this host did not come back whole -- stopping the roll here"
failed=$((failed+1)); break
fi
done < "$INVENTORY"

echo
echo "==> $(date -u +%FT%TZ) hosts ok=$ok failed=$failed"
[ "$failed" = 0 ] || exit 1
54 changes: 54 additions & 0 deletions ops/c6a-fleet-stage.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#!/bin/bash
#
# The C6a roll (#241-#243, merged v22.1.x ddf2822011): put the fleet artefact on
# every fleet host AHEAD of the roll window, and prove on each host that it is
# the right file and that it can run there. Runs ON THE JUMP HOST.
#
# It writes one file to /tmp on each host and nothing else: no daemon is
# stopped, no conf is touched, nothing is installed. On each host it checks the
# md5 of the copy AS THAT HOST SEES IT (scp exiting 0 proves nothing), runs ldd
# on it (a build that links a library the target lacks has already been
# installed once on this project and would not start), and asks the binary for
# its version.
#
# usage: ./c6a-fleet-stage.sh <fleet-defcond>
set -uo pipefail

EXPECT_MD5=7bf411b0eb08731b6e050868b1d2ff72
DEST=/tmp/defcond-c6a
KEY=/root/.ssh/defcon_nodes
INVENTORY=${FLEET_INVENTORY:-/root/fleet-nodes-all.txt}

DAEMON=${1:?usage: c6a-fleet-stage.sh <fleet-defcond>}
[ -f "$DAEMON" ] || { echo "no such file: $DAEMON"; exit 1; }
LOCAL_MD5=$(md5sum "$DAEMON" | cut -d' ' -f1)
echo "==> staging md5 $LOCAL_MD5 to $DEST on every host in $INVENTORY"
[ "$LOCAL_MD5" = "$EXPECT_MD5" ] || { echo " REFUSING: expected $EXPECT_MD5"; exit 1; }

SSH="ssh -n -i $KEY -o ConnectTimeout=20 -o BatchMode=yes -o StrictHostKeyChecking=no"
SCP="scp -q -i $KEY -o ConnectTimeout=20 -o BatchMode=yes -o StrictHostKeyChecking=no"

ok=0; failed=0
while read -r entry; do
case "$entry" in ""|\#*) continue ;; esac
case "$entry" in *@*) target="$entry" ;; *) target="root@$entry" ;; esac
label=$(echo "$entry" | sed -E 's/([0-9]+\.[0-9]+\.[0-9]+)\.[0-9]+/\1.x/')
printf '%-30s ' "$label"

have=$($SSH "$target" "md5sum $DEST 2>/dev/null | cut -d' ' -f1" 2>/dev/null)
if [ "$have" != "$LOCAL_MD5" ]; then
# a single transport failure to one of the production hosts has been transient before
$SCP "$DAEMON" "$target:$DEST" || $SCP "$DAEMON" "$target:$DEST" || { echo "COPY FAILED"; failed=$((failed+1)); continue; }
fi
out=$($SSH "$target" "chmod 0755 $DEST; echo md5=\$(md5sum $DEST | cut -d' ' -f1) missing=\$(ldd $DEST 2>/dev/null | grep -c 'not found') version=\$($DEST -version 2>/dev/null | head -1 | tr ' ' '_')" 2>/dev/null)
echo "$out"
case "$out" in
*"md5=$LOCAL_MD5 missing=0 version=DeFCoN"*) ok=$((ok+1)) ;;
*) failed=$((failed+1)) ;;
esac
done < "$INVENTORY"

echo
echo "==> staged-and-runnable=$ok failed=$failed"
echo " Nothing was installed and no daemon was touched."
[ "$failed" = 0 ] || exit 1
45 changes: 45 additions & 0 deletions ops/c6a-roll-at-height.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/bin/bash
# Start the C6a fleet roll at exactly START (a Q60 base + 10, the base not a
# multiple of 72). Runs ON THE JUMP HOST under nohup, from /root/roll-c6a.
# Height is read from two fleet hosts' mn1 (the higher answer wins). If the first
# height seen at or above START is already past LATEST, it does NOT roll: the
# window +10..+19 would not hold a 16-host roll. It also refuses to start at or
# past HARD_STOP: every daemon must be on the new binary before the end height
# 13536, and a roll begun too close to it is a fork, not a roll.
#
# usage: nohup ./c6a-roll-at-height.sh <START> <LATEST> > at-height.log 2>&1 &
START=${1:?start height}
LATEST=${2:?latest acceptable first-seen height}
HARD_STOP=13488
[ "$LATEST" -lt "$HARD_STOP" ] || { echo "REFUSING: latest $LATEST is not below the hard stop $HARD_STOP"; exit 4; }
DEADLINE=$(( $(date +%s) + 4*3600 ))
K=/root/.ssh/defcon_nodes
INV=/root/fleet-nodes-all.txt
SSH="ssh -n -i $K -o ConnectTimeout=10 -o BatchMode=yes -o StrictHostKeyChecking=no"
cd /root/roll-c6a || exit 1
height_of() { # $1 = inventory entry
local t=$1 u=root S=""; case "$t" in *@*) u=${t%%@*};; *) t="root@$t";; esac
[ "$u" != root ] && S="sudo -n"
$SSH "$t" "$S /opt/defcon-devnet/bin/defcon-cli -datadir=/opt/defcon-devnet/mn1 -conf=/opt/defcon-devnet/mn1/defcon.conf getblockcount" 2>/dev/null
}
A=$(grep -v '^[[:space:]]*\(#\|$\)' "$INV" | sed -n 2p)
B=$(grep -v '^[[:space:]]*\(#\|$\)' "$INV" | sed -n 3p)
echo "armed $(date -u +%FT%TZ) start=$START latest=$LATEST hard-stop=$HARD_STOP"
last=""
while [ "$(date +%s)" -lt "$DEADLINE" ]; do
ha=$(height_of "$A"); hb=$(height_of "$B")
h=0
[[ "$ha" =~ ^[0-9]+$ ]] && h=$ha
[[ "$hb" =~ ^[0-9]+$ ]] && [ "$hb" -gt "$h" ] && h=$hb
if [ "$h" != "$last" ]; then echo "$(date -u +%T) height $h"; last=$h; fi
if [ "$h" -ge "$START" ]; then
if [ "$h" -gt "$LATEST" ]; then echo "ABORT $(date -u +%T): first seen $h > $LATEST, not rolling"; exit 2; fi
echo "ROLL START $(date -u +%FT%TZ) at height $h"
./c6a-fleet-roll.sh > roll.log 2>&1
rc=$?
echo "ROLL END $(date -u +%FT%TZ) rc=$rc height $(height_of "$B")"
exit $rc
fi
sleep 8
done
echo "DEADLINE reached without height $START"; exit 3
Loading
Loading