Skip to content

Published archive: message text empty since 2026-10-09, and text repair overwrote existing text with empty content #321

Open

Description

@roboclaw-bot

Summary

Since about 2026-10-09 18:00 UTC, every new human message in the OpenClaw guild's published git-share archive has content = "" and text_parts_json = [].

Separately, on 2026-10-26 16:41–16:49 UTC (with a few rows on Oct 23 and 25), about 2,100 older rows were rewritten to text_version = 2 with empty text. Many of those rows had text in earlier share commits.

The same messages read through a bot that has message-content access do return their text. Two problems seem to be combined:

  1. The publisher appears to have lost message-content access. This is operational, but discrawl published empty rows without any warning.
  2. Text repair replaced existing non-empty text with an empty re-read, which loses data in the archive.

Evidence

Observed from a share subscriber running discrawl 0.17.0. Aggregates only. Scope: OpenClaw guild, non-bot authors, message_type 0 or 19.

New messages captured empty. In rows still at text_version = 0 (never rewritten), the empty rate is normal until Oct 9 17:00 UTC, then becomes total:

Hour (UTC) Rows Empty
2026-10-09 17:00 77 7
2026-10-09 18:00 43 43
Every hour after that, through 2026-10-27 03:00 all all

Before that, normal. From Sep 24 to 30, 0–3 rows per day out of roughly 1,000–2,000 were empty, after excluding rows with attachments, stickers or embeds.

Existing text overwritten.

  • There are 2,123 rows created in October 2026 with text_version = 2, across 53 channels. Of these, 2,027 have empty content and text_parts_json = [].
  • Their updated_at values fall between 2026-10-23T16:55Z and 2026-10-26T16:49Z. 2,188 of the 2,197 text_version = 2 rows created before Oct 10 were updated on Oct 26.
  • I sampled 300 of these rows: human, created Oct 1–8, no attachments.
    • In share commit 8aa145b (2026-10-21T23:13Z), 297 of the 300 were present and 268 had non-empty content.
    • In share HEAD 8054648 (2026-10-27T03:22Z), all 300 are empty, at text_version = 2, and updated 2026-10-26 16:41–16:49 UTC.

Likely causes (unconfirmed)

  • Lost content access. Around 2026-10-09 18:00 UTC, the publishing bot most likely lost Discord's privileged MESSAGE_CONTENT access, for example through the intent toggle, a token or app change, or its config. Without it, Discord returns empty content to both Gateway and REST, which matches every new row being empty.
  • Repair trusts empty re-reads. The 2026-10-26 rewrite looks like text repair or re-read (resumable text repair shipped in v0.17.0) running under the same missing access. It accepted an empty re-read as authoritative and replaced stored text.

Expected behavior

  • Repair or re-read should never replace non-empty stored text with empty content when nothing else indicates the message changed (for example, edited_at is unchanged and attachments, embeds and components are unchanged). At minimum, it should refuse when the read lacks content access.
  • Sync, doctor or status should flag missing message-content access, for example when every non-bot message in a batch arrives with empty content, rather than silently publishing empty rows.

Recovery notes

  • Rows rewritten on Oct 23–26 can be restored from earlier share commits (for example 8aa145b).
  • Messages first captured after 2026-10-09 18:00 UTC were never archived with text. They need a re-read once content access is restored on the publisher.

Activity

  1. added
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    clawsweeper:intake:needs-reproPreliminary submission classification; full source review pending.
    P0Emergency: data loss, security bypass, crash loop, or unusable core runtime.
    clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.
    clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.
    clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.
    impact:data-lossThis issue is about lost, corrupted, or silently dropped user/session/config data.
    issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.
    and removed
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    clawsweeper:intake:needs-reproPreliminary submission classification; full source review pending.
    on Oct 27, 2026
  2. clawsweeper commented on Oct 27, 2026

    @clawsweeper
    Contributor

    Codex review: this still needs some work.

    Summary
    Keep this issue open: v0.19.0 includes bounded overwrite protection, but publisher visibility, historical recovery, and the requested warning remain unresolved.

    Reproducibility: The production incident was not reproduced; source confirms the released preservation boundary and missing visibility measurement, but private archive evidence and publisher access still require operator verification.

    Root-cause cluster
    Relationship: canonical
    Canonical: #321
    Summary: This issue retains the broader incident scope; the preservation fix and diagnostic proposal address separate parts without establishing recovery.

    Members:

    Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

    Maintainer decision needed

    • Question: Should suspicious empty captures produce an advisory Doctor warning or also block publication?
    • Recommendation: Start with an advisory warning: After operator verification, define a bounded signal and actionable recovery guidance without silently treating every empty message as an access failure.
    • Why: Empty content alone does not prove missing access, and neither the report nor the released preservation guard defines a safe publication policy.

    Next step
    Complete the existing operator visibility handoff, assign archive recovery ownership, and decide Doctor warning semantics while preserving the diagnostic hold.

    Review details

    Best possible solution:

    Verify publisher visibility, restore recoverable text with backup and sample validation, and provide an actionable Doctor warning for suspected missing content.

    Do we have a high-confidence way to reproduce the issue?

    The production incident was not reproduced; source confirms the released preservation boundary and missing visibility measurement, but private archive evidence and publisher access still require operator verification.

    Is this the best way to solve the issue?

    The released guard addresses the bounded overwrite case; remaining work should follow the existing operator handoff and an agreed warning policy rather than assume derived text repair caused canonical loss.

    AGENTS.md: found but not applied because it conflicted with ClawSweeper's review contract.

    Remaining risk / open question:

    • Publisher content access, deployment of the preservation fix, and recovery completeness remain unverified.
    • The reported archive comparisons do not identify the canonical-text erasing writer.
    • A missing-content warning needs semantics that distinguish suspected access loss from legitimate empty messages.

    Codex review notes: model internal, reasoning medium; reviewed against 4ea7d52ed6f8.

    Label changes

    Label changes:

    No label changes.

    Label justifications:

    • P0: The report documents widespread archive text loss and empty subsequent captures, with present publisher health and recovery still unverified.
    • impact:data-loss: Previously populated archive rows reportedly became empty, and later messages were published without authored text.
    Evidence reviewed

    What I checked:

    • Review policy and revision: Read the complete root AGENTS.md. Its shared-review-helper instruction conflicts with the explicit single-reviewer requirement, so no helper or second reviewer ran. GitHub main and the checkout both resolve to this SHA; the working tree remained clean. (AGENTS.md:3, 4ea7d52ed6f8)
    • Reported archive loss: The full report describes 268 previously populated sampled messages becoming empty and entirely empty new human-message captures after October 9. These private archive comparisons and the suspected publisher access loss were not independently verified.
    • Released preservation guard: fix: preserve authored text on ambiguous empty rereads #325 preserves the entire stored mutation when an unordered empty reread only loses authored text without changing message shape. Current main retains the guard; git tag ancestry includes v0.19.0. New rows, distinct rich-message changes, and historical restoration are outside its stated scope. (internal/store/write.go:663, b68eb958a1e5)
    • Preservation coverage inspected: Existing tests exercise real SDK decoding, REST persistence and Gateway refetches into SQLite and search, comparing canonical and related state after empty rereads. Tests were inspected but not executed during this read-only review. (internal/syncer/empty_snapshot_test.go:179, 4ea7d52ed6f8)
    • Repair does not establish the erasing writer: RepairMessageTextBatch derives projections from stored receipts, rejects canonical-content mismatches, and updates normalized text, attributed parts, version and indexes without replacing canonical content or fetching Discord. A text_version transition alone therefore does not prove which writer erased text. (internal/store/text_repair.go:105, 4ea7d52ed6f8)
    • Remaining visibility boundary: Doctor checks bot identity and accessible guilds, without measuring message-content visibility. Bot setup requires Message Content Intent, while the storage guard accepts observations when no stored message exists. (internal/cli/admin_commands.go:662, 4ea7d52ed6f8)

    Likely related people:

    • Hannes Rudolph: Raw commit 6d23bc9 adds internal/store/text_repair.go:31 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 6d23bc9a9286; files: internal/store/text_repair.go)
    How this review workflow works
    • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
    • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
    • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
    • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
    • Maintainers can also comment @clawsweeper review to request a fresh review only.
    • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge; those execution lanes are retired.
    • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

    Reviewed November 2, 2026, 2:20 PM ET / 19:20 UTC.

  3. added
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    clawsweeper:intake:needs-reproPreliminary submission classification; full source review pending.
    clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.
    and removed
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    on Oct 27, 2026
  4. 281 remaining items

  5. added
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    clawsweeper:intake:needs-reproPreliminary submission classification; full source review pending.
    and removed
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    clawsweeper:intake:needs-reproPreliminary submission classification; full source review pending.
    on Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Emergency: data loss, security bypass, crash loop, or unusable core runtime.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.impact:data-lossThis issue is about lost, corrupted, or silently dropped user/session/config data.issue-rating: 🦪 silver shellfishThin issue quality; more reproduction proof or environment detail is needed.

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions