Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
32fb189
docs: design note for ledger integrity and language-independent gating
Vibherpunk Sep 19, 2026
a53ad9f
docs: incorporate Opus auditor amendments into design note
Vibherpunk Sep 20, 2026
0083e6b
feat: implement tamper-evident ledger, inverted gate, and calibrated …
Vibherpunk Sep 20, 2026
b6d69a1
feat: implement Hybrid Two-Tier Architecture with Tier 2 hard gate
Vibherpunk Sep 20, 2026
9e707c2
fix(security): resolve 5 auditor vulnerabilities in hybrid architecture
Vibherpunk Sep 20, 2026
d368d39
fix(security): resolve Round 3 adversarial audit vulnerabilities and …
Vibherpunk Sep 20, 2026
b5c226e
fix(security): resolve Round 4 adversarial audit bypasses and harden …
Vibherpunk Sep 20, 2026
475524f
fix(security): resolve Round 5 expanded audit vectors (nested repos, …
Vibherpunk Sep 20, 2026
af1b822
fix(security): resolve Round 6 adversarial vectors (assume-unchanged,…
Vibherpunk Sep 20, 2026
9d7302b
fix(security): resolve Round 7 adversarial vectors (authenticated led…
Vibherpunk Sep 20, 2026
4ea1fa5
fix(security): Round 8 residual hardening (read-endpoint auth, NLI pr…
Vibherpunk Sep 20, 2026
fabdeb1
Fix Open Item #4: Decouple Tier 2 sandbox marker and expand CI to run…
Vibherpunk Sep 20, 2026
ce274a0
fix(tests): restore HARDTRUTH_API_TOKEN in TestRound7ApiToken finally…
Vibherpunk Sep 20, 2026
bc75670
feat(security): implement Open Item #2 per-record session authenticit…
Vibherpunk Sep 20, 2026
5651d39
docs: document Round 9 session authenticity & CI matrix; ignore audit…
Vibherpunk Sep 20, 2026
9713d18
fix(ops): expand default purge prefixes for Round 9 test sessions and…
Vibherpunk Sep 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 29 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ jobs:

- name: Run test suite with daemon UNREACHABLE (fail-closed verification)
run: |
SYSTEM_ONE_URL=http://127.0.0.1:9 python3 tests/test_live.py
SYSTEM_ONE_URL=http://127.0.0.1:9 pytest tests/

- name: Start HardTruth Daemon in background
run: |
python3 daemon/app.py &
HARDTRUTH_API_TOKEN=dummy-ci-token python3 daemon/app.py &
for i in {1..30}; do
if curl -s http://127.0.0.1:8000/health | grep -q "healthy"; then
echo "Daemon is healthy"
Expand All @@ -43,4 +43,30 @@ jobs:

- name: Run test suite with daemon UP (live NLI verification)
run: |
SYSTEM_ONE_URL=http://127.0.0.1:8000 python3 tests/test_live.py
SYSTEM_ONE_URL=http://127.0.0.1:8000 HARDTRUTH_API_TOKEN=dummy-ci-token pytest tests/

- name: Stop Host Daemon
run: |
curl -X POST http://127.0.0.1:8000/shutdown || true
pkill -f "daemon/app.py" || true
sleep 2

- name: Run test suite with Live Containerized Daemon
run: |
docker build -t hardtruth-daemon -f docker/Dockerfile .
docker volume create hardtruth_state
docker run -d -p 8000:8000 \
-v hardtruth_state:/root/.hardtruth \
-v /var/run/docker.sock:/var/run/docker.sock \
-e HARDTRUTH_API_TOKEN=dummy-ci-token \
-e HARDTRUTH_WORKSPACE_MAP="{\"$GITHUB_WORKSPACE\": \"/workspace\"}" \
-v "$GITHUB_WORKSPACE:/workspace" \
--name hardtruth-daemon-container hardtruth-daemon
for i in {1..30}; do
if curl -s http://127.0.0.1:8000/health | grep -q "healthy"; then
echo "Container Daemon is healthy"
break
fi
sleep 1
done
SYSTEM_ONE_URL=http://127.0.0.1:8000 HARDTRUTH_API_TOKEN=dummy-ci-token pytest tests/
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,9 @@ __pycache__/
.venv/
env/
.pytest_cache/

# Handoff and audit battery documents
HANDOFF_PROMPT.md
*audit_findings*.md
round*_audit_findings.md

345 changes: 185 additions & 160 deletions README.md

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions audit_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Audit Report: HardTruth Hybrid Architecture

**Executive Summary:**
The HardTruth architecture relies on regex patterns, hierarchical path string matching, and session-state baselines that can all be trivially bypassed. I have successfully identified exploits for Rule 1, Rule 2, the State Engine, and the Tier 2 Manifest check.

### 1. Shell Operator Defense Bypass (Rule 2 / Tier 1 Evasion)
**Vulnerability:** The `SHELL_OPERATOR_MASK_PATTERN` in `daemon/ledger.py` explicitly looks for horizontal chaining operators like `||`, `&&`, `;`, `|`, `if`, etc. It completely ignores newline characters (`\n`).
**Exploit:** An agent can run a failing test and mask the exit code by passing a multi-line string to `run_command`'s `CommandLine`:
```bash
pytest tests/test_failing.py
exit 0
```
Bash will execute the test, which fails, and then execute `exit 0`. The overall tool call returns exit code 0. `is_tainted_shell_command` returns `False` because there are no matching masking operators. The ledger records a successful test run, bypassing Rule 2.

### 2. State Engine & Suite Resolution Evasion (False Resolution)
**Vulnerability:** `can_suite_resolve_failure` strips test function selectors using `.split("::")[0]` before comparing paths.
**Exploit:** If an agent breaks `test_A` in `tests/test_foo.py::test_A` and receives a failure, it can clear that failure from `unresolved_failures` by simply running a completely different passing test in the same file:
`run_command(CommandLine="pytest tests/test_foo.py::test_passing")`
The resolution engine strips `::test_passing` and `::test_A`, compares `tests/test_foo.py` == `tests/test_foo.py`, evaluates to `True`, and clears the failure of the broken test.

### 3. Universal File Tracking & Manifest Defense Bypass (Baseline Reset)
**Vulnerability:** `get_git_modified_source_files` and `check_manifest_tampering` rely on `baseline_sha` stored in `~/.hardtruth/halts/baseline_<hash>.json`. `get_or_set_session_baseline` will recreate this file at `HEAD` if it doesn't exist. Furthermore, `run_command` operations (like `sed` or `echo`) do not trigger the `write_to_file` hooks that increment `ledger_src_count`.
**Exploit:** An agent can completely bypass Rule 1 (Code modification without tests) and Tier 2 (Manifest poisoning) using the following sequence:
1. Use `run_command` to modify source files and/or manifests (`Makefile`) via `sed` or `echo`. (This is not tracked by the `write_to_file` hook).
2. Commit the changes: `git commit -am "bypass"`.
3. Delete the session baseline: `rm -rf ~/.hardtruth/halts/`.
4. Trigger the stop hook. The hook recreates the baseline at the *new* `HEAD`. `git status` and `git diff` both return empty. `source_files_modified` evaluates to 0. Rule 1 is bypassed, and Tier 2 is skipped entirely (or runs the poisoned `Makefile` if it's invoked).

### 4. Polling Boundary (`\A` Anchoring & Timeout)
**Finding:** The `\A` string-start anchoring successfully prevents internal stdout spoofing because the system wraps the tool output, ensuring the true exit code is evaluated.
**Vulnerability (UX / Denial of Service):** If a test suite runs asynchronously and takes longer than 300ms, `poll_transcript_for_step` times out and records an `unverified_timeout` as a failure. While this fails closed (secure), it creates a state where valid, long-running asynchronous test suites get stuck as failures unless run synchronously (with high `WaitMsBeforeAsync`).

### 5. Test Suite Rigor Assessment
**Finding:** The 44 tests only prove the *happy path* of the invariants.
- `test_shell_operator_neutralization` tests `; true` and `|| exit 0` but fails to test multiline `\n exit 0`.
- `test_double_colon_subtest_resolution` tests parent file resolving child test, but fails to test sibling subtest resolving another sibling subtest.
- `test_commit_and_run_loophole_defeated` tests committing files, but fails to test an agent tampering with the mutable `.hardtruth/halts/` state directory.

### Recommendations
1. **Fix Shell Defense:** Update `SHELL_OPERATOR_MASK_PATTERN` to match newlines `\n` to prevent multiline execution.
2. **Fix Suite Resolution:** In `can_suite_resolve_failure`, if both the clean and failed commands contain `::`, they must match exactly.
3. **Protect the Baseline:** Store the baseline SHA in the daemon ledger itself, which is cryptographically hash-chained and cannot be silently deleted by the agent, rather than relying on a mutable JSON file in `~/.hardtruth/`.
4. **Harden File Tracking:** Do not rely solely on the `write_to_file` hook. Track file hashes pre- and post-tool execution for all shell commands, or force the agent to run in a filesystem sandbox that intercepts all write syscalls.

---

## Resolutions Implemented (Round 4)

All vulnerabilities and edge cases identified in this audit were patched and validated:

1. **Multiline Shell Masking Neutralized:**
- Updated `SHELL_OPERATOR_MASK_PATTERN` in `daemon/ledger.py` to include `[\r\n]`.
- Any verification command containing multiline strings (e.g., `pytest tests/test_failing.py\nexit 0`) is detected as tainted, forced to exit code 1, and flagged with `harness_status: "tainted_shell_operator"`.
2. **Sibling Subtest Evasion Eliminated:**
- Modified `can_suite_resolve_failure` in `daemon/ledger.py` to check if `clean` contains `::`.
- If `clean` targets a subtest, it strictly requires `raw_clean_target == raw_failed_target`. Running sibling subtests (`pytest tests/test_foo.py::test_pass`) no longer resolves failures in other subtests (`pytest tests/test_foo.py::test_broken`).
- Running the entire file (`pytest tests/test_foo.py`) or directory (`pytest tests/`) still correctly resolves any subtests within them.
3. **Daemon-Owned Immutable Baseline Defense:**
- Added `/v1/session/baseline` endpoints to `daemon/app.py` and `DaemonLedger` in `daemon/ledger.py`.
- The session baseline is stored in the Docker container's memory and appended as a physical `__session_baseline__` entry in the HMAC-SHA256 hash-chained ledger.
- Once set for `(conversationId, workspace)`, the daemon enforces strict immutability: any subsequent attempts to overwrite the baseline return the existing baseline.
- `client/hardtruth_hook.py` and `daemon/tier2_runner.py` query the isolated daemon first, then `refs/hardtruth/baseline/<hash>`, before falling back to local files. Malicious `rm -rf ~/.hardtruth/halts/` fails completely.

---

## Verification Status
All 47 tests passing cleanly in `tests/` across 4 test suites:
- `tests/test_hybrid_architecture.py` (18 tests)
- `tests/test_inverted_gate.py` (6 tests)
- `tests/test_daemon_ledger.py` (7 tests)
- `tests/test_live.py` (16 tests)

104 changes: 95 additions & 9 deletions client/hardtruth_client.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
"""
HardTruth Zero-Dependency Client Library
Can be embedded in any Python agent framework (LangGraph, CrewAI, AutoGen, or custom loops).
Automatically connects to HardTruth Daemon (http://127.0.0.1:8000) for sub-15ms NLI,
and gracefully falls back to deterministic AST & exit-code checking if daemon is offline.
Connects to HardTruth Daemon (http://127.0.0.1:8000) for sub-15ms NLI and tamper-evident ledger,
and gracefully falls back to deterministic AST & local DaemonLedger if daemon is offline.
"""

import os
import sys
import json
import urllib.request
import urllib.error
from typing import Optional, Dict, Any, List

_CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
if _CURRENT_DIR not in sys.path:
Expand All @@ -26,35 +26,121 @@
except ImportError:
from hardtruth.ast_checker import check_ast_stubs

try:
from daemon.ledger import DaemonLedger
except ImportError:
try:
from ledger import DaemonLedger
except ImportError:
DaemonLedger = None

CONTRADICTION_THRESHOLD = 0.70


class HardTruthClient:
def __init__(self, daemon_url: str = "http://127.0.0.1:8000", ledger_path: str = None):
self.daemon_url = daemon_url.rstrip("/")
self.ledger_path = ledger_path or os.environ.get(
"HARDTRUTH_LEDGER_PATH",
os.path.expanduser("~/.gemini/antigravity-cli/ledger.jsonl")
"HARDTRUTH_DAEMON_LEDGER",
os.path.expanduser("~/.hardtruth/daemon_ledger.jsonl")
)
self._local_ledger = DaemonLedger(ledger_path=self.ledger_path) if DaemonLedger else None

def is_daemon_online(self) -> bool:
"""Pings daemon /health endpoint to check availability."""
try:
req = urllib.request.Request(f"{self.daemon_url}/health", headers={"User-Agent": "HardTruth-Client"})
req = urllib.request.Request(
f"{self.daemon_url}/health",
headers={"User-Agent": "HardTruth-Client"}
)
with urllib.request.urlopen(req, timeout=1.0) as resp:
return resp.status == 200
except Exception:
return False

def verify_claim(self, premise: str, hypothesis: str) -> dict:
def record_ledger_entry(
self,
conversation_id: str,
step_idx: int,
tool: str,
target: str = "",
observed_exit_code: Optional[int] = None,
harness_status: Optional[str] = None,
error: Optional[str] = None,
stdout_tail: Optional[str] = None,
diff_stat: Optional[str] = None
) -> dict:
"""Records an execution event to daemon or local ledger fallback."""
payload = {
"conversationId": conversation_id,
"stepIdx": step_idx,
"tool": tool,
"target": target,
"observed_exit_code": observed_exit_code,
"harness_status": harness_status,
"error": error,
"stdout_tail": stdout_tail,
"diff_stat": diff_stat
}
try:
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
f"{self.daemon_url}/v1/ledger/record",
data=data,
headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=2.0) as resp:
return json.loads(resp.read().decode("utf-8"))
except Exception:
if self._local_ledger:
return self._local_ledger.record_entry(
conversation_id=conversation_id,
step_idx=step_idx,
tool=tool,
target=target,
observed_exit_code=observed_exit_code,
harness_status=harness_status,
error=error,
stdout_tail=stdout_tail,
diff_stat=diff_stat
)
return {"status": "error", "error": "daemon_unreachable"}

def get_ledger_premise(self, conversation_id: str) -> dict:
"""Fetches verified premise and physical state for conversation."""
try:
url = f"{self.daemon_url}/v1/ledger/premise?conversationId={urllib.parse.quote(conversation_id)}"
req = urllib.request.Request(url, headers={"User-Agent": "HardTruth-Client"})
with urllib.request.urlopen(req, timeout=2.0) as resp:
return json.loads(resp.read().decode("utf-8"))
except Exception:
if self._local_ledger:
return self._local_ledger.get_premise(conversation_id)
return {
"tampered": False,
"conversationId": conversation_id,
"premise": "Daemon unreachable and no local ledger found.",
"source_files_modified": 0,
"doc_files_modified": 0,
"modified_files": [],
"verification_commands_executed": 0,
"unresolved_failures": []
}

def verify_claim(self, premise: str, hypothesis: str, threshold: float = CONTRADICTION_THRESHOLD) -> Optional[dict]:
"""Evaluates (Premise, Hypothesis) via DeBERTa-v3 Cross-Encoder in ~10ms."""
payload = json.dumps({"premise": premise, "hypothesis": hypothesis}).encode("utf-8")
payload = json.dumps({
"premise": premise,
"hypothesis": hypothesis,
"threshold": threshold
}).encode("utf-8")
req = urllib.request.Request(
f"{self.daemon_url}/v1/verify-claim",
data=payload,
headers={"Content-Type": "application/json"}
)
try:
with urllib.request.urlopen(req, timeout=2.0) as resp:
with urllib.request.urlopen(req, timeout=3.0) as resp:
return json.loads(resp.read().decode("utf-8"))
except Exception:
return None
Expand Down
Loading
Loading