AsAt

Point-in-time retrieval over Australian tax law, where a citation that cannot be verified is not returned.

Ask "was this deductible in FY2020?" and most RAG systems answer from whatever the law says today. AsAt answers from what the law said then, and refuses to answer at all rather than cite something it cannot substantiate.

Two properties drive the whole design:

Temporal correctness is structural, not a ranking hint. A provision carries three dates, not one. ITAA97 s 8-1 sits in a compilation dated 2026-07-01, but the provision itself commenced 1997-07-01. Gating on the compilation date asserts that s 8-1 did not exist before 2026 — which is how a retrieval system confidently answers a FY2020 question from FY2027 text. AsAt derives an income_years set from commencement (not compilation) and applies it as a hard must pre-filter in Qdrant. Out-of-period sources are structurally unreachable; they are never scored and then rejected.

A quote is checked against the source, byte for byte. Every claim the model emits must name a source from the retrieved set, and its quote must be a literal substring of that source's text after NFKC and U+2011 normalisation. Not "similar to". Not "entailed by". A substring. Quotes shorter than 25 characters or 4 words are rejected as degenerate, because "a" substring-matches almost anything. The check costs no model call and no network round trip, and it is the reason the system can abstain honestly instead of paraphrasing plausibly.

Abstention is a first-class outcome, decided outside the LLM. Small models do not abstain unprompted, so the decision is deterministic and gated on reranker score, evidence count, groundedness and disagreement — never on plain cosine similarity.

---

Quick start

What you need

	With an API key (recommended)	Fully local AI
Operating system	64-bit Windows 10/11 (Docker Desktop needs WSL 2), macOS, or Linux	same
Software	Docker with the Compose v2 plugin — the installer checks and points you at instructions if missing	same
Memory	roughly 8 GB RAM free for the containers	roughly 16 GB RAM
Disk	roughly 20 GB free (container images plus the search index)	roughly 30 GB free (adds the local models)
GPU	none	NVIDIA GPU with 8 GB+ VRAM
Internet	for install and for each AI call	for install only; queries stay on your machine

The RAM and disk figures are sizing guidance, not measurements — actual use depends on how many documents you load. The GPU line is measured: the reference machine is an 8 GB RTX 3060 Ti, and on CPU alone local generation has never completed a single answer (details under "Why BYOK rather than local Ollama" below).

One line, if Docker is already installed (the script checks and tells you exactly what to do if it is not):

Linux / macOS:

curl -fsSL https://raw.githubusercontent.com/Aldiharley/asat/master/install.sh | bash

Windows (PowerShell):

irm https://raw.githubusercontent.com/Aldiharley/asat/master/install.ps1 | iex

Either one downloads AsAt, offers to take an API key (paste one for fast answers, or press Enter to run fully local), starts everything, waits for it to come up, and tells you where to go next. Re-running the same command updates and restarts. Your API key lives in .env and survives updates.

Or by hand — bring an API key; see the note on local generation below:

cp .env.example .env      # then paste your key into ASAT_GENERATION_API_KEY
docker compose -f docker-compose.dev.yml up

Then open http://localhost:3000 and go to Research in the sidebar.

That single command pulls the model, builds the index from the committed fixtures, runs the database migrations, and starts every service in dependency order — nothing serves queries against an empty index. First run takes a few minutes; later runs are seconds.

.env.example documents every accepted provider value, each read from services/research/src/asat_research/composition.py rather than guessed. A bare copy leaves the provider unset, which means fully local Ollama — the only configuration that works without you supplying anything. Uncomment one block to opt into BYOK.

Why BYOK rather than local Ollama

The stack runs fully locally with ASAT_GENERATION_PROVIDER=ollama and no key at all. On CPU, that path does not finish. Measured on a 16-vCPU VM: qwen2.5-coder:7b processes prompt at ~14 tokens/sec, so a 2,050-token prompt spends 143 seconds before emitting its first output token — and two_pass_generate makes two model calls per query. Across this project's entire audit-log history the local path has produced zero completed answers: four horizon refusals, five timeouts, no successes.

The design targets a single 16 GB GPU, where this is not a problem. Without one, use a key.

If you do run local, two Ollama settings on the HOST server roughly halve KV memory and measurably cut latency on small cards (measured on the 8 GB reference GPU: a warm research query fell from ~22 s to ~11 s, and a vision extraction from ~40 s to ~19 s):

OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve

Any inference-config change alters model numerics — eval runs before and after it are not comparable, per the reproducibility section below.

Four adapters cover the field. Anything speaking the OpenAI Chat Completions shape goes through one of them and differs only in base URL and model name, so there is no per-vendor code:

ASAT_GENERATION_PROVIDER	Covers
openai_compatible	DeepSeek, Kimi (Moonshot), MiniMax, OpenAI, Mistral, xAI, Groq, vLLM, LM Studio, LiteLLM
aggregator	OpenRouter and work-alikes — one key, ~200+ models
anthropic	Claude (a forced tool call, not response_format)
ollama	fully local, no key, no egress

.env.example carries a ready-to-uncomment block with the base URL and a model for each. Your key goes in .env, which is gitignored — never in .env.example, never in a compose file, never in a commit.

You are sending your queries and retrieved sources to whichever provider you configure. That is your call to make, and the system does not second-guess it — it does record which provider and model answered, in the audit row and the response footer, because two answers from different models are two different answers.

Check your provider once, after configuring it:

docker compose -f docker-compose.dev.yml run --rm research \
  python3 -m asat_research.provider_probe

This matters more than it sounds. The citation source field is a JSON-schema enum over the retrieved passage ids — that enum is what stops a model citing something it was never shown. Providers vary in whether they enforce it, and they fail silently in both directions: an aggregator may fall back to json_object and drop the enum entirely; Anthropic may return an enum value with different capitalisation and complete normally. Both look like success. The probe asks for a source that was never offered, so a provider under a real constraint cannot return it and one ignoring the enum will.

A provider that fails the probe is still usable. The verifier — not the schema — is the safety property, so a weak schema costs recall, not correctness. Knowing simply means a higher abstention rate reads as a configuration fact rather than a quality problem.

Running the gold-set gate

docker compose -f docker-compose.dev.yml run --rm research \
  python3 -m asat_research.eval_cli --limit 5

It exits non-zero unless referential integrity is exactly 1.0 — including when the rate cannot be measured at all, because a gate that cannot measure itself must not report a pass.

---

What actually runs today

This section is deliberately blunt: the internal design spec describes a larger system than this tree implements.

Real:

Component	Status
Bitemporal filtering	Real. 4,908 chunks across legislation, rulings and guidance.
Embedder	Real. qwen3-embedding:0.6b via Ollama, 1024-dim, vectors cached on disk.
Reranker	Real. Cosine against the pipeline's own embedder. A bi-encoder, not a cross-encoder — see below.
Abstention thresholds	Real. Measured against the reranker's own score distribution, not inherited.
Hybrid retrieval	Real. Dense + sparse BM25 named vectors in Qdrant, RRF fusion (k=60).
Citation integrity	Real. Deterministic, no model call, hard fail.
Abstention gate	Real. Pure and fully unit-tested.
Groundedness (HHEM-2.1-open)	Real when ASAT_HHEM_MODEL=1, which compose sets.
Audit log	Real, append-only Postgres, self-migrating.
Generation	Real. BYOK across four adapters, or local Ollama.
Row-level security	Real. Postgres RLS on app_rw; the shell refuses to boot as a role RLS does not apply to.
Gold suite	Real. 250 cases in gold-set/pit_suite.yaml.
Research surfaces	Real, two of them: /research (form, transcript of turns) and /chat (chat-shaped, conversation history, numbered sources rail). Same pipeline, same SSE stage streaming; every message answered independently, and both surfaces say so.
Xero connection	Real, BYO-app: you register your own free app at developer.xero.com and connect one organisation per client (Settings → Connections). Extracted transactions push to Xero as draft purchase bills with the source document attached. See below.
HubSpot CRM import	Real, BYO-token: paste a Service Key (or legacy private-app token) from your own HubSpot account and import companies as clients — no OAuth, no app review, free tier included. Read-only; existing client records are never overwritten. See below.

Stand-ins — no production implementation exists in this tree:

Component	Stand-in	Consequence
Bitemporal store	InMemoryBitemporalStore, hydrated from committed fixtures	PostgresBitemporalStore exists and is tested, but nothing ingests into it, so it would serve an empty corpus.

HashEmbedder and FakeReranker still exist and are still the right choice for the test suite — ASAT_EMBEDDER=hash keeps it offline and deterministic — but they are no longer what the service runs.

Connecting Xero

Self-hosted means there is no central server to hold a shared Xero app secret, so AsAt uses the same bring-your-own model as its LLM keys: you create your own free Xero app and AsAt talks to Xero as your app. Nothing passes through anyone else's infrastructure.

1. Create an app at developer.xero.com (New app → integration type Web app; any private name works, though Xero rejects names containing its product names).
2. In the app's Configuration, add the redirect URI shown on AsAt's Settings → Connections page — http://localhost:3000/api/xero/callback by default. It must say localhost; Xero rejects 127.0.0.1. Serving AsAt from a domain instead? Set ASAT_APP_URL in .env and use that URL.
3. Copy the Client ID, generate a Client Secret, paste both into the Connections page, save, and click Connect to Xero. Authorise exactly one organisation — one Xero org maps to one AsAt client.

Then the Transactions page grows a Push to Xero button: each extracted transaction becomes a draft purchase bill (supplier contact found or created by name, amounts sent GST-inclusive, original PDF or image attached). Drafts stay uncoded on purpose — account codes and tax treatment are the accountant's call, made in Xero, not this system's guess. Rows already pushed are skipped, so the button is safe to press twice.

Worth knowing up front: Xero's free app tier allows 5 connected organisations (more needs their Core plan at $35 AUD/month on your developer account); each organisation accepts at most two uncertified apps; and a connection unused for 60 days must be re-authorised — AsAt refreshes tokens in the background to push that horizon out, and the Connections page shows when a reconnect is needed. Credentials and tokens are stored AES-256-GCM encrypted in your own database, keyed from BETTER_AUTH_SECRET — so set that to a real value and don't lose it: rotating it invalidates stored connections (they reappear as "reconnect required", nothing worse).

QuickBooks Online is deliberately absent for now: Intuit gates production API keys behind a per-app review and bans localhost redirect URIs, which breaks the self-hosted BYO model this integration depends on.

Connecting your CRM (HubSpot)

If your practice already keeps its client list in HubSpot, you don't type it twice. This one is even simpler than Xero — no OAuth at all:

1. In HubSpot: Settings → Development → Keys → Service keys → Create service key (on older accounts, a legacy private app from Development → Legacy apps works identically). Works on every tier including free CRM; there is no app review.
2. Grant it crm.objects.companies.read and crm.schemas.companies.read.
3. Paste the token (starts with pat-) into Settings → Connections. It is verified against HubSpot before being stored encrypted, and the page tells you immediately if a scope is missing.

Import clients then brings each HubSpot company in as a client. The import follows the same never-invent discipline as everything else here: the company name is the only field taken as-is; if your HubSpot has an ABN-ish custom property, its value is carried over only when it passes the real ABN checksum (a value that fails imports as blank, never as a wrong ABN); entity type, GST registration and BAS cycle stay unset for you to fill in, because a CRM does not know them and guessed defaults in tax records are worse than blanks. Existing clients are matched by name and skipped — an import never overwrites a hand-maintained record. Re-running is safe.

Disconnecting removes the token from your database; the key itself stays valid in HubSpot until you rotate or delete it there.

The reranker is a bi-encoder, not the bge-reranker-v2-m3 cross-encoder the spec names. A cross-encoder reads the query and passage together and is materially better at this judgement. The bi-encoder's real contribution is calibration: RRF fusion discards raw scores, and the abstention gate needs a number on a known scale.

How the thresholds were set

Measured over 25 live gold cases against the real index:

	median	p05	p95
Gold-matching candidates (n=25)	0.7121	0.5301	0.8103
Everything else retrieved (n=11,339)	0.3322	0.1616	0.4781

The distributions separate cleanly — negative p95 (0.4781) sits below the lowest per-case best-gold score (0.5301) — so the floor is set at 0.50: above 95% of irrelevant material, below every answerable case measured. The corroboration bar is the negative p95 verbatim, because "beats 95% of what retrieval surfaced and rejected" is the meaning and a rounder number would not have one.

The previous values were 0.35 / 0.30. Those are bge-reranker-v2-m3 numbers, and the measured negative median on this scale is 0.3322 — so the old floor sat inside the noise and the gate could not meaningfully fire. scripts/calibrate_rerank_thresholds.py reproduces the measurement; tests assert the constants still sit inside the measured window.

---

Measured results

The 249 live gold-set cases have been scored against the real pipeline three times. Every number below came out of a run; none is a target. The current numbers are from expanded03, the first run produced under a fixed sampling seed — see the reproducibility section below for why the first two runs could not be compared with each other, and why that matters more than any single number here.

The run

Suite	gold-set/pit_suite.yaml — 250 cases, 249 live
Generation	qwen2.5:7b via local Ollama, one RTX 3060 Ti (8 GB), fixed seed
Embedder / reranker	qwen3-embedding:0.6b, bi-encoder cosine
--as-of	2027-06-30 — the corpus horizon, not today, so the run is reproducible
corpus_generation	3cb95462f8a6cf81 — CGT, Division 7A and superannuation included
Corpus	4,712 chunks: 4,063 legislation / 549 rulings / 100 guidance (the corpus has since grown to 4,908 with the R&D Tax Incentive; the numbers below measure the generation shown)
Wall clock	114.0 minutes

docker compose -f docker-compose.dev.yml up -d
docker compose -f docker-compose.dev.yml exec research \
  python3 -m asat_research.eval_cli --out /tmp/eval.json

Reproducibility: the finding that reframes every other number

The gate was run twice before (f779618963c46363; then 2b03e19ff9db3097 after the CGT and Division 7A expansion), and comparing those runs showed 59 of 249 cases changing decision, net −10 — which read as the corpus expansion making things worse. It had not. Generation was unseeded: temperature 0.1 with no seed sent, so Ollama drew a fresh seed per call, and every eval run this project had ever recorded was sampling from a different point.

Measured rather than asserted — two 30-case runs on the identical corpus and configuration disagree on 13–23% of decisions (three pairwise estimates: 13%, 17%, 23%; the runs are committed as eval-noise-floor-A.json and -B.json). Every flip is answer ↔ abstain_ungrounded, in both directions: the model quotes differently on each run, and the integrity gate scores what it is given. The 24% churn between the first two runs is indistinguishable from that floor, so the effect of the CGT + Division 7A expansion was never measurable — in either direction.

generation.py now sends a fixed seed. Two seeded runs disagree on 1 case in 15 (~7%; committed as eval-seeded-determinism-A.json and -B.json) — a ~2.5× cut in variance, not bit-exactness. The residual is GPU inference nondeterminism: a sampling seed pins which draw is made from the distribution, not the distribution itself, and parallel-reduction float error can flip a near-tie token at temperature 0.1. Evaluating at temperature 0 would have measured a system production does not serve, so the operating point stays and the residual is stated instead.

The reading discipline this buys, applied throughout this section: per-case deltas under roughly 7% of cases are weather. Run-to-run comparison is done case by case with python -m asat_research.eval_compare, never by eyeballing aggregate rates.

What it decided

Decision	Count	Share
abstain_ungrounded	188	75.5%
answer	49	19.7%
abstain_thin_evidence	10	4.0%
error	2	0.8%

The metrics

Metric	Value
referential_integrity_rate	1.0000 — see the circularity caveat below
crag_truthfulness	+0.1245
hallucination_rate	0.0361
abstention_rate_on_answerable	0.7871
false_premise_rejection_rate	0.8889
generation_attempted_bad_citation_count	133
Errored cases	2 (every rate above is over n=249 including these)

Of the 49 answers: 40 accurate, 9 citing the wrong authority. Overall behaviour_matched was 57/249. Against expanded02 the case-level diff reads 25 fixed, 21 broken, net +4 — but expanded02 was unseeded, so that comparison still carries the old noise and the honest summary is: no measurable regression from tripling the corpus, and no measurable gain either. The next corpus change will be the first with a clean before/after.

Reading these numbers honestly

It abstains on roughly three quarters of answerable questions, and the audit log says why. Analysed over a full run's abstain_ungrounded responses (cross-checked against the system's own recorded warnings, 101/101 agreement):

• ~72% failed citation integrity — the model produced a quote that was not a literal substring of the source it cited. The 7b model paraphrases instead of quoting; the gate catches every instance. This is the system working, at the price of abstaining.
• ~28% passed integrity on every quote and failed groundedness — the quotes were real and the model's characterisation of them was not supported.

The response now carries that work instead of discarding it: 78.6% of failing responses contained at least one fully verified quote, and verified_quotes plus the ranked retrieved list turn an abstention into a reading list — for a registered agent, arguably a more defensible output than a borderline answer.

The corpus is no longer the dominant limit; the model is. At the first run, the operative statute was 72 chunks across 12 sections and an unknown share of abstention was retrieval having nothing to find. The served corpus now holds 1,410 operative-provision chunks across 270 sections (ITAA97 585 including the CGT Parts and Divisions 290–307, SIS Act 405, ITAA36 267 including Division 7A). The abstention rate barely moved. What remains is the measured fact that a 7-billion-parameter model quotes statute verbatim about a quarter of the time — and every failure is caught, which is the design working as specified with a model at the floor of what the design needs.

The integrity rate is partially circular, and should be read as such

referential_integrity_rate: 1.0000 means every gold_chunk_id resolves against the served corpus. For the original 15 gold provisions this is circular by construction — keep_sections bounded the ingest to exactly what the gold set names. It is now only partially circular: the CGT, Division 7A and superannuation expansions selected sections from the Register's own tables of contents on subject-matter grounds, not from gold ids. The rate still is not evidence of adequacy; it catches drift, nothing more.

What is in the corpus now, and what still is not

Moved in since the first run:

CGT	Parts 3-1 and 3-3 operative spine — 45 sections including all of Subdiv 152 (small business), Div 115 (discount), Subdiv 118-B (main residence), Div 128 (death)
Division 7A	ITAA36 ss 109B–109ZE complete, plus s 100A, s 318 ("associate") and s 6 definitions
Superannuation	ITAA97 Divs 290–307 (115 sections: contributions, caps, transfer balance, fund tax incl. s 295-550 NALI, benefits) and SIS Act 1993 (43 sections: SMSF definition, sole purpose, ss 62–85 investment rules, s 109 arm's length, s 166 penalties)
R&D Tax Incentive	ITAA97 Div 355 operative spine — 48 sections (activity definitions incl. the June 2026 gambling/tobacco exclusion, s 355-100 offset entitlement with every threshold, notional deductions, clawback and feedstock, partnerships, IISA findings) plus s 67-30 (refundability) and the aggregated-turnover cluster (328-110–130). Added after the published run below; commencement floor FY2012, sourced from the compilation's own transitional note

Still absent, honestly:

	Why	Parked at
SIS Regulations	Regulation numbering (6.01) defeats the section parser; reg citations unbuilt. Conditions of release and minimum pension factors live there	recorded in the internal deferred-work register
Section-level commencement	Provisions are retrievable from their Act's commencement, not their own — Div 294 reads as available from FY1998	recorded in the internal deferred-work register
Div 855, TAA53 Sch 1 Subdiv 14-D	CGT non-resident machinery, not yet acquired	—
FBTAA 1986	Absent, though the suite asks an FBT question	—

The must_not_cite traps

One structurally inert pattern (TR2022D2 — nothing in the corpus can match it, so it passes for free). The behavioural caveat from earlier runs stands: a trap that only fires on a citation cannot fire on an abstention, and most trap-carrying cases abstain.

Where it is genuinely wrong

hallucination_rate is 0.0361 — 9 of 49 answers cited the wrong authority. As before, every one of those quotes was a literal substring of the chunk it cited; what failed is which chunk was cited. Citation integrity is not citation relevance, and this project's safety property only ever claimed the first. Closing the gap needs a relevance gate the reranker score is currently too coarse to provide.

Two errored cases. One is AUTAX-0124, where the model emits degenerate JSON on a rate-table question — a model pathology deliberately not papered over with a retry, which would trade reproducibility for cosmetics.

One defect these numbers depend on

An earlier attempt abstained on every case. The cause was not the model. reconstruct_parents expanded each selected parent to all its children with no cap: the ITAA97 Dictionary (s 995-1, 1,955 children against a median of 3) got selected and dragged every child in — 2,425 context chunks in a single query, and a 2,425-value source enum the model answered by crediting verbatim text to the wrong sources. Capped at 16 children per parent, windowed around the chunks that scored; 16 leaves 91.6% of parents untouched. The Dictionary is still 1,955 of the 4,063 legislation chunks served.

What is not measured

• Extraction accuracy. The 200-document AU test set does not exist; a synthetic-case harness (services/extraction/tests/accuracy/) provides a regression tripwire, not a field-accuracy claim.
• Any cloud provider. These numbers are local Ollama only. The Anthropic adapter cannot even be made run-reproducible — the Messages API has no seed parameter, and the adapter says so rather than inventing a field.
• Inter-rater agreement on the gold set — needs a second qualified rater.

---

How it fits together

At runtime

flowchart LR
    U([Tax agent]) --> SH["shell · Next.js<br/>:3000"]
    SH --> EX["extraction<br/>:8000"]
    SH --> RS["research · FastAPI<br/>:8001"]

    RS --> QD[("Qdrant<br/>dense + sparse")]
    RS --> GEN{{"generation<br/>Ollama or BYOK"}}
    RS --> VF["verifier<br/>:8003"]
    RS --> PG[("Postgres<br/>research.audit_log")]
    SH --> PG2[("Postgres<br/>app schema · RLS")]

    VF -. "quote must be a literal<br/>substring of its source" .-> RS

    classDef svc fill:#e7ecfb,stroke:#2f5fd0,color:#12161f
    classDef store fill:#e0f4ec,stroke:#12a06a,color:#12161f
    classDef ext fill:#fbf0dd,stroke:#a86a12,color:#12161f
    class SH,RS,EX,VF svc
    class QD,PG,PG2 store
    class GEN ext

Loading

A query runs as one pass through seven stages, in this order:

flowchart LR
    A["resolve<br/>income year"] --> B["hard temporal<br/>pre-filter"]
    B --> C["hybrid retrieval<br/>RRF k=60"]
    C --> D["rerank<br/>+ parent merge"]
    D --> E["two-pass<br/>generation"]
    E --> F["verify<br/>integrity + HHEM"]
    F --> G["abstain<br/>or answer"]
    G --> H["provenance<br/>+ audit"]

    classDef pure fill:#e0f4ec,stroke:#12a06a,color:#12161f
    classDef io fill:#e7ecfb,stroke:#2f5fd0,color:#12161f
    class A,D,G pure
    class B,C,E,F,H io

Loading

Green stages are pure functions — no network, no database, no clock. That is where every safety rule lives, which is why they are unit-testable with no infrastructure at all.

Package layering

flowchart RL
    RS["asat_research<br/><small>retrieval · generation · gate</small>"] -->|"27 imports ✓"| CO["asat_corpus<br/><small>0 upward imports</small>"]
    VF["asat_verifier<br/><small>integrity · groundedness</small>"] -->|"4 imports ✗"| RS

    classDef ok fill:#e0f4ec,stroke:#12a06a,color:#12161f
    classDef bad fill:#fbf0dd,stroke:#a86a12,color:#12161f,stroke-dasharray: 5 3
    class CO,RS ok
    class VF bad

Loading

asat_corpus is a clean foundation: it imports nothing from the layers above it. asat_research depends on it in the correct direction.

asat_verifier does not, and that is a known defect. It imports Claim and compute_income_years from asat_research, while research calls the verifier over HTTP — so the Python dependency and the runtime call point in opposite directions. The verifier exists to be an independent check that a quote is a literal substring of its cited source; sharing a definition with the thing it checks undercuts that. Moving those two symbols down into asat_corpus fixes it.

Every collaborator is a port

Eight typing.Protocol seams, each with a real implementation and a deterministic fake:

Port	Real	Stand-in
Embedder	OllamaEmbedder · qwen3-embedding:0.6b	HashEmbedder
Reranker	EmbeddingReranker	FakeReranker
GenerationClient	Ollama · OpenAI-compatible · Anthropic · aggregator	FakeGenerationClient
VerifierClient	HttpVerifierClient	FakeVerifierClient
BitemporalStore	PostgresBitemporalStore — nothing ingests into it yet	InMemoryBitemporalStore
AuditStore	PostgresAuditStore	InMemoryAuditStore
GroundednessScorer	HHEM-2.1-open	word overlap
Provider (extraction)	vision model	fake

composition.py is the only place any of these is constructed. It reads the environment and raises on an unknown value rather than falling back silently, so two callers can never disagree about what is running. Swapping the embedder and the reranker for real models changed no calling code.

The index is deliberately ephemeral

No Qdrant volume. The index rebuilds from committed fixtures on every docker compose up, which is a stronger guarantee than persistence: version-independent, diffable in git, and incapable of silently migrating into a state a newer Qdrant refuses to read.

That rebuild costs a 20+ minute CPU embed of 4,908 chunks with a real embedding model. So the vectors — not the index — are cached, keyed by sha256(embedder identity + chunk text), on a Docker volume shared by ingest and research. First run pays the cost; the second took 26 seconds.

The embedder's identity is in that cache key deliberately. Without it, changing models would return vectors computed by the previous one — a cache hit from a different vector space, which is exactly the failure corpus_generation exists to catch, reintroduced one layer below where the stamp can see it.

The stamp that makes staleness visible

Every index carries a corpus_generation — a sha256 over chunk ids, content hashes, payloads and the vector configuration, including which embedder produced it. The pipeline compares it on every query, so an index built from superseded text or by a different embedder is reported rather than trusted.

Vector values are excluded on purpose: float reproducibility does not hold across real models, so including them would make the stamp fail for reasons that are not staleness.

Layout

services/corpus/           parsing, chunking, bitemporal model, embedders, gold suite
services/research/         retrieval, temporal, generation, abstention, audit, composition
services/verifier/         citation integrity + groundedness (HHEM-2.1-open)
services/extraction/       document field extraction
services/document-intake/  Next.js shell — upload, review, and the research UI
gold-set/                  250-case point-in-time evaluation suite
scripts/                   threshold calibration and other one-off measurement tools

Tests: pytest per Python service; vitest in services/document-intake. All six suites run on every push (.github/workflows/ci.yml).

Suite
services/corpus	186 passed, 3 skipped
services/research	328 passed, 2 skipped
services/analysis	203 passed
services/verifier	35 passed, 1 skipped
services/extraction	31 passed
services/document-intake (vitest)	76 passed, 0 skipped

859 tests. The vitest line is the one to read carefully: 0 skipped is the assertion, not 76. Five models/*.rls.test.ts suites fail in beforeAll without a database, and vitest reports the tests under a failed beforeAll as skipped — so a completely dead security suite reports zero failures. If you see skips there, treat the run as failing and give it a Postgres.

Known structural weaknesses

Recorded rather than hidden — a reviewer will find these anyway:

Issue	Detail
_run_research_query still covers most of the pipeline	Retrieval is now _retrieve_context; generation, verification, abstention and provenance are still inline. The file itself grew — the extraction moved code out of the function and added a docstring explaining the two distinct routes to "no basis".
ingest.py is 834 lines	And lives in asat_research though it is a corpus concern. Moving it is not a file rename: it imports chunk_to_point, point_payload and SPARSE_BUCKETS from asat_research.qdrant_hybrid_search, so the point-building half of that module has to move with it or the layering inverts. python -m asat_research.ingest is also the command in docker-compose.dev.yml, the CI workflow and the Dockerfile comments.
Pipeline has 17 fields	Adding a capability touches the dataclass, composition and 8 construction sites.

---

Corpus and attribution

Primary Australian sources only: the Federal Register of Legislation (CC BY 4.0, commercial reuse express), ATO public rulings and guidance (a bare permission, not Creative Commons), and the MLEB Australian tax guidance set (CC BY 4.0).

Attribution is required in-product, in this prescribed wording, emitted at runtime by asat_research.provenance:

Based on content from the Federal Register of Legislation at [date of download]. For the latest information on Australian Government legislation please go to https://www.legislation.gov.au.

Some material is deliberately excluded, and the ingest fails loudly rather than accepting an aggregator's blanket licence claim: Fair Work Ombudsman rows (CC BY-NC), a 1989 Parliamentary Library Bills Digest (licence undetermined, which is not the same as licence bad), and state revenue offices (only VIC and SA permit commercial incorporation, so au_state_revenue is empty by design). Commercial legal publishers and AustLII are not in the corpus and cannot be.

Full detail, including the exclusions and their reasons, is in DATA-LICENCES.md.

---

Status

Working research prototype, not a product. The gold set has been scored end to end — see Measured results. Headline: it answers 23.7% of 249 cases, abstains on 74.3% of answerable ones because a 7b model will not quote verbatim reliably, and cites the wrong authority 4.0% of the time.

The embedder and reranker are no longer stand-ins — qwen3-embedding:0.6b does both, and the abstention thresholds were measured against the reranker's own score distribution rather than inherited. What remains true is that the reranker is a bi-encoder, not the bge-reranker-v2-m3 cross-encoder the spec names, so retrieval quality is the weakest link rather than the safety machinery.

Treat the numbers in the internal design spec as targets, not measurements. The measured numbers in this README are marked as measured and say what they were measured over.

Extraction accuracy is unmeasured against real documents. The plan set a target of ≥ 92% field-level accuracy on a 200-document Australian test set (ABN and GST amount ≥ 97%). That test set does not exist and no accuracy number has been produced, so the target is quoted here only as a target. See the internal deferred-work register.

Eight further items are not built, and are recorded rather than dropped in an internal deferred-work register — each with what is missing, why it was parked, the source and licence position, and the concrete next action. Most are blocked on acquiring a number from a primary source, because this project does not ship a rate, threshold or coefficient it has not verified.

Nothing here is tax advice, and the design assumes a registered tax agent reviews every output.

Licence

AGPL-3.0-only — see LICENSE — covering the source code. Copyright (c) 2026 Dennis Liu.

The choice is deliberate, and the network clause is the point: this is a server-side product, and AGPL is the licence under which nobody can run a modified copy as a closed service without publishing their modifications. Self-hosting is unencumbered — running AsAt for yourself or your own practice triggers nothing; the licence binds those who modify it and serve it to others. The sole copyright holder can offer separate commercial licences; contributions are accepted under the Developer Certificate of Origin (DCO) to keep that path clean — see CONTRIBUTING.md.

The licence does not extend to the committed corpus material, which cannot be sublicensed — not as MIT before, and not as AGPL now: CC BY 4.0 is offered by the licensor to each recipient directly, and the ATO material is a bare permission with a no-endorsement condition. DATA-LICENCES.md records every source, its terms, its required attribution, and what was excluded and why.

The name AsAt and the project's branding are not licensed. The AGPL grant covers the software; it does not licence a fork to present itself as this project — the same reasoning ATTRIBUTION.md applies to the upstream project's mascot, pointed at ourselves.