Summary
Standard keripy clients (keri.app.agenting.WitnessReceiptor → HTTPMessenger → streamCESRRequests) POST inbound CESR events to the root path (/) of the URL declared in the witness's /loc/scheme reply. keripy's LocationRecord.url is parsed for scheme + host + port only — any path component is ignored when building the HTTP client.
keri.host's API Gateway today routes only:
| Path |
Methods |
/ |
OPTIONS only |
/witness |
GET — status |
/witness/oobi/{id} |
GET — OOBI bundle (controller role) |
/witness/oobi/{id}/witness/{witness} |
GET — witness-role OOBI bundle (from issue #1) |
/witness/introduce |
GET |
/witness/process |
POST — inbound events |
/witness/query |
POST — queries |
So a wallet's POST to the URL declared by the OOBI (https://witness.keri.host/) has no matching route — the request hangs at the API Gateway until timeout. CloudWatch confirms zero invocations of KerihostWitnessStack-{process,query,oobi} for our test traffic.
Repro
After issue #1 fixed (witness-role OOBI now serves correct endpoint declarations), Locksmith successfully:
-
Resolved /oobi/BNbRf.../witness — ends table now has (BNbRf..., 'witness', BNbRf...) and locs has (BNbRf..., 'https') → https://witness.keri.host.
-
Created an inception event EAat5ezAlaTezaWi1Njx866poW-6_l5xXIUO5DTqvuG4 with wits=[BNbRf...], toad=1.
-
keri.app.agenting.WitnessReceiptor picked up the event, constructed an HTTPMessenger with url=https://witness.keri.host, and submitted the CESR bytes to streamCESRRequests(client, ims=..., dest=BNbRf...).
-
streamCESRRequests posted to / (the default path when path=None).
-
API Gateway found no POST / route and the request hung. AWS confirms zero Lambda invocations during the test window:
$ aws logs tail /aws/lambda/KerihostWitnessStack-process --since 5m
(empty)
$ aws logs tail /aws/lambda/KerihostWitnessStack-query --since 5m
(empty)
Direct repro at the HTTP layer:
$ curl -sS -w "HTTP %{http_code}\n" --max-time 5 -X POST "https://witness.keri.host/" \
-H "Content-Type: application/cesr" --data-binary "test"
HTTP 000 # (timeout — no response at all)
$ curl -sS -w "HTTP %{http_code}\n" --max-time 5 -X POST "https://witness.keri.host/process" \
-H "Content-Type: application/cesr" --data-binary "test"
HTTP 403 # (API Gateway "no route" — confirms /process is not a top-level route either)
Why this matters
Standard keripy wallets cannot publish events to keri.host today. The wallet's WitnessReceiptor will sit indefinitely in "Waiting for witness receipts for ..." with no error surfaced — because the HTTP layer just times out, and keripy's outer wait-loop doesn't have a meaningful timeout that propagates back up.
Three approaches considered
A. Add POST / that routes to KerihostWitnessStack-process
One API Gateway route change. Standard keripy wallets work immediately. Right answer for an unblocking patch.
Downside: process Lambda is now the catch-all for any POST. If a wallet sends a qry (query) message — also a valid CESR payload that some flows produce — it lands at process rather than query. Either process handles it (duplicate logic) or proxies to query (added latency + indirection inside the Lambda).
B. Declare URL with path in /loc/scheme
E.g., return "url":"https://witness.keri.host/process" from the OOBI. Doesn't work. keripy's LocationRecord.url is parsed by urllib.parse.urlparse and only the scheme/hostname/port are passed to hio.http.clienting.Client. The path component is dropped. Confirmed by reading keri.app.agenting.HTTPMessenger.__init__ and keri.app.agenting.streamCESRRequests.
C. CESR-typed dispatch at the root — recommended
Add POST / mapped to a small router Lambda (or routing logic in the existing process Lambda) that:
- Reads enough of the incoming CESR stream to identify the message type (
t field on the first Serder).
- Dispatches to the appropriate downstream:
icp, rot, ixn, dip, drt (establishment + interaction events) → process Lambda
qry (query messages) → query Lambda
rct (receipts) → process Lambda
exn (peer-to-peer exchange) → depending on route, an exchange handler or process
- Returns the downstream's response (or a CESR-encoded receipt stream when applicable).
This is the same pattern keria-aws uses (/agent/{proxy+} with internal dispatch). Production-grade benefits:
- Single entry surface — one place for auth, rate limiting, request size limits, DLQ semantics, structured logging.
- Observability — log message type before dispatch; CloudWatch metrics per message type.
- Evolvability — adding a new message type is a router-Lambda change, not an API Gateway change.
- Standard keripy compatibility — no client-side changes needed.
- The existing
/witness/process and /witness/query routes can remain for direct-dispatch testing/diagnostics.
Recommendation
Ship C. A is acceptable as a stopgap if C is more than ~a day of work, but the architecture should converge on C because that's what production-grade KERI-on-AWS deployments look like (keria-aws being the reference).
If implementing C in the existing KerihostWitnessStack-process Lambda (vs adding a dedicated router Lambda), the existing process handler can be the dispatcher itself — it already runs provided.al2023 (Rust/native), which makes parsing the CESR header for message-type-based dispatch cheap.
Environment
- Witness AID:
BNbRfMPQQge6wKO0uof9Y0e_QYOfiF08k9drc6pOgzjt
- Deployment:
https://witness.keri.host, region us-west-2
- API Gateway (REST v1):
KerihostApiStack-api (id: lbu6rznno5)
- Client tested: Locksmith on
dev branch carrying:
- keripy:
2.0.0.dev6
Summary
Standard keripy clients (
keri.app.agenting.WitnessReceiptor→HTTPMessenger→streamCESRRequests) POST inbound CESR events to the root path (/) of the URL declared in the witness's/loc/schemereply. keripy'sLocationRecord.urlis parsed for scheme + host + port only — any path component is ignored when building the HTTP client.keri.host's API Gateway today routes only:
//witness/witness/oobi/{id}/witness/oobi/{id}/witness/{witness}/witness/introduce/witness/process/witness/querySo a wallet's POST to the URL declared by the OOBI (
https://witness.keri.host/) has no matching route — the request hangs at the API Gateway until timeout. CloudWatch confirms zero invocations ofKerihostWitnessStack-{process,query,oobi}for our test traffic.Repro
After issue #1 fixed (witness-role OOBI now serves correct endpoint declarations), Locksmith successfully:
Resolved
/oobi/BNbRf.../witness—endstable now has(BNbRf..., 'witness', BNbRf...)andlocshas(BNbRf..., 'https') → https://witness.keri.host.Created an inception event
EAat5ezAlaTezaWi1Njx866poW-6_l5xXIUO5DTqvuG4withwits=[BNbRf...],toad=1.keri.app.agenting.WitnessReceiptorpicked up the event, constructed anHTTPMessengerwithurl=https://witness.keri.host, and submitted the CESR bytes tostreamCESRRequests(client, ims=..., dest=BNbRf...).streamCESRRequestsposted to/(the default path whenpath=None).API Gateway found no
POST /route and the request hung. AWS confirms zero Lambda invocations during the test window:Direct repro at the HTTP layer:
Why this matters
Standard keripy wallets cannot publish events to keri.host today. The wallet's
WitnessReceiptorwill sit indefinitely in "Waiting for witness receipts for ..." with no error surfaced — because the HTTP layer just times out, and keripy's outer wait-loop doesn't have a meaningful timeout that propagates back up.Three approaches considered
A. Add
POST /that routes toKerihostWitnessStack-processOne API Gateway route change. Standard keripy wallets work immediately. Right answer for an unblocking patch.
Downside: process Lambda is now the catch-all for any POST. If a wallet sends a
qry(query) message — also a valid CESR payload that some flows produce — it lands atprocessrather thanquery. Eitherprocesshandles it (duplicate logic) or proxies toquery(added latency + indirection inside the Lambda).B. Declare URL with path in
/loc/schemeE.g., return
"url":"https://witness.keri.host/process"from the OOBI. Doesn't work. keripy'sLocationRecord.urlis parsed byurllib.parse.urlparseand only thescheme/hostname/portare passed tohio.http.clienting.Client. The path component is dropped. Confirmed by readingkeri.app.agenting.HTTPMessenger.__init__andkeri.app.agenting.streamCESRRequests.C. CESR-typed dispatch at the root — recommended
Add
POST /mapped to a small router Lambda (or routing logic in the existing process Lambda) that:tfield on the firstSerder).icp,rot,ixn,dip,drt(establishment + interaction events) →processLambdaqry(query messages) →queryLambdarct(receipts) →processLambdaexn(peer-to-peer exchange) → depending on route, an exchange handler orprocessThis is the same pattern keria-aws uses (
/agent/{proxy+}with internal dispatch). Production-grade benefits:/witness/processand/witness/queryroutes can remain for direct-dispatch testing/diagnostics.Recommendation
Ship C. A is acceptable as a stopgap if C is more than ~a day of work, but the architecture should converge on C because that's what production-grade KERI-on-AWS deployments look like (keria-aws being the reference).
If implementing C in the existing
KerihostWitnessStack-processLambda (vs adding a dedicated router Lambda), the existing process handler can be the dispatcher itself — it already runsprovided.al2023(Rust/native), which makes parsing the CESR header for message-type-based dispatch cheap.Environment
BNbRfMPQQge6wKO0uof9Y0e_QYOfiF08k9drc6pOgzjthttps://witness.keri.host, regionus-west-2KerihostApiStack-api(id:lbu6rznno5)devbranch carrying:_resize_to_contentcrash fix2.0.0.dev6