Database support

Mere ships pure-Mere PostgreSQL, MySQL, and Redis clients. All three wire protocols, their auth flows (SCRAM-SHA-256 for PG, mysql_native_password and caching_sha2_password for MySQL, AUTH for Redis), and everything on top are implemented in Mere itself; the host only exposes low-level TCP and crypto primitives. No npm packages involved.

The rest of this page walks the stack top-down, then catalogs the 32 examples/db_*.mere demos.

At a glance

Engine	Wire	Auth	Features
PostgreSQL	v3	trust, SCRAM-SHA-256	prepared / tx+savepoints / COPY / LISTEN-NOTIFY (async queue) / pool / TLS+verify+SAN
MySQL	v10	`mysql_native_password`, `caching_sha2_password` (fast + RSA)	prepared / tx+savepoints / pool / TLS+verify+SAN
Redis	RESP2 + RESP3 (9 types)	`AUTH`, `HELLO 3`	pipelining / PUB-SUB / binary-safe I/O / TLS+verify+SAN / Sentinel / Cluster (CRC16 + MOVED)

Contents

Layered architecture — 5-layer diagram
Quick start — docker + build + run
Public API reference — grouped tables per concern
Demo catalog — 32 demos grouped by engine + concern
Limitations and future work

Layered architecture

┌────────────────────────────────────────────────────────────────────┐
│  Mere application code                                             │
│  (e.g. examples/http_todo_pg.mere)                                 │
├────────────────────────────────────────────────────────────────────┤
│  contrib/db/pg.mere            contrib/db/mysql.mere               │
│  ─────────────────             ─────────────────────               │
│  Wire protocol (v3)            Wire protocol (v10)                 │
│  SCRAM-SHA-256                 mysql_native_password (SHA-1)       │
│  Simple + extended query       caching_sha2_password (SHA-256 +    │
│  Prepared statements            RSA-OAEP-SHA1)                     │
│  Transactions + savepoints     Simple query (COM_QUERY)            │
│  COPY FROM/TO STDOUT           Prepared statements (COM_STMT_*)    │
│  LISTEN / NOTIFY + async queue Text + binary row decode            │
│  URL parser (RFC 3986)         URL parser (mysql://)               │
│                                                                    │
│  contrib/db/pg_pool.mere       contrib/db/mysql_pool.mere          │
│  ─────────────────────         ────────────────────────            │
│  Keep-alive pool               Keep-alive pool                     │
│  Idle event pump                                                   │
│                                                                    │
│  contrib/db/redis.mere                                             │
│  ─────────────────────                                             │
│  Full RESP2 + RESP3 (Null / Double / Bool / Map / Set / BigNum /   │
│                       Verbatim / Push / Attribute)                 │
│  redis_command + typed extractors + AUTH + HELLO 3                 │
│  PUB/SUB (SUBSCRIBE / PSUBSCRIBE / redis_wait_message)             │
│  Pipelining (N commands / N replies in one round trip)             │
│  TLS (redis_connect_ssl / _verify — direct handshake, no in-band)  │
│                                                                    │
│  contrib/db/redis_sentinel.mere                                    │
│  ─────────────────────────                                         │
│  Sentinel master resolution + list-masters                         │
│                                                                    │
│  contrib/db/redis_cluster.mere                                     │
│  ────────────────────────                                          │
│  CRC16-XMODEM slot hash + hash-tag extraction                      │
│  CLUSTER SLOTS bootstrap, MOVED redirection, per-node fd cache     │
├────────────────────────────────────────────────────────────────────┤
│  Crypto helpers                                                    │
│  sha256, hmac_sha256, pbkdf2_sha256, base64_encode/decode,         │
│  random_hex / random_b64, hex_xor                                  │
├────────────────────────────────────────────────────────────────────┤
│  Byte-buffer primitives                                            │
│  mem_alloc, mem_{set,get}_{u8,u16be,u32be}, mem_copy_str,          │
│  mem_to_str, str_ptr                                               │
├────────────────────────────────────────────────────────────────────┤
│  Sync TCP + TLS transport                                          │
│  tcp_connect / tcp_write / tcp_read / tcp_close / tcp_set_timeout  │
│  tcp_starttls  — in-place TLS upgrade of an established socket     │
│  worker_thread + SharedArrayBuffer + Atomics.wait                  │
│  (scripts/tcp_worker.js)                                           │
└────────────────────────────────────────────────────────────────────┘

Everything below contrib/db/pg.mere lives in scripts/pg_env.js and is shared between the CLI runner (run_wasm.js) and the HTTP harness (run_http_server.js). Wasm's synchronous model composes naturally with the request/response DB protocols; the worker-thread transport bridges Node's async sockets so a Wasm-level tcp_read blocks the way the caller expects.

Values crossing the extern boundary are hex-encoded whenever they'd otherwise contain NUL bytes — Mere's str is C-string, so raw digests and binary frames can't ride as strings. Frame construction uses mem_alloc + mem_set_u*be so the resulting buffer is opaque to the str layer.

Quick start

# 1. Boot a Postgres. Trust-mode works for everything except db_scram,
#    which needs POSTGRES_HOST_AUTH_METHOD=scram-sha-256.
docker run -d --name mere-pg -p 15432:5432 \
  -e POSTGRES_HOST_AUTH_METHOD=trust \
  -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres postgres:16

# 2. Build the driver + a demo.
./_build/default/bin/mere.exe -w examples/db_hello.mere > /tmp/db.wat
wat2wasm --enable-tail-call /tmp/db.wat -o /tmp/db.wasm

# 3. Run it.
node scripts/run_wasm.js /tmp/db.wasm

Expected output for db_hello:

connected
1 | hello
42 | world

Public API reference

Import contrib/db/pg.mere for everything except the pool helpers, which live in contrib/db/pg_pool.mere.

Connect

symbol	signature	notes
`pg_connect`	`host -> port -> user -> pw -> db -> fd`	trust / SCRAM auth
`pg_connect_ssl`	same shape	SSLRequest + TLS upgrade before auth
`pg_connect_ssl_verify`	+ `ca_pem`	verified TLS — reject bad chain
`pg_connect_url`	`str -> fd`	libpq URL; `?sslmode=require\|verify-*`
`pg_parse_url`	`str -> (host, port, user, pw, db)`	IPv6 brackets ok
`pg_close`	`fd -> unit`	sends Terminate + closes tcp

Auth methods: trust (any password accepted, pass "") and SCRAM-SHA-256. Cleartext-password and MD5 aren't wired — the driver prints a diagnostic and returns -1 if the server asks for them.

Query

symbol	signature
`pg_query`	`fd -> sql -> (str option list) list`
`pg_query_meta`	`fd -> sql -> ((str, int) list, rows)` — columns + rows
`pg_query_params`	`fd -> sql -> str option list -> rows`
`pg_query_params_meta`	`fd -> sql -> params -> (cols, rows)`

_params variants use the extended-query protocol (Parse/Bind/Execute/Sync). Parameters are sent out of band — SQL injection is impossible. Text format for both parameters and results; callers add ::int / ::uuid etc. casts where PG can't infer.

Prepared statements

Parse once, execute many:

symbol	signature
`pg_prepare`	`fd -> name -> sql -> handle`
`pg_execute`	`fd -> handle -> params -> rows`
`pg_execute_meta`	`fd -> handle -> params -> (cols, rows)`
`pg_deallocate`	`fd -> handle -> unit`

Transactions

Callbacks return 'a option — Some commits, None rolls back:

symbol	signature
`pg_tx`	`fd -> (fd -> 'a option) -> 'a option`
`pg_tx_iso`	`fd -> opts -> body -> 'a option`
`pg_savepoint`	`fd -> name -> unit`
`pg_rollback_to`	`fd -> name -> unit`
`pg_release_savepoint`	`fd -> name -> unit`
`pg_savepoint_scope`	`fd -> name -> body -> 'a option`

pg_tx_iso's opts is spliced verbatim after BEGIN — pass constants like "ISOLATION LEVEL SERIALIZABLE READ ONLY", never HTTP input.

Bulk transfer (COPY)

10-100× faster than a loop of INSERTs once batch sizes reach a few hundred rows.

symbol	signature
`pg_copy_from`	`fd -> table -> cols -> rows -> int` — row count or `-1`
`pg_copy_to`	`fd -> sql -> rows` — accepts subquery form

Text format only — backslash / tab / newline / CR are escaped per PG's COPY rules and None maps to \N.

Pub/sub (LISTEN / NOTIFY)

symbol	signature
`pg_listen` / `pg_unlisten`	`fd -> channel -> unit`
`pg_notify`	`fd -> channel -> payload -> unit` — interior `'` doubled
`pg_wait_notify`	`fd -> timeout_ms -> (int, str, str) option`
`pg_drain_notifications`	`fd -> (int, str, str) list` — pulls queued
`pg_poll_once` / `pg_poll`	`fd -> timeout_ms -> bool` / `unit`

Every drain function inside pg.mere (query result, COPY, handshake) captures async NotificationResponse messages into a module-level FIFO queue instead of dropping them, so notifications that arrived during a pg_query surface on the next pg_drain_notifications without another wire read.

Connection pool

Single-fd keep-alive — sufficient for the single-threaded runtime.

symbol	signature
`pg_pool_open`	`url -> pool` — no connection yet
`pg_pool_acquire`	`pool -> fd` — lazy connect on first call
`pg_pool_release`	`pool -> fd -> unit` — no-op today, API symmetry
`pg_pool_release_notifs`	`pool -> fd -> notifs` — release + drain
`pg_pool_pump`	`pool -> timeout_ms -> notifs` — idle-tick polling
`pg_pool_close`	`pool -> unit`

Row access

pg_query returns (str option list) list. Helpers turn cells into typed values:

symbol	signature
`pg_val_or`	`str option -> str -> str`
`pg_col_int` / `pg_col_int_or`	`-> int option` / `-> int`
`pg_col_bool` / `pg_col_bool_or`
`pg_col_at`	`row -> i -> str option`
`pg_first_col`	`rows -> i -> str option`

Metadata

pg_type_name : int -> str maps type OIDs to their canonical names (int4, text, uuid, jsonb, …). Unknown OIDs surface as "oid=<n>" so callers can pattern-match on the tag.

Demo catalog

All demos live under examples/db_*.mere. Build recipe (db_hello shown; substitute the file name):

./_build/default/bin/mere.exe -w examples/db_hello.mere > /tmp/db.wat
wat2wasm --enable-tail-call /tmp/db.wat -o /tmp/db.wasm
node scripts/run_wasm.js /tmp/db.wasm

Every demo is self-contained (its file header lists the exact docker command needed) and cleans up on process exit.

PostgreSQL (16 demos)

Auth & connect

demo	shows
db_hello	trust auth + simple query, minimal starter
db_scram	SCRAM-SHA-256 handshake against a scram-only server
db_url	libpq URL — IPv6 brackets, `?query`, `%40` percent decoding
db_pool	single-fd keep-alive: same `fd` + backend pid across acquires

Query surface

demo	shows
db_params	parameterized query — SQL injection payload lands as data, not SQL
db_prepared	named prepared statement, 5× execute, deallocate
db_types	RowDescription OIDs — `pg_type_name` on int4/text/bool/uuid/jsonb/…
db_typed	`pg_col_int` / `pg_col_bool` / defaults / `pg_first_col`

Transactions

demo	shows
db_tx	`pg_tx` commit / rollback via body return value
db_savepoint	isolation level + nested savepoint scope

Bulk transfer (COPY)

demo	shows
db_copy	`pg_copy_from` — 1000 rows in one round trip, edge-case escapes
db_copy_out	`pg_copy_to` — subquery form, escapes round-trip

Async pub/sub

demo	shows
db_notify	LISTEN / NOTIFY with two connections + timeout path
db_notify_async	notifications arriving inside another query's response stream
db_pool_pump	pool + LISTEN — pump / release-with-notifs / quiet tick

End-to-end integration

demo	shows
http_todo_pg	`contrib/http` + `pg_pool` — signup / login / todos backed by PG

MySQL (4 demos)

demo	shows
db_mysql	MySQL 8 — auto-selects `mysql_native_password` (SHA-1) or `caching_sha2_password` (SHA-256 fast + RSA-OAEP-SHA1 slow); NULL round-trip
db_mysql_prepared	`COM_STMT_PREPARE` + `_EXECUTE`, binary-protocol row decode; SQL-injection defense
db_mysql_pool	`mysql_pool` keep-alive + `mysql_parse_url` + percent decoding
db_mysql_tx	`mysql_tx_iso` (REPEATABLE READ) + `mysql_savepoint_scope` nested rollback

Redis (7 demos)

Core

demo	shows
db_redis	Redis 7 — RESP2 replies (str / err / int / bulk / array), typed extractors
db_redis_pipeline	`redis_pipeline` — 100 INCRs in one round trip; mixed SET/GET batch replies in order
db_redis_binary	Binary-safe args via `redis_command_b` — 5-byte payload with embedded NULs

RESP3 & async

demo	shows
db_redis_resp3	`HELLO 3` upgrade — HGETALL → `RRMap`, SMEMBERS → `RRSet`, `RRNull`
db_redis_pubsub	Redis PUB/SUB — two-fd `pubsub_client`; SUBSCRIBE + PSUBSCRIBE classified into a `pubsub_msg` variant (message / pmessage / subscribed / unsubscribed / pong / timeout / closed) via `contrib/db/redis_pubsub`
db_redis_push	CLIENT TRACKING → real `RRPush` invalidation on key mutation
db_redis_queue	List-backed work queue — LPUSH producer + BRPOP consumer with priority multi-queue fallback + timeout path via `contrib/db/redis_queue`
db_redis_pubsub_reconnect	`redis_pubsub_run_forever` — auto-reconnects + resubscribes after `CLIENT KILL TYPE PUBSUB` mid-run; handler counts 4 deliveries across the fd swap
db_redis_hll	HyperLogLog cardinality — `hll_add` / `hll_count` / `hll_merge` for 12 KiB approximate distinct-count. Two-shard union demo (5 true distinct across shard-a + shard-b) via `contrib/db/redis_hll`
db_redis_stream	Redis Streams — `stream_add` / `stream_read` / `stream_len` durable append-only log. 3 XADDs, XLEN=3, full replay via id `0`, tail-only replay via previous id, past-the-tail empty. Via `contrib/db/redis_stream`
db_redis_stream_groups	Redis Streams consumer groups — `stream_group_create` / `stream_group_read` / `stream_ack` / `stream_pending_len`. Two workers in one group, XREADGROUP `>` load-balances (A: 1-2, B: 3-4), PEL 4→2→0 as each ACKs. Same module as above
db_redis_lock	Distributed mutex via `SET NX PX` + compare-and-delete `EVAL` — `redis_lock_acquire` / `redis_lock_release`. Contention returns None; impostor release with wrong token silently fails. Via `contrib/db/redis_lock`
db_redis_ratelimit	Distributed fixed-window rate limiter — `INCR` + `EXPIRE` per bucket. 3-per-2s policy: attempts 1-3 pass, 4-5 blocked, window roll resets. Via `contrib/db/redis_ratelimit` (multi-instance version of `contrib/http/ratelimit`)

Cluster / high availability

demo	shows
db_redis_cluster	Slot hashing + hash-tag; live routing / MOVED redirect wired

Transport / TLS (5 demos — one per engine plus verification variants)

demo	shows
db_ssl	Postgres over TLS 1.3 — `pg_connect_ssl` + `?sslmode=require`
db_mysql_ssl	MySQL over TLS 1.3 — `mysql_connect_ssl` + `?ssl=true`
db_redis_ssl	Redis 7 over TLS — permissive / system-CA reject / custom-CA accept
db_ssl_verify	Cert-chain verification — accept-any / system CAs / custom CA
db_ssl_san	Hostname / SAN matching under verify-full — DNS-only SAN cert rejects 127.0.0.1

Limitations and future work

Auth: SCRAM-SHA-256 is the only strong method wired. Cleartext password and MD5 aren't; SCRAM-SHA-256-PLUS (channel binding) isn't either.
TLS: both drivers negotiate TLS 1.2/1.3 through Node's tls module. pg_connect_ssl sends PG's 8-byte SSLRequest and waits for the single-byte 'S' reply before upgrading; mysql_connect_ssl sends the 32-byte "SSL Request Packet" with the CLIENT_SSL bit at MySQL seq=1, then hands the full HandshakeResponse41 over the encrypted channel at seq=2. The _verify variants (pg_connect_ssl_verify / mysql_connect_ssl_verify / redis_connect_ssl_verify) take an optional CA PEM (empty = Node's built-in trust store), require a valid chain, AND run tls.checkServerIdentity against the caller-supplied hostname — so a cert without a matching CN or Subject Alternative Name entry short-circuits the connect with -1 even if the chain itself is valid. Hostname / SAN checks apply to both DNS and IP literal targets (Node's default checker handles IP: SAN entries). URL parsers auto-select: sslmode=verify-full / sslmode=verify-ca → verified, sslmode=require / ssl=true → accept-anything, otherwise plain.
Binary column format: everything is text. Encoding / decoding of PG binary format would let us skip a pg_type_name-based decode step for hot paths.
Column NULL asymmetry: pg_query results distinguish NULL from "" via str option; some helpers (pg_col_int_or) collapse them to a default for ergonomics.
Async event loop: NotificationResponse handling is queue-based — the pool has a pump primitive but no background loop. A real event loop would need cooperation from the Node harness.
MySQL client: connect + auth + COM_QUERY + prepared statements (COM_STMT_PREPARE / _EXECUTE / _CLOSE) with binary-protocol row decoding + mysql:// URL parsing + single-fd mysql_pool + transactions (mysql_tx / mysql_tx_iso) with savepoints. Both auth plugins are wired — mysql_native_password (SHA-1) and caching_sha2_password (SHA-256 fast-path + RSA-OAEP-SHA1 public-key exchange when the server's auth cache is cold). TLS is wired (mysql_connect_ssl / ?ssl=true). LISTEN-style pub/sub is not — MySQL has no server-side pub/sub (NOTIFY is a PG feature).
Redis client: redis_command speaks RESP2 out of the box; call redis_hello3 to upgrade to RESP3. The full RESP3 wire spec is parsed — RRNull, RRDouble, RRBool, RRMap, RRSet, RRBigNum, RRVerbatim, RRPush, and RRAttr are all recognized on the wire and surfaced through typed extractors (redis_as_bool / redis_as_map / redis_as_bignum / redis_as_verbatim / redis_as_push) plus a redis_unwrap_attr metadata peeler. Binary-safe command args ride through redis_command_b with a redis_arg variant — RATxt s for ASCII, RABin (ptr, len) for arbitrary byte buffers built via mem_alloc / bytes_from_hex_alloc. Reads still surface as RRBulk (Some str) where str is NUL-terminated at the Mere layer; the raw byte buffer is intact at that pointer, so callers who need binary reads peek via mem_get_u8. Also wired: PUB/SUB (redis_subscribe / redis_psubscribe / redis_wait_message / redis_publish), pipelining (redis_pipeline), TLS (redis_connect_ssl / redis_connect_ssl_verify).
SQLite: not implemented. Would need either a fresh Mere implementation of the file format or a bundled Wasm build (sql.js / wa-sqlite).

FilesExpand file tree

db.md

Latest commit

History