Exec routes: run the request's SQL directly (CREATE ROUTE … AS $sql)

README.md

@@ -71,7 +71,7 @@ CREATE [OR REPLACE] ROUTE <name> <METHOD> '<pattern>'
   [BODY TYPE '<duckdb-json-transform-structure>']
   [PARAM <name> [<type>] [HEADER|COOKIE|QUERY [wire-name]]
          [DEFAULT <lit>] [GE|GT|LE|LT <n>] [MIN_LENGTH|MAX_LENGTH <n>] …]
-  AS <select>;
+  AS <select> | AS $<param>;
 
 DROP ROUTE <name>;
 ```
@@ -81,6 +81,7 @@ DROP ROUTE <name>;
 | **Method** | `GET`, `POST`, `PUT`, `DELETE`, `PATCH`, `HEAD` |
 | **Path** | Must start with `/` (unless joined via `GROUP`). `:param` and `{param}` bind to `$param` |
 | **Handler** | Any SQL statement that returns a result (typically `SELECT` / `INSERT … RETURNING` / `COPY`) |
+| **Exec handler** | Exactly one parameter, `AS $sql`: the request's bound text is executed as SQL — see [Exec routes](#exec-routes-as-sql) |
 | **Params** | Cast to the types the prepared handler expects; failures → **422** with FastAPI-shaped `detail` |
 | **Success status** | Default `200`; override with `STATUS 201` (etc.) |
 | **Auth** | `REQUIRE <scheme>` runs a `CREATE AUTH` scheme before the handler |
@@ -91,6 +92,39 @@ Handler SQL is validated at `CREATE` time (broken SQL fails at create, not on
 first request). `CREATE OR REPLACE` does not leave a half-applied route on
 failure.
 
+### Exec routes (`AS $sql`)
+
+A handler that is exactly one parameter runs the request's SQL text itself —
+an HTTP door to the database, without a loopback through another client:
+
+```sql
+CREATE AUTH ops AS API_KEY;
+CREATE ROUTE sql POST '/sql' REQUIRE ops
+  PARAM sql VARCHAR MIN_LENGTH 1
+  AS $sql;
+```
+
+```sh
+curl -X POST http://127.0.0.1:8000/sql -H 'X-API-Key: …' \
+  -H 'Content-Type: application/json' \
+  -d '{"sql":"CREATE TABLE t AS SELECT 1 AS id; INSERT INTO t VALUES (2); SELECT * FROM t"}'
+# [{"id":1},{"id":2}]
+```
+
+| Behavior | Detail |
+|----------|--------|
+| **Recognised** | The handler, trimmed, is one `$name` and nothing else. `CREATE` skips handler validation for it; `$claims_*` never qualifies |
+| **Binding** | The parameter binds like any other (JSON body field, form field, query string, `HEADER`/`COOKIE`), so `PARAM … MIN_LENGTH`/`DEFAULT` apply |
+| **Execution** | The text is split into statements with DuckDB's parser and each runs once, in order, on the request's connection — DDL, DML, `PRAGMA`, `SET`, `INSTALL`/`LOAD`, `COPY`, `ATTACH` |
+| **Response** | The **last** statement's result, through the usual negotiation (JSON default, `Accept: text/csv`, `application/x-ndjson`, `application/vnd.apache.arrow.stream`, route `FORMAT`; `html`/`text` columns). A last statement with no columns (`SET`, `PRAGMA …`) answers `[]` |
+| **Errors** | The first failing statement stops the batch — later statements never run — and answers **422** with a FastAPI-shaped `detail` whose `loc` is the parameter (`["body","sql"]`) and whose `msg` is DuckDB's error (`Statement 2 of 3 failed: …` for a batch). Earlier statements are not rolled back unless the batch opened its own `BEGIN` |
+| **Limits** | `REQUIRE` runs before any request SQL; `query_timeout_ms` bounds the whole batch (504); `max_response_bytes` applies to the last result |
+| **Session** | Each request gets its own connection: `SET`, `TEMP` objects and open transactions end with the request. `SET GLOBAL`, tables and `ATTACH` persist |
+| **Policies** | Request SQL cannot be policy-rewritten, so while any row-access or masking policy is bound an exec route answers 403 |
+
+An exec route executes arbitrary SQL with the server's privileges. Keep it on
+loopback or behind `REQUIRE`.
+
 ### Scheduled routes
 
 quackapi stores `SCHEDULE` metadata but does not run a scheduler. Timing stays

src/include/quackapi_util.hpp

@@ -18,6 +18,13 @@ string QuackapiTrim(const string &input);
 //! JSON string escape including control bytes as \u00XX (auth + OpenAPI + responses).
 string QuackapiJsonEscape(const string &input);
 
+//! An exec route's handler is exactly one named parameter (`AS $sql`). The
+//! request binds that parameter and quackapi executes its text as SQL on the
+//! request connection instead of preparing the handler. Returns the parameter
+//! name. $claims_* names are server-bound, never request SQL, so they do not
+//! qualify.
+bool QuackapiExecRouteParam(const string &handler_sql, string &param_name);
+
 //===--------------------------------------------------------------------===//
 // DDL apply-table-function shell (CREATE/DROP ROUTE|GROUP|AUTH|… plan→exec)
 //

src/quackapi_ddl.cpp

@@ -1256,8 +1256,13 @@ void ApplyRouteExec(ClientContext &context, TableFunctionInput &data_p, DataChun
 					throw InvalidInputException("Invalid BODY SCHEMA JSON for route \"%s\"", bind_data.route.name);
 				}
 			}
-			auto prepared = con.Prepare(bind_data.route.handler_sql);
-			if (prepared->HasError()) {
+			// An exec route (`AS $sql`) has no SQL of its own to validate: the
+			// request supplies it, and each request reports its own errors.
+			string exec_param;
+			auto prepared = QuackapiExecRouteParam(bind_data.route.handler_sql, exec_param)
+			                    ? unique_ptr<PreparedStatement>()
+			                    : con.Prepare(bind_data.route.handler_sql);
+			if (prepared && prepared->HasError()) {
 				// Native libpq path (pg_dsn): handler SQL is Postgres dialect and may
 				// not prepare under DuckDB (domains, DO, INSERT…RETURNING, …). Allow
 				// CREATE when pg_dsn is configured; request time uses libpq, else 500.

src/quackapi_openapi.cpp

@@ -207,10 +207,16 @@
 		case_insensitive_map_t<LogicalType> expected_types;
 		vector<string> named_params;
 
-		auto prepared = con.Prepare(route.handler_sql);
+		// An exec route (`AS $sql`) documents its one parameter; its response
+		// columns depend on the request, so it advertises none.
+		string exec_param;
+		const bool exec_route = QuackapiExecRouteParam(route.handler_sql, exec_param);
+		auto prepared = con.Prepare(exec_route ? "SELECT $" + exec_param + "::VARCHAR" : route.handler_sql);
 		if (!prepared->HasError()) {
-			col_names = prepared->GetNames();
-			col_types = prepared->GetTypes();
+			if (!exec_route) {
+				col_names = prepared->GetNames();
+				col_types = prepared->GetTypes();
+			}
 			expected_types = prepared->GetExpectedParameterTypes();
 			for (auto &entry : prepared->named_param_map) {
 				named_params.push_back(entry.first);
@@ -504,7 +510,7 @@
 		} else {
 			schemes += "{\"type\":\"apiKey\",\"in\":\"header\",\"name\":" + JsonString(a.header) + "}";
 		}
 		(void)AuthKindString;
 	}
 	schemes += "}";
 

src/quackapi_server.cpp

@@ -993,6 +993,45 @@ bool ValidateRequestSql(Connection &con, const string &sql, string &error) {
 	}
 }
 
+//! Execute request SQL text for an exec route (`AS $sql`): every statement in
+//! order on the request connection, stopping at the first failure. The last
+//! statement's result is the response. Unlike Connection::Query(string), which
+//! answers with the first statement that produces rows, a batch such as
+//! `CREATE …; INSERT …; SELECT …` answers with its final SELECT.
+bool ExecuteRequestSql(Connection &con, const string &sql, unique_ptr<QueryResult> &result, string &error) {
+	// DuckDB's statement extraction truncates at the first null byte, which
+	// would silently drop the tail of the batch while executing the prefix.
+	// Null bytes are never legitimate in request SQL text.
+	if (sql.find('\0') != string::npos) {
+		error = "SQL text must not contain null bytes";
+		return false;
+	}
+	vector<unique_ptr<SQLStatement>> statements;
+	try {
+		statements = con.ExtractStatements(sql);
+	} catch (std::exception &ex) {
+		error = ErrorData(ex).Message();
+		return false;
+	}
+	if (statements.empty()) {
+		error = "No statement to execute";
+		return false;
+	}
+	const auto statement_count = statements.size();
+	for (idx_t i = 0; i < statement_count; i++) {
+		unique_ptr<QueryResult> statement_result = con.Query(std::move(statements[i]));
+		if (statement_result->HasError()) {
+			error = statement_result->GetError();
+			if (statement_count > 1) {
+				error = StringUtil::Format("Statement %llu of %llu failed: %s", i + 1, statement_count, error);
+			}
+			return false;
+		}
+		result = std::move(statement_result);
+	}
+	return true;
+}
+
 //! Resolve the native DuckDB type represented by a BODY TYPE declaration.
 //! This lets route SQL use $body.items directly: the server inserts the cast
 //! from the one declaration before DuckDB prepares the handler.
@@ -3682,6 +3721,11 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 	string policy_error;
 	string handler_sql =
 	    RewriteHandlerWithPolicies(*db, match.route.handler_sql, authenticated, deny_unauth, policy_error);
+	// Exec route (`AS $sql`): the bound parameter is the SQL to run. With row
+	// access or masking policies bound, the handler cannot be policy-rewritten,
+	// so the check above already fails closed (403) before any request SQL runs.
+	string exec_param;
+	const bool exec_route = QuackapiExecRouteParam(match.route.handler_sql, exec_param);
 	if (deny_unauth || !policy_error.empty()) {
 		SetJson(res, 403, "{\"detail\":\"Policy denies unauthenticated access\"}");
 		finish();
@@ -3953,7 +3997,7 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 		// Native Postgres path (optional): same model as FastAPI+psycopg —
 		// thread-local libpq, fresh PQexecParams per request. BEFORE DuckDB
 		// prepare so INSERT…RETURNING and other ATTACH-hostile SQL still work.
-		if (!options.pg_dsn.empty()) {
+		if (!options.pg_dsn.empty() && !exec_route) {
 			string pg_body, pg_err;
 			auto pg_result =
 			    QuackapiTryPgNative(options.pg_dsn, handler_sql, provided, options.max_response_bytes, pg_body, pg_err);
@@ -3979,7 +4023,7 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 		// DuckDB prepare (after libpq short-circuit). Fresh prepare every request:
 		// no SQL-keyed body/plan cache — mutable tables and volatile functions
 		// (now/uuid) must re-execute.
-		if (has_body_named && !match.route.body_type.empty()) {
+		if (has_body_named && !match.route.body_type.empty() && !exec_route) {
 			string body_sql_type = has_validated_typed_body ? validated_typed_body.type().ToString() : string();
 			if (body_sql_type.empty()) {
 				string type_error;
@@ -3992,7 +4036,10 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 			handler_sql = RewriteTypedBodyParameter(handler_sql, body_sql_type);
 		}
 		auto prepare_started = std::chrono::steady_clock::now();
-		auto prepared_owned = con.Prepare(handler_sql);
+		// An exec route prepares a stand-in that names its one parameter, so the
+		// request's SQL text goes through the same binding and PARAM validation
+		// (MIN_LENGTH, DEFAULT, HEADER, …) as any other route parameter.
+		auto prepared_owned = con.Prepare(exec_route ? "SELECT $" + exec_param : handler_sql);
 		sql_prepare_ms =
 		    std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - prepare_started).count();
 		if (prepared_owned->HasError()) {
@@ -4061,6 +4108,11 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 					expected = LogicalType::VARCHAR;
 				}
 			}
+			const bool is_exec_sql = exec_route && StringUtil::CIEquals(param_name, exec_param);
+			if (is_exec_sql) {
+				// The exec parameter is SQL text, whatever its declaration says.
+				expected = LogicalType::VARCHAR;
+			}
 
 			auto it = provided.find(param_name);
 			// Default loc for missing errors: path / header / cookie / query.
@@ -4126,7 +4178,8 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 					    {ValidationLocJson(loc_kind, param_name), "Input should be a valid integer", "type_error"});
 					continue;
 				}
-			} else if (expected.id() == LogicalTypeId::UNKNOWN || expected.id() == LogicalTypeId::VARCHAR) {
+			} else if (!is_exec_sql &&
+			           (expected.id() == LogicalTypeId::UNKNOWN || expected.id() == LogicalTypeId::VARCHAR)) {
 				// If raw is not a strict integer but DuckDB would accept it as
 				// INTEGER (float/scientific/hex-ish), reject so execute-time
 				// `$param::INTEGER` cannot round. Plain non-numeric strings
@@ -4233,7 +4286,7 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 		// The SQL route delegates the body value to a dynamic SQL function. Validate
 		// it locally first so remote parser/binder failures cannot become 200 []
 		// when the delegate reports an empty result instead of an error.
-		if (match.route.pattern == "/sql") {
+		if (match.route.pattern == "/sql" && !exec_route) {
 			auto sql_it = provided.find("sql");
 			if (sql_it != provided.end()) {
 				string sql_error;
@@ -4246,9 +4299,36 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 		}
 
 		auto execute_started = std::chrono::steady_clock::now();
-		auto result = prepared->Execute(named_values, false);
-		sql_execute_ms =
-		    std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - execute_started).count();
+		unique_ptr<QueryResult> result;
+		if (exec_route) {
+			// The request's SQL is client input: any statement failure is a 422
+			// on the parameter that carried it, and later statements never run.
+			auto sql_it = bound_raw.find(exec_param);
+			if (sql_it == bound_raw.end()) {
+				SetJson(res, 422,
+				        ValidationErrorJson(IsBodyMethod(req.method) ? "body" : "query", exec_param, "Field required",
+				                            "missing"));
+				finish();
+				return;
+			}
+			string exec_error;
+			const bool exec_ok = ExecuteRequestSql(con, sql_it->second.second, result, exec_error);
+			sql_execute_ms =
+			    std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - execute_started).count();
+			if (!exec_ok) {
+				if (deadline.Expired()) {
+					SetJson(res, 504, "{\"detail\":\"Query execution deadline exceeded\"}");
+				} else {
+					SetJson(res, 422, ValidationErrorJson(sql_it->second.first, exec_param, exec_error, "value_error"));
+				}
+				finish();
+				return;
+			}
+		} else {
+			result = prepared->Execute(named_values, false);
+			sql_execute_ms =
+			    std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - execute_started).count();
+		}
 		if (result->HasError()) {
 			// Preserve the status mapping while returning DuckDB's diagnostic so clients can fix bad input.
 			// - Conversion errors → 422 with recovered param name (not "_")
@@ -4403,6 +4483,13 @@ void QuackapiHttpServer::HandleRequest(const duckdb_httplib::Request &req, duckd
 		}
 
 		// No data columns: empty body (redirect / cookie-only responses).
+		if (data_cols.empty() && exec_route && location_col == names.size() && set_cookie_col == names.size()) {
+			// A final SET / PRAGMA / BEGIN returns no columns at all; answer the
+			// same empty JSON array a zero-row SELECT gets, not an empty body.
+			SetJson(res, match.route.status, "[]");
+			finish();
+			return;
+		}
 		if (data_cols.empty()) {
 			res.status = match.route.status;
 			res.body.clear();

src/quackapi_util.cpp

@@ -19,6 +19,30 @@ string QuackapiTrim(const string &input) {
 	return input.substr(begin, end - begin);
 }
 
+bool QuackapiExecRouteParam(const string &handler_sql, string &param_name) {
+	auto handler = QuackapiTrim(handler_sql);
+	if (handler.size() < 2 || handler[0] != '$') {
+		return false;
+	}
+	auto is_start = [](char c) {
+		return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_';
+	};
+	if (!is_start(handler[1])) {
+		return false;
+	}
+	for (idx_t i = 2; i < handler.size(); i++) {
+		if (!is_start(handler[i]) && !(handler[i] >= '0' && handler[i] <= '9')) {
+			return false;
+		}
+	}
+	auto name = handler.substr(1);
+	if (StringUtil::StartsWith(StringUtil::Lower(name), "claims_")) {
+		return false;
+	}
+	param_name = name;
+	return true;
+}
+
 string QuackapiJsonEscape(const string &input) {
 	string result;
 	result.reserve(input.size() + 2);

test/conformance/cases.jsonl

@@ -83,3 +83,10 @@
 {"id": "post_users_age_float_str", "group": "validation", "method": "POST", "path": "/users?name=x&age=1.5", "headers": {"Content-Type": "application/json"}, "body": "{}", "fastapi_doc": "https://fastapi.tiangolo.com/tutorial/body/", "expect_status": 422, "notes": "Float-string for int should 422"}
 {"id": "get_user_head_explicit", "group": "methods", "method": "HEAD", "path": "/users/1", "headers": {}, "body": null, "fastapi_doc": "https://fastapi.tiangolo.com/tutorial/path-params/", "expect_status": 200, "notes": "Auto-HEAD for GET /users/:id not registered", "class_hint": "NOT-BUILT-YET"}
 {"id": "search_repeated_param", "group": "params", "method": "GET", "path": "/search?q=a&q=b", "headers": {}, "body": null, "fastapi_doc": "https://fastapi.tiangolo.com/tutorial/query-params/", "expect_status": 200, "notes": "Repeated query: FastAPI takes first; httplib first-wins in CollectHeaders; params may last-win"}
+{"id":"exec_sql_json","group":"exec","method":"POST","path":"/exec-sql","headers":{"Content-Type":"application/json"},"body":"{\"sql\":\"SELECT 42 AS answer\"}","expect_status":200,"expect_ct_substr":"application/json","expect_json_path_eq":{"0.answer":42},"notes":"exec route: request SQL runs directly"}
+{"id":"exec_sql_last_statement","group":"exec","method":"POST","path":"/exec-sql","headers":{"Content-Type":"application/json"},"body":"{\"sql\":\"CREATE OR REPLACE TABLE exec_conf AS SELECT 1 AS id; INSERT INTO exec_conf VALUES (2); SELECT list(id ORDER BY id) AS ids FROM exec_conf\"}","expect_status":200,"expect_json_len":1,"expect_json_path_eq":{"0.ids":[1,2]},"notes":"DDL, DML, SELECT in one request; the last statement answers"}
+{"id":"exec_sql_csv","group":"exec","method":"POST","path":"/exec-sql","headers":{"Content-Type":"application/json","Accept":"text/csv"},"body":"{\"sql\":\"SELECT 1 AS a, 2 AS b\"}","expect_status":200,"expect_ct_substr":"text/csv","expect_body_contains":["a,b","1,2"]}
+{"id":"exec_sql_arrow","group":"exec","method":"POST","path":"/exec-sql","headers":{"Content-Type":"application/json","Accept":"application/vnd.apache.arrow.stream"},"body":"{\"sql\":\"SELECT 1 AS a; SELECT 2 AS b\"}","expect_status":200,"expect_ct_substr":"application/vnd.apache.arrow.stream","notes":"Arrow IPC stream via the community nanoarrow extension"}
+{"id":"exec_sql_error_422","group":"exec","method":"POST","path":"/exec-sql","headers":{"Content-Type":"application/json"},"body":"{\"sql\":\"SELECT 1; SELECT nope FROM nowhere\"}","expect_status":422,"expect_body_contains":["Statement 2 of 2 failed","nowhere"],"expect_422":{"loc0":"body","loc1":"sql","type":"value_error"}}
+{"id":"exec_sql_auth_401","group":"exec","method":"POST","path":"/exec-sql-secure","headers":{"Content-Type":"application/json"},"body":"{\"sql\":\"SELECT 1 AS x\"}","expect_status":401}
+{"id":"exec_sql_auth_ok","group":"exec","method":"POST","path":"/exec-sql-secure","headers":{"Content-Type":"application/json","X-API-Key":"k-secret"},"body":"{\"sql\":\"SELECT 1 AS x\"}","expect_status":200,"expect_json_path_eq":{"0.x":1}}

test/conformance/routes.sql

@@ -128,3 +128,8 @@ SELECT $name::VARCHAR AS name, $age::INTEGER AS age;
 
 CREATE ROUTE upload POST '/upload' AS
 SELECT $file::VARCHAR AS content, $filename::VARCHAR AS filename;
+
+-- Exec routes: the handler is one parameter; the request's SQL runs directly.
+CREATE ROUTE exec_sql POST '/exec-sql' AS $sql;
+
+CREATE ROUTE exec_sql_secure POST '/exec-sql-secure' REQUIRE site AS $sql;