Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ curl http://127.0.0.1:8000/items/abc
CREATE [OR REPLACE] ROUTE <name> <METHOD> '<pattern>'
[STATUS <n>]
[REQUIRE <auth>]
[SCHEDULE '<6-field cron expression, seconds first>']
[GROUP <group> | IN GROUP <group>]
[BODY SCHEMA '<json-schema>']
[BODY TYPE '<duckdb-json-transform-structure>']
Expand All @@ -83,12 +84,53 @@ DROP ROUTE <name>;
| **Params** | Cast to the types the prepared handler expects; failures → **422** with FastAPI-shaped `detail` |
| **Success status** | Default `200`; override with `STATUS 201` (etc.) |
| **Auth** | `REQUIRE <scheme>` runs a `CREATE AUTH` scheme before the handler |
| **Schedule** | Optional six-field cron expression (`seconds minutes hours day-of-month month day-of-week`); stored on the route and registered by the community `cronjob` extension |

Live updates: a route created after `quackapi_serve()` is served immediately.
Handler SQL is validated at `CREATE` time (broken SQL fails at create, not on
first request). `CREATE OR REPLACE` does not leave a half-applied route on
failure.

### Scheduled routes

quackapi stores `SCHEDULE` metadata but does not run a scheduler. Timing stays
with the community `cronjob` extension. Scheduled routes must be callable
without path parameters: a `:param` or `{param}` in the final route pattern is
rejected at `CREATE` time. `SCHEDULE` is route-only; `CREATE GROUP` does not
accept it. A route may join a group and schedule its own parameter-free,
group-expanded path.

At application startup, load `cronjob` and register each scheduled route with
its own `cron()` statement. `quackapi_request` executes the route handler
in-process, so no HTTP listener or external request is needed:

```sql
INSTALL cronjob FROM community;
LOAD cronjob;

-- one statement per route
SELECT cron('SELECT * FROM quackapi_request(''POST'', ''/jobs/refresh'')', '0 */5 * * * *');
```

Register one route per `cron()` call. With the `cronjob` extension on DuckDB
1.5.x, a single `SELECT cron(query, schedule) FROM quackapi_routes() WHERE
schedule IS NOT NULL` over several rows registers every job with the **first
row's schedule**, so generate the statements from `quackapi_routes()` and run
them one at a time (self-dispatch):

```sql
SELECT 'SELECT cron(' || chr(39) || 'SELECT * FROM quackapi_request('
|| chr(39) || chr(39) || method || chr(39) || chr(39) || ', '
|| chr(39) || chr(39) || pattern || chr(39) || chr(39) || ')' || chr(39)
|| ', ' || chr(39) || schedule || chr(39) || ');' AS statement
FROM quackapi_routes()
WHERE schedule IS NOT NULL;
```

The optional `test/sql/quackapi_schedule_cronjob.test` SQLLogic case asserts
a single-route registration when `cronjob` is available; otherwise run the
statements above manually in the same DuckDB v1.5.6 environment.

### Response modes (column names)

| Column name | Response |
Expand Down Expand Up @@ -209,7 +251,7 @@ the “PDF service” is a function call in the same address space — not an RP
| `quackapi_wait` | `(port [, timeout_ms], host := …)` — TCP readiness | `ready`, `listen_url` |
| `quackapi_stop` | `([port])` — omit port to stop all | `status` |
| `quackapi_telemetry_flush` | `([port])` — drain pending sink rows without stopping | `status` |
| `quackapi_routes` | `()` | `name, method, pattern, status, handler, require_auth, group_name, tags, format, envelope, empty_status, timeout_sec` |
| `quackapi_routes` | `()` | `name, method, pattern, status, handler, require_auth, group_name, tags, format, envelope, empty_status, timeout_sec, schedule` |
| `quackapi_servers` | `()` | `host, port, listen_url, http_client, http_client_reason` |
| Setting | `SET quackapi_cors_origins = '*' \| 'https://a,https://b'` | empty = CORS off |
| Setting | `SET quackapi_memory_limit = '4GB' \| '512MB' \| …` | empty = non-clobber default logic |
Expand Down
3 changes: 3 additions & 0 deletions src/include/quackapi_state.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,9 @@ struct QuackapiRoute {
//! When set, HandleRequest extends SO_RCVTIMEO/SO_SNDTIMEO and the active
//! SocketStream select deadlines for this connection only.
int32_t timeout_sec = 0;
//! Optional six-field cron expression (seconds first), owned by cronjob.
//! Empty means this route is not scheduled.
string schedule;
};

//! Row-access policy: predicate over table columns + $claims_* (JWT/auth claims).
Expand Down
127 changes: 117 additions & 10 deletions src/quackapi_ddl.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,41 @@ bool IsParamTypeName(const string &tok) {
u == "BOOLEAN" || u == "BOOL";
}

bool HasRoutePathParams(const string &pattern) {
idx_t start = 0;
while (start <= pattern.size()) {
auto end = pattern.find('/', start);
auto segment = pattern.substr(start, end == string::npos ? string::npos : end - start);
if ((!segment.empty() && segment[0] == ':') ||
(segment.size() >= 2 && segment.front() == '{' && segment.back() == '}')) {
return true;
}
if (end == string::npos) {
break;
}
start = end + 1;
}
return false;
}

bool ValidateRouteSchedule(const string &schedule, string &error) {
idx_t fields = 0;
bool in_field = false;
for (auto c : schedule) {
if (StringUtil::CharacterIsSpace(c)) {
in_field = false;
} else if (!in_field) {
fields++;
in_field = true;
}
}
if (fields != 6) {
error = "SCHEDULE must contain exactly 6 whitespace-separated cron fields (seconds first)";
return false;
}
return true;
}

//! Serialize param specs for the apply_route table function (plan → exec).
//! Format fields (FS=\x1f): name, type, has_def, def_null, def_raw, has_ge, ge, has_gt, gt,
//! has_le, le, has_lt, lt, has_min, min, has_max, max, source, external_name
Expand Down Expand Up @@ -283,7 +318,8 @@ bool ParseRouteTimeoutSec(const string &raw, int32_t &out_sec, string &err) {
//! CREATE [OR REPLACE] ROUTE <name> <METHOD> '<pattern>'
//! [STATUS <n>] [REQUIRE <auth>] [FORMAT json|ndjson|csv|parquet|arrow]
//! [ENVELOPE array|object] [EMPTY STATUS <n> [BODY '<json>']]
//! [TIMEOUT <n>|'30s'|'5m'] [GROUP <name> | IN GROUP <name>]
//! [TIMEOUT <n>|'30s'|'5m'] [SCHEDULE '<6-field cron, seconds first>']
//! [GROUP <name> | IN GROUP <name>]
//! [BODY SCHEMA '<json-schema>'] [BODY TYPE '<duckdb-json-structure>']
//! [PARAM <name> [<type>] [HEADER|COOKIE [wire-name]]
//! [DEFAULT <lit>] [GE/GT/LE/LT/MIN_LENGTH/MAX_LENGTH <n>] ... ]
Expand Down Expand Up @@ -378,9 +414,58 @@ ParserExtensionParseResult RouteDdlParse(ParserExtensionInfo *, const string &qu
int empty_status = 0;
string empty_body;
int32_t timeout_sec = 0;
string schedule;
bool schedule_seen = false;
string schedule_error;
auto rest_upper = StringUtil::Upper(rest);
auto TryConsumeSchedule = [&]() -> bool {
if (!(StringUtil::StartsWith(rest_upper, "SCHEDULE") &&
(rest.size() == 8 || StringUtil::CharacterIsSpace(rest[8])))) {
return false;
}
if (schedule_seen) {
schedule_error = "SCHEDULE specified more than once";
return true;
}
auto after_schedule = QuackapiTrim(rest.substr(8));
if (after_schedule.empty() || after_schedule[0] != '\'') {
schedule_error = "SCHEDULE expects a quoted 6-field cron expression (seconds first)";
return true;
}
string value;
idx_t i = 1;
while (i < after_schedule.size()) {
if (after_schedule[i] == '\'') {
if (i + 1 < after_schedule.size() && after_schedule[i + 1] == '\'') {
value += '\'';
i += 2;
continue;
}
break;
}
value += after_schedule[i++];
}
if (i >= after_schedule.size() || after_schedule[i] != '\'') {
schedule_error = "Unterminated SCHEDULE string";
return true;
}
if (!ValidateRouteSchedule(value, schedule_error)) {
return true;
}
schedule = std::move(value);
schedule_seen = true;
rest = QuackapiTrim(after_schedule.substr(i + 1));
rest_upper = StringUtil::Upper(rest);
return true;
};
for (int clause_round = 0; clause_round < 18; clause_round++) {
rest_upper = StringUtil::Upper(rest);
if (TryConsumeSchedule()) {
if (!schedule_error.empty()) {
return ParserExtensionParseResult(schedule_error);
}
continue;
}
// [STATUS <n>]
if (StringUtil::StartsWith(rest_upper, "STATUS") &&
(rest.size() == 6 || StringUtil::CharacterIsSpace(rest[6]))) {
Expand Down Expand Up @@ -777,7 +862,7 @@ ParserExtensionParseResult RouteDdlParse(ParserExtensionInfo *, const string &qu
auto mu = StringUtil::Upper(maybe);
if (mu != "DEFAULT" && mu != "GE" && mu != "GT" && mu != "LE" && mu != "LT" && mu != "MIN_LENGTH" &&
mu != "MAX_LENGTH" && mu != "PARAM" && mu != "BODY" && mu != "AS" && mu != "HEADER" &&
mu != "COOKIE" && mu != "QUERY" && !IsParamTypeName(maybe)) {
mu != "COOKIE" && mu != "QUERY" && mu != "SCHEDULE" && !IsParamTypeName(maybe)) {
spec.external_name = maybe;
rest = QuackapiTrim(rest.substr(te));
}
Expand Down Expand Up @@ -809,6 +894,10 @@ ParserExtensionParseResult RouteDdlParse(ParserExtensionInfo *, const string &qu
(rest.size() == 7 || StringUtil::CharacterIsSpace(rest[7]))) {
break;
}
if (StringUtil::StartsWith(rest_upper, "SCHEDULE") &&
(rest.size() == 8 || StringUtil::CharacterIsSpace(rest[8]))) {
break;
}
// HEADER / COOKIE may appear after DEFAULT/constraints too.
if (TryConsumeSource()) {
continue;
Expand Down Expand Up @@ -895,6 +984,14 @@ ParserExtensionParseResult RouteDdlParse(ParserExtensionInfo *, const string &qu
}
}

// SCHEDULE may also follow PARAM/BODY clauses.
rest_upper = StringUtil::Upper(rest);
if (TryConsumeSchedule()) {
if (!schedule_error.empty()) {
return ParserExtensionParseResult(schedule_error);
}
}

// Optional late TIMEOUT (after PARAM / BODY SCHEMA)
rest_upper = StringUtil::Upper(rest);
if (StringUtil::StartsWith(rest_upper, "TIMEOUT") && (rest.size() == 7 || StringUtil::CharacterIsSpace(rest[7]))) {
Expand Down Expand Up @@ -1014,6 +1111,7 @@ ParserExtensionParseResult RouteDdlParse(ParserExtensionInfo *, const string &qu
data->route.empty_status = empty_status;
data->route.empty_body = empty_body;
data->route.timeout_sec = timeout_sec;
data->route.schedule = schedule;
return ParserExtensionParseResult(std::move(data));
}

Expand Down Expand Up @@ -1082,6 +1180,9 @@ unique_ptr<FunctionData> ApplyRouteBind(ClientContext &, TableFunctionBindInput
if (input.inputs.size() > 18 && !input.inputs[18].IsNull()) {
bind_data->route.timeout_sec = input.inputs[18].GetValue<int32_t>();
}
if (input.inputs.size() > 20 && !input.inputs[20].IsNull()) {
bind_data->route.schedule = input.inputs[20].GetValue<string>();
}
BindStatusColumn(return_types, names);
return std::move(bind_data);
}
Expand Down Expand Up @@ -1112,6 +1213,10 @@ void ApplyRouteExec(ClientContext &context, TableFunctionInput &data_p, DataChun
} else if (bind_data.route.pattern.empty() || bind_data.route.pattern[0] != '/') {
throw InvalidInputException("Route pattern must start with '/'");
}
if (!bind_data.route.schedule.empty() && HasRoutePathParams(bind_data.route.pattern)) {
throw InvalidInputException(
"Scheduled route \"%s\" cannot have path parameters; use a parameter-free route", bind_data.route.name);
}
// Validate the handler SQL now so a broken route fails at CREATE time,
// not at first request. Do this BEFORE mutating the registry so
// CREATE OR REPLACE does not leave a half-applied route on failure.
Expand Down Expand Up @@ -1184,14 +1289,15 @@ void ApplyRouteExec(ClientContext &context, TableFunctionInput &data_p, DataChun
TableFunction MakeApplyRouteFunction() {
// action, or_replace, name, method, pattern, handler, status, require_auth,
// params_json, body_schema, group_name, rate_n, rate_per, rate_by, format,
// envelope, empty_status, empty_body, timeout_sec, body_type
return MakeApplyDdlFunction(
"quackapi_apply_route",
{LogicalType::VARCHAR, LogicalType::BOOLEAN, LogicalType::VARCHAR, LogicalType::VARCHAR, LogicalType::VARCHAR,
LogicalType::VARCHAR, LogicalType::INTEGER, LogicalType::VARCHAR, LogicalType::VARCHAR, LogicalType::VARCHAR,
LogicalType::VARCHAR, LogicalType::INTEGER, LogicalType::INTEGER, LogicalType::VARCHAR, LogicalType::VARCHAR,
LogicalType::VARCHAR, LogicalType::INTEGER, LogicalType::VARCHAR, LogicalType::INTEGER, LogicalType::VARCHAR},
ApplyRouteExec, ApplyRouteBind);
// envelope, empty_status, empty_body, timeout_sec, body_type, schedule
return MakeApplyDdlFunction("quackapi_apply_route",
{LogicalType::VARCHAR, LogicalType::BOOLEAN, LogicalType::VARCHAR, LogicalType::VARCHAR,
LogicalType::VARCHAR, LogicalType::VARCHAR, LogicalType::INTEGER, LogicalType::VARCHAR,
LogicalType::VARCHAR, LogicalType::VARCHAR, LogicalType::VARCHAR, LogicalType::INTEGER,
LogicalType::INTEGER, LogicalType::VARCHAR, LogicalType::VARCHAR, LogicalType::VARCHAR,
LogicalType::INTEGER, LogicalType::VARCHAR, LogicalType::INTEGER, LogicalType::VARCHAR,
LogicalType::VARCHAR},
ApplyRouteExec, ApplyRouteBind);
}

ParserExtensionPlanResult RouteDdlPlan(ParserExtensionInfo *, ClientContext &,
Expand Down Expand Up @@ -1219,6 +1325,7 @@ ParserExtensionPlanResult RouteDdlPlan(ParserExtensionInfo *, ClientContext &,
result.parameters.push_back(Value(data.route.empty_body));
result.parameters.push_back(Value::INTEGER(data.route.timeout_sec));
result.parameters.push_back(Value(data.route.body_type));
result.parameters.push_back(Value(data.route.schedule));
FinishDdlPlan(result);
return result;
}
Expand Down
3 changes: 3 additions & 0 deletions src/quackapi_extension.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -783,6 +783,8 @@ static unique_ptr<FunctionData> RoutesBind(ClientContext &, TableFunctionBindInp
names.emplace_back("empty_status");
return_types.emplace_back(LogicalType::INTEGER);
names.emplace_back("timeout_sec");
return_types.emplace_back(LogicalType::VARCHAR);
names.emplace_back("schedule");
return make_uniq<RoutesBindData>();
}

Expand All @@ -809,6 +811,7 @@ static void RoutesExec(ClientContext &, TableFunctionInput &data_p, DataChunk &o
output.SetValue(9, row, Value(route.response_envelope.empty() ? "array" : route.response_envelope));
output.SetValue(10, row, Value::INTEGER(route.empty_status));
output.SetValue(11, row, Value::INTEGER(route.timeout_sec));
output.SetValue(12, row, route.schedule.empty() ? Value() : Value(route.schedule));
row++;
state.offset++;
}
Expand Down
22 changes: 22 additions & 0 deletions test/sql/quackapi_group.test
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,28 @@ SELECT * FROM quackapi_add_api_key('other', 'route-other-key', 'other-user');
statement ok
CREATE GROUP v1 WITH (prefix='/api/v1', auth=api);

# Scheduling belongs to routes, not groups.
statement error
CREATE GROUP scheduled WITH (prefix='/scheduled', schedule='*/5 * * * * *');
----
Unknown GROUP option

# A grouped route may own a schedule after its prefix is expanded.
query I
CREATE ROUTE scheduled_group GET '/scheduled' GROUP v1 SCHEDULE '*/5 * * * * *' AS SELECT 'ok' AS msg;
----
Route scheduled_group: GET /api/v1/scheduled

query II
SELECT pattern, schedule FROM quackapi_routes() WHERE name = 'scheduled_group';
----
/api/v1/scheduled */5 * * * * *

query I
DROP ROUTE scheduled_group;
----
Dropped route scheduled_group

# Member with no REQUIRE — inherits group auth; path joins prefix
statement ok
CREATE ROUTE items_list GET '/items' GROUP v1 AS
Expand Down
50 changes: 50 additions & 0 deletions test/sql/quackapi_routes.test
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,56 @@ SELECT pattern FROM quackapi_routes() WHERE name = 'hello';
----
/hello2

# SCHEDULE stores a six-field cron expression (seconds first); absent is NULL
query I
CREATE ROUTE scheduled GET '/scheduled' SCHEDULE '*/5 * * * * *' AS SELECT 'ok' AS msg;
----
Route scheduled: GET /scheduled

query III
SELECT name, schedule, schedule IS NULL FROM quackapi_routes() WHERE name IN ('hello', 'scheduled') ORDER BY name;
----
hello NULL true
scheduled */5 * * * * * false

# SCHEDULE requires exactly six fields
statement error
CREATE ROUTE bad_schedule GET '/bad-schedule' SCHEDULE '*/5 * * * *' AS SELECT 1;
----
6 whitespace-separated cron fields

# Scheduled routes cannot require a path parameter
statement error
CREATE ROUTE scheduled_param GET '/scheduled/:id' SCHEDULE '*/5 * * * * *' AS SELECT $id::INTEGER AS id;
----
cannot have path parameters

# OR REPLACE changes and then removes the schedule
query I
CREATE OR REPLACE ROUTE scheduled GET '/scheduled' SCHEDULE '0 0 1 * * *' AS SELECT 'changed' AS msg;
----
Route scheduled: GET /scheduled

query I
SELECT schedule FROM quackapi_routes() WHERE name = 'scheduled';
----
0 0 1 * * *

query I
CREATE OR REPLACE ROUTE scheduled GET '/scheduled' AS SELECT 'unscheduled' AS msg;
----
Route scheduled: GET /scheduled

query I
SELECT schedule IS NULL FROM quackapi_routes() WHERE name = 'scheduled';
----
true

query I
DROP ROUTE scheduled;
----
Dropped route scheduled

# Path params and STATUS clause parse
query I
CREATE ROUTE item GET '/items/:id' STATUS 201 AS SELECT $id::INT AS id;
Expand Down
Loading
Loading