Skip to content

Use latest Hotdata API: inline Arrow results, pagination, real error messages - #66

Merged
eddietejeda merged 1 commit into
mainfrom
api-latest-arrow-pagination
Oct 24, 2026
Merged

eddietejeda merged 1 commit into
mainfrom
api-latest-arrow-pagination

Conversation

@eddietejeda

@eddietejeda eddietejeda commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Aligns the backend client with the current Hotdata API reference (https://www.hotdata.dev/docs/api-reference).

  • Inline Arrow results: POST /v1/query sends Accept: application/vnd.apache.arrow.stream. Fast queries return Arrow in one request (metadata from X-Query-Run-Id / X-Result-Id / X-Execution-Time-Ms headers). A 202 with X-Result-Id (result too large) reads from the Results API without polling the run; a plain 202 still polls the run.
  • Real error messages: read error_message from failed query runs and failed results (was reading error/message, so users saw only "query run failed"). Error bodies {"error":{"code","message"}} now render as message (CODE) instead of raw JSON.
  • Pagination: follow has_more / next_cursor on /v1/databases and /v1/information_schema. Before, the database picker and autocomplete only saw the first page.
  • Results API status codes: handle 202 + Retry-After while persisting, and 409 on failure.
  • Health check: GET /v1/databases?limit=1 instead of listing every database.

Testing

  • go test ./pkg/... — new tests for inline Arrow, too-large 202, failed run error_message, JSON fallback, pagination, error message parsing.
  • Ran the client against the live API: inline Arrow query, result refetch, bad-table and bad-result errors, full database pagination.
  • No frontend or plugin.json changes.

…_message

- Request Arrow from POST /v1/query so fast queries take one round trip; fall back to the Results API on 202 (async or too large).
- Read error_message from failed query runs and results; show code/message from Hotdata error bodies.
- Follow next_cursor pagination on /v1/databases and /v1/information_schema.
- Handle Results API 202 (Retry-After) and 409 responses.
- Health check pings GET /v1/databases?limit=1.
@eddietejeda
eddietejeda requested a review from a team as a code owner October 24, 2026 22:31
@eddietejeda
eddietejeda requested review from anoop-narang and removed request for a team October 24, 2026 22:31
Comment thread pkg/hotdata/client.go
}
q.Set("cursor", p.NextCursor)
}
return nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Return an error or log a warning when listPages exits at maxPages with has_more still true (not blocking).

At the cap, the function returns nil, so callers get a partial list with no signal. ListDatabases stops at 10,000 databases. InformationSchema sends no limit, so its cap depends on the server's default page size and can be much lower. The database picker and autocomplete then hide the missing entries without any error.

@eddietejeda
eddietejeda merged commit 4e2f48e into main Oct 24, 2026
11 checks passed
@eddietejeda
eddietejeda deleted the api-latest-arrow-pagination branch October 24, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant