Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions docs/adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,12 +292,43 @@ spreadsheet = info.spreadsheet # already opened, no need to open it again
an unshared file as "not found", so on a Drive 404 the helper asks the Sheets
API, which distinguishes "not shared" (403) from "does not exist" (404). Any
other Google error propagates unchanged as `gspread.exceptions.APIError`.
Network failures (`requests.exceptions.RequestException`, such as a timeout)
and access-token refresh failures (`google.auth.exceptions.GoogleAuthError`)
are not Google API errors, and propagate unwrapped from `open_gsheet()` and
`inspect_gsheet()` alike.
- `GSheetInfo` is a frozen dataclass: `spreadsheet`, `file_id`, `title`, `url`,
`mimetype`, `can_edit` and `service_account_email`. Read-only access is data,
not an error, because reading a Viewer-shared sheet is legitimate;
`require_writable()` raises `SpreadsheetReadOnlyError` for the callers that
will write.

To check a sheet without opening it for use (for example, to show a "sheet is
ready" status before a selection), use `inspect_gsheet()` instead. It raises the
same errors, and also returns the tab titles, using one Drive request and one
Sheets request. Opening with `open_gsheet()` and then calling `worksheets()`
costs two Sheets requests. That matters because Google allows a service account
only 60 Sheets read requests per minute, shared by everything that uses it.

```python
from sortition_algorithms import GSheetSummary, inspect_gsheet

summary: GSheetSummary = inspect_gsheet(client, "https://docs.google.com/spreadsheets/d/.../edit")
if "Respondents" not in summary.worksheet_titles:
...
```

- `GSheetSummary` is a frozen dataclass: `file_id`, `title`, `url`, `mimetype`,
`can_edit`, `worksheet_titles` (a tuple, including hidden tabs),
`fetched_at` and `service_account_email`. It has no `spreadsheet`, so use
`open_gsheet()` when you need to read or write.
- `fetched_at` is a timezone-aware UTC `datetime` taken when the Sheets
response arrived. If you keep a summary around, use it to decide when to
check again. It is ignored when comparing summaries, so two summaries of an
unchanged sheet are equal.
- If Drive allows the file but the Sheets API refuses it (for example, the
Sheets API is not enabled for the Google Cloud project), the error propagates
as `gspread.exceptions.APIError`.

All four errors are `SelectionError` subclasses with an `error_code`
(`spreadsheet_not_found`, `spreadsheet_not_shared`, `spreadsheet_read_only`,
`not_native_gsheet`) and `error_params` for translation - see the
Expand All @@ -309,6 +340,21 @@ Celery failure callback.
property, `get_title()` and the `read_*` methods raise the errors above for every
access failure.

#### How often `GSheetDataSource` reads the tab list

To stay well inside the 60 reads per minute, `GSheetDataSource` fetches the
spreadsheet's tab list once (one Sheets request) and reuses it to find the
input tabs and to choose names for the output tabs. Tabs it creates or deletes
itself are kept in step with the list.

The list is fetched again before use once it is older than
`data_source.tab_cache_max_age_seconds` (60 by default), because people may add
or remove tabs while a slow selection runs. A fast selection therefore reads the
tab list once, and a slow one twice. If an input tab is missing from the list,
the list is fetched once more before the "tab not found" error is raised, in
case someone has just fixed the tab name. Set `tab_cache_max_age_seconds = 0`
to fetch the list every time it is used.

## Writing custom Data Source classes

You can create custom data source classes for other data sources like Excel files, SQL databases, or APIs.
Expand Down
Loading
Loading