Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ IMAGE_TAG=latest
# Server-local IANA timezone used for statistics day boundaries.
TZ=America/New_York
# Existing network hostname that coworkers use to reach this machine.
LABEL_CHECK_HOSTNAME=replace-with-machine-hostname
INSLIDE_HOSTNAME=replace-with-machine-hostname
PIPELINE_MAX_WORKERS=8
PIPELINE_MAX_THUMBNAIL_DIMENSION=4096
LOGIN_PAIR_ATTEMPT_LIMIT=5
Expand All @@ -14,8 +14,8 @@ LOGIN_RATE_WINDOW_SECONDS=900
SESSION_COOKIE_SECURE=true
# Inside-container allowlists separated by colon. Input/output paths submitted to
# pipeline UI or API must resolve beneath these roots.
PIPELINE_INPUT_ROOTS=/data/gt450-images:/data/label-check-batches
PIPELINE_OUTPUT_ROOTS=/data/label-check-batches
PIPELINE_INPUT_ROOTS=/data/gt450-images:/data/inslide-batches
PIPELINE_OUTPUT_ROOTS=/data/inslide-batches

# Generate long random values. SECRET_KEY must be at least 32 characters and
# ADMIN_DEFAULT_PASSWORD at least 12. Placeholder values are rejected. Do not
Expand All @@ -37,11 +37,11 @@ GT450_IMAGES_HOST_PREFIX=\\chp.clarian.org\app\Philips_Slide_Images\GT450_images

# Docker Desktop must have permission to share each local Windows path below.
SCANNER_INVENTORIES_HOST=D:\scanner_inventories
LABEL_CHECK_BATCHES_HOST=D:\label_check_batches
LABEL_CHECK_BATCHES_HOST_PREFIX=D:\label_check_batches
INSLIDE_BATCHES_HOST=D:\label_check_batches
INSLIDE_BATCHES_HOST_PREFIX=D:\label_check_batches
IMAGE_STAGING_HOST=D:\image_staging
COPATH_CLONE_HOST=D:\copath_clone
LABEL_CHECK_STATE_HOST=D:\label_check_batches\state
INSLIDE_STATE_HOST=D:\label_check_batches\state
BATCH_CATALOG_RECONCILE_SECONDS=60
TQ_HOME_HOST=D:\label-check-tq
SSH_HOME_HOST=D:\label-check-ssh
Expand Down
22 changes: 11 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ FROM python:${PYTHON_VERSION}-slim-bookworm AS python-base
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
LABEL_CHECK_CONTAINER=true \
INSLIDE_CONTAINER=true \
EASYOCR_FORCE_CPU=true \
EASYOCR_MODEL_DIR=/opt/easyocr-models \
HOME=/home/labelcheck \
TQ_HOME_DIR=/home/labelcheck/.tq
HOME=/home/inslide \
TQ_HOME_DIR=/home/inslide/.tq

RUN apt-get update \
&& apt-get install --yes --no-install-recommends \
Expand Down Expand Up @@ -92,31 +92,31 @@ FROM python-base AS runtime

COPY --from=rust-builder /build/tq/target/release/tq /app/bin/tq

RUN groupadd --gid 10001 labelcheck \
&& useradd --uid 10001 --gid labelcheck --create-home labelcheck \
&& mkdir -p /data/state/instance /home/labelcheck/.ssh /home/labelcheck/.tq \
RUN groupadd --gid 10001 inslide \
&& useradd --uid 10001 --gid inslide --create-home inslide \
&& mkdir -p /data/state/instance /home/inslide/.ssh /home/inslide/.tq \
&& sha256sum /app/bin/tq > /app/bin/tq.sha256 \
&& chown -R labelcheck:labelcheck /app /data/state /home/labelcheck \
&& chown -R inslide:inslide /app /data/state /home/inslide \
&& sh -c '/app/bin/tq >/dev/null 2>&1; test "$?" -eq 1'

ENV TQ_EXECUTABLE=/app/bin/tq \
INSTANCE_DIR=/data/state/instance \
SDL_FILE_PATH=/data/state/Slide_Digitization_Log.xlsx \
BACKUP_DIR=/data/state/csv_backups \
SCANNER_INVENTORIES=/data/scanner-inventories \
LABEL_CHECK_BATCHES=/data/label-check-batches \
INSLIDE_BATCHES=/data/inslide-batches \
IMAGE_STAGING_ROOT=/data/image-staging \
COPATH_CLONE=/data/copath-clone \
TQ_TRANSFER_LOG_DIR=/data/label-check-batches/transfer_logs \
TQ_TRANSFER_LOG_DIR=/data/inslide-batches/transfer_logs \
GT450_IMAGES_CONTAINER_ROOT=/data/gt450-images \
LABEL_CHECK_BATCHES_CONTAINER_ROOT=/data/label-check-batches \
INSLIDE_BATCHES_CONTAINER_ROOT=/data/inslide-batches \
COPATH_QUERY_MODE=windows_queue \
COPATH_QUERY_QUEUE=/data/state/copath-query \
COPATH_QUERY_TIMEOUT_SECONDS=300 \
PORT=5000

WORKDIR /app/src
USER labelcheck
USER inslide
EXPOSE 5000

HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
Expand Down
91 changes: 49 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Label-Check
# InSlide
A label corrector

## Pipeline API

The versioned pipeline API is available under `/api/v1` and requires HTTPS plus a
scoped personal access token. Create a token for an existing Label-Check user:
scoped personal access token. Create a token for an existing InSlide user:

```bash
flask --app src/app.py api-token create USERNAME --label "integration name"
Expand All @@ -17,8 +17,8 @@ manage credentials.
Submit a job using server-visible input and output paths:

```bash
curl --request POST https://label-check.example/api/v1/pipeline/jobs \
--header "Authorization: Bearer $LABEL_CHECK_TOKEN" \
curl --request POST https://inslide.example/api/v1/pipeline/jobs \
--header "Authorization: Bearer $INSLIDE_TOKEN" \
--header "Content-Type: application/json" \
--header "Idempotency-Key: unique-client-request-id" \
--data '{"input_dir":"/data/incoming","output_dir":"/data/output"}'
Expand All @@ -39,8 +39,8 @@ the proxy over plaintext HTTP.
Pipeline paths are restricted after path translation and symbolic-link
resolution. `PIPELINE_INPUT_ROOTS` and `PIPELINE_OUTPUT_ROOTS` contain
platform-path-separator-delimited allowlists. Compose permits input beneath
`/data/gt450-images` or `/data/label-check-batches` and output beneath
`/data/label-check-batches`. Worker fields default to a maximum of 8 and
`/data/gt450-images` or `/data/inslide-batches` and output beneath
`/data/inslide-batches`. Worker fields default to a maximum of 8 and
thumbnail dimensions to 4096 pixels; deployments can lower these limits with
`PIPELINE_MAX_WORKERS` and `PIPELINE_MAX_THUMBNAIL_DIMENSION`.

Expand All @@ -57,7 +57,7 @@ Cargo nor GCC.
- Repository cloned with its pinned TQ submodule:

```powershell
git clone --recurse-submodules <label-check-repository-url>
git clone --recurse-submodules <inslide-repository-url>
```

- Windows directories shared with Docker Desktop.
Expand All @@ -79,16 +79,16 @@ Cargo nor GCC.
network policy.

Copy `.env.example` to `.env`, replace all placeholders, and create the host
directories. `LABEL_CHECK_STATE_HOST` must contain
directories. `INSLIDE_STATE_HOST` must contain
`Slide_Digitization_Log.xlsx` before SDL workflows run.

Batch workflow stages, queues, and leases are stored in
`LABEL_CHECK_STATE_HOST\instance\batch_catalog.sqlite3`. Batch list pages query
`INSLIDE_STATE_HOST\instance\batch_catalog.sqlite3`. Batch list pages query
this catalog and load `enriched.csv` only after a batch is selected. A
background reconciliation scans for externally-created batches at startup and
every `BATCH_CATALOG_RECONCILE_SECONDS` seconds (60 by default).

Before the first catalog-aware deployment, stop the Label-Check container and
Before the first catalog-aware deployment, stop the InSlide container and
validate legacy state from a PowerShell prompt:

```powershell
Expand Down Expand Up @@ -125,7 +125,7 @@ application when the trusted proxy reports an HTTPS request.
Compose uses Caddy's internal certificate authority because the organization
does not supply the deployment certificate. Each authorized client must trust
the generated root certificate before using the app. Protect and back up the
`label-check-caddy-data` volume: it contains the local CA private key. Deleting
`inslide-caddy-data` volume: it contains the local CA private key. Deleting
that volume creates a new CA and requires every client to trust the replacement.

New user passwords must contain 12–128 characters. Login failures are stored in
Expand All @@ -137,7 +137,7 @@ counters. The limits and window are configurable through the corresponding
Application startup creates runtime directories with mode `0700`, files with
mode `0600`, and repairs existing instance state before loading it. Symbolic
links in sensitive instance state are rejected. On Windows bind mounts, NTFS
ACLs remain the security boundary: restrict `LABEL_CHECK_STATE_HOST` to the
ACLs remain the security boundary: restrict `INSLIDE_STATE_HOST` to the
service account, Docker Desktop service account, and administrators.

User activity is accumulated in `/data/state/instance/statistics.sqlite3`.
Expand All @@ -152,10 +152,17 @@ out-of-band channel. After verifying it, place the corresponding OpenSSH
`known_hosts` entry in `${SSH_HOME_HOST}\known_hosts`. `ssh-keyscan` output must
not be trusted until its fingerprint has been independently verified.

Only Label-Check administrators can view or edit the global TQ connection
Only InSlide administrators can view or edit the global TQ connection
configuration. Authenticated operators can continue transferring approved
slides through that administrator-managed destination.

Administrators can open the configuration directly from **Transfers > Edit
Config**. The container reads `${TQ_HOME_HOST}\config.toml` through its
`/home/inslide/.tq/config.toml` bind mount. Write Windows paths as TOML literal
strings, for example `source = 'D:\image_staging'`. InSlide automatically
repairs unambiguous Windows-path escaping during saves and transfer preflight;
each repair retains a timestamped `config.toml.bak-*` copy.

The GT450 CIFS password is stored in the gitignored `.env` file and in local
Docker volume metadata. Restrict Docker access to trusted administrators and do
not reuse a personal account. Because CIFS mount options are comma-delimited,
Expand All @@ -175,7 +182,7 @@ does not contain the Windows username or password.

```powershell
git submodule update --init --recursive
docker build --target test --tag label-check:test .
docker build --target test --tag inslide:test .
docker compose build
```

Expand All @@ -195,7 +202,7 @@ EasyOCR is CPU-only and its English models are baked into the image. Obtain the
Linux binary hash when needed with:

```powershell
docker run --rm --entrypoint sha256sum label-check:latest /app/bin/tq
docker run --rm --entrypoint sha256sum inslide:latest /app/bin/tq
```

### Paths
Expand All @@ -204,20 +211,20 @@ Inside the container, Windows resources appear at stable Linux paths:

- GT450 images: `/data/gt450-images`
- scanner inventories: `/data/scanner-inventories`
- label-check batches: `/data/label-check-batches`
- Label-Check pipeline batches: `/data/inslide-batches`
- deidentified transfer staging: `/data/image-staging`
- CoPath clone: `/data/copath-clone`
- persistent application state: `/data/state`

Persisted UNC GT450 paths and `D:\label_check_batches` paths are translated to
these mounts. New pipeline output records Linux mount paths directly.

The GT450 mount is the named Docker volume `label-check-gt450-images`. It mounts
The GT450 mount is the named Docker volume `inslide-gt450-images`. It mounts
`//chp.clarian.org/app/Philips_Slide_Images/GT450_Images` through CIFS with
read-only permissions. The remaining paths are ordinary Windows bind mounts.

Create and share `IMAGE_STAGING_HOST` (normally `D:\image_staging`) with Docker
Desktop. Before each TQ upload, Label-Check copies selected GT450 slides to
Desktop. Before each TQ upload, InSlide copies selected GT450 slides to
`<IMAGE_STAGING_HOST>\<destination directory>\<renamed slide>.svs`, removes the
identifying label and macro from those copies, then gives the staged slide paths
and a generated metadata CSV to TQ. Each transfer uploads that CSV as
Expand All @@ -234,9 +241,9 @@ Docker Desktop service account.

CoPath queries are delegated to a worker that you start after signing into
Windows. Docker and the worker communicate only through
`LABEL_CHECK_STATE_HOST\copath-query`; no SQL credentials enter the container.
`INSLIDE_STATE_HOST\copath-query`; no SQL credentials enter the container.
The queue contains accessions and report data. Restrict the entire
`LABEL_CHECK_STATE_HOST` directory to the signed-in Windows account, the Docker
`INSLIDE_STATE_HOST` directory to the signed-in Windows account, the Docker
Desktop service account, and administrators. Do not share it broadly or use a
world-writable network directory.

Expand Down Expand Up @@ -267,7 +274,7 @@ batch:

```powershell
.\.venv-copath-worker\Scripts\python.exe src\copath_windows_worker.py `
--queue "$env:LABEL_CHECK_STATE_HOST\copath-query" `
--queue "$env:INSLIDE_STATE_HOST\copath-query" `
--connection-string-file "$env:COPATH_CONNECTION_STRING_FILE_HOST"
```

Expand Down Expand Up @@ -313,24 +320,24 @@ docker compose up -d
docker compose ps
```

Set `LABEL_CHECK_HOSTNAME` in `.env` to the machine's existing network hostname.
Set `INSLIDE_HOSTNAME` in `.env` to the machine's existing network hostname.
After the first startup, export Caddy's root CA certificate to the current
user's Downloads directory:

```powershell
docker compose cp caddy:/data/caddy/pki/authorities/local/root.crt `
"$env:USERPROFILE\Downloads\label-check-local-ca.crt"
"$env:USERPROFILE\Downloads\inslide-local-ca.crt"
```

On the host and each authorized coworker's Windows device, import
`label-check-local-ca.crt` into **Trusted Root Certification Authorities** for
`inslide-local-ca.crt` into **Trusted Root Certification Authorities** for
the current user. Treat the certificate as trusted software and distribute it
through an authenticated channel. Do not distribute anything else from the
Caddy data volume, especially `root.key`.

```powershell
Import-Certificate `
-FilePath "$env:USERPROFILE\Downloads\label-check-local-ca.crt" `
-FilePath "$env:USERPROFILE\Downloads\inslide-local-ca.crt" `
-CertStoreLocation Cert:\CurrentUser\Root
```

Expand All @@ -341,13 +348,13 @@ Confirm that the hostname resolves to the Docker host and that HTTPS is
reachable, then open the app:

```powershell
$labelCheckHostname = "replace-with-configured-hostname"
Resolve-DnsName $labelCheckHostname
Test-NetConnection $labelCheckHostname -Port 443
Start-Process "https://$labelCheckHostname"
$inSlideHostname = "replace-with-configured-hostname"
Resolve-DnsName $inSlideHostname
Test-NetConnection $inSlideHostname -Port 443
Start-Process "https://$inSlideHostname"
```

Use the exact hostname configured in `LABEL_CHECK_HOSTNAME`; an IP address or a
Use the exact hostname configured in `INSLIDE_HOSTNAME`; an IP address or a
different alias will fail certificate validation. Port 80 is intentionally not
published, so include `https://` in the URL. If name resolution or inbound port
443 is blocked, ask IT to add the hostname/firewall allowance. Application port
Expand All @@ -357,20 +364,20 @@ Verify that the SMB mount contains the expected scanner directories and is
read-only for the application user:

```powershell
docker compose exec label-check ls -la /data/gt450-images
docker inspect $(docker compose ps -q label-check) `
docker compose exec inslide ls -la /data/gt450-images
docker inspect $(docker compose ps -q inslide) `
--format '{{range .Mounts}}{{if eq .Destination "/data/gt450-images"}}{{println "RW:" .RW "Name:" .Name}}{{end}}{{end}}'
```

The inspection output must report `RW: false` and
`Name: label-check-gt450-images`.
`Name: inslide-gt450-images`.

Docker volume options are fixed when the volume is created. After changing the
SMB password or any `GT450_SMB_*` mount setting, recreate only this mount:

```powershell
docker compose down
docker volume rm label-check-gt450-images
docker volume rm inslide-gt450-images
docker compose up -d
```

Expand All @@ -381,13 +388,13 @@ The default command initializes persistent state and starts Waitress on port
5000. Other applications use the same image:

```powershell
docker compose run --rm label-check pipeline `
docker compose run --rm inslide pipeline `
--input-dir /data/gt450-images/SS12797 `
--output-dir /data/label-check-batches/SS12797/2026-07-31 `
--output-dir /data/inslide-batches/SS12797/2026-07-31 `
--end-at name --ocr-use-cpu

docker compose run --rm label-check nightly
docker compose run --rm label-check python /app/src/deidentify_anonymize.py --help
docker compose run --rm inslide nightly
docker compose run --rm inslide python /app/src/deidentify_anonymize.py --help
```

Schedule `nightly` externally; it performs one cycle and exits. State, SDL,
Expand All @@ -399,12 +406,12 @@ backups, TQ configuration, and transfer logs survive container replacement.
account credentials and domain, SMB 3.0 connectivity, and access to the
`app` share. Docker should fail container startup on a mount error instead of
substituting an empty local directory. After changing mount options, remove
and recreate `label-check-gt450-images` as described above.
and recreate `inslide-gt450-images` as described above.

### CoPath troubleshooting

- **Worker offline:** confirm the worker console is still running and that both
Windows and Docker can access `LABEL_CHECK_STATE_HOST\copath-query`. Check the
Windows and Docker can access `INSLIDE_STATE_HOST\copath-query`. Check the
Windows clock if `worker.json` is present but considered stale.
- **Query timeout:** the default is 300 seconds. Inspect the worker console and
SQL connectivity before increasing `COPATH_QUERY_TIMEOUT_SECONDS`.
Expand All @@ -425,9 +432,9 @@ direct-mode override:

```powershell
docker compose -f compose.yaml -f compose.direct.yaml up -d
docker compose -f compose.yaml -f compose.direct.yaml exec label-check `
docker compose -f compose.yaml -f compose.direct.yaml exec inslide `
kinit YOUR_USERNAME@YOUR.AD.REALM
docker compose -f compose.yaml -f compose.direct.yaml exec label-check klist
docker compose -f compose.yaml -f compose.direct.yaml exec inslide klist
```

In direct mode, use the SQL Server DNS name associated with its `MSSQLSvc`
Expand Down
4 changes: 2 additions & 2 deletions compose.direct.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
services:
label-check:
inslide:
environment:
COPATH_QUERY_MODE: direct
COPATH_CONNECTION_STRING_FILE: /run/secrets/copath_connection_string
KRB5_CONFIG: /etc/krb5.conf
KRB5CCNAME: /tmp/krb5cc_labelcheck
KRB5CCNAME: /tmp/krb5cc_inslide
volumes:
- type: bind
source: ${KRB5_CONFIG_HOST:?Set KRB5_CONFIG_HOST to the domain Kerberos configuration file}
Expand Down
Loading