You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reworks the /web UI of openenv harbor serve (and of Spaces made with openenv harbor push) into a task explorer and rollout viewer: browse and filter every served task, read exactly what the agent receives, run an agent with the server's model, a Hugging Face Inference Providers model or any OpenAI-compatible/Anthropic endpoint, then read what it did. It also adds deployment settings, so the same UI is safe to run as a public Space: who may start rollouts, whose endpoint they use, who sees which runs, and whether datasets can be added from the page.
The first three commits are the change, reviewable one at a time; the commits after them address review (see Behaviour changes):
fix(harbor): find grouped task folders; keep probed engines off public /health: small changes to shared code.
Task discovery finds datasets that group tasks by field (tasks/<field>/<subfield>/<task>/, e.g. terminal-bench-science). The nested search runs only when no top-level folder holds a task.toml, so flat datasets keep exactly the order, and so the indexes, they had.
resolve_task_dirs(..., tqdm_class=) for callers that show download progress.
UpstreamPool.forget(upstream) drops a cached engine client, and the key it holds.
/health lists probed per-session engines only to the admin key.
feat(harbor): rework the serve web UI and make it safe to deploy publicly: the UI.
docs(harbor): document the web UI, its deployment settings and new flags: README plus the regenerated docs page.
The UI
Tab
What it does
Tasks
A card grid with search and facets (dataset, category, difficulty, tags). The task page has the instruction the agent receives, the task's files in a viewer with a full-window mode, the environment and verifier settings, and the task's runs. Answer-like metadata (gold_answer, solution, …) is left out of the summary. Add from the Hub lists public harbor-tagged datasets with task count and size, and adds one in the background with progress.
Run card
Three model sources. This server: its key stays in the capture proxy. Hugging Face: a searchable Inference Providers model list, with this machine's token locally or optional "sign in with Hugging Face" on a Space. Your endpoint: any OpenAI-compatible or Anthropic URL, probed before use.
Runs
Status filters and compare for 2–4 runs. A run page shows a timeline of the prompt, thinking, tool calls with their outputs, and the final answer, plus result, reward and trace checks. It also has downloads of the result JSON and the training contract.
Setup
The endpoint, which sandboxes have credentials, the deployment settings, datasets and agents.
Deployment settings
Defaults depend on where the server runs: local means --host 127.0.0.1; network means any other bind address; Space is detected from SPACE_ID. Each setting is an env var, and most have a serve/push flag.
setting
variable
flag
local
network
Space
Visitors may start rollouts
OPENENV_HARBOR_UI_ROLLOUTS
--rollouts
on
on
on
…use the server's endpoint
OPENENV_HARBOR_UI_SERVER_ENDPOINT
--share-endpoint
on
on
off
…connect their own
OPENENV_HARBOR_UI_VISITOR_ENDPOINTS
--visitor-endpoints
on
on
on
Visitor URL may be private/local
OPENENV_HARBOR_UI_PRIVATE_URLS
--private-urls (serve only)
on
off
off
Offer this machine's HF token
OPENENV_HARBOR_UI_LOCAL_TOKEN
on
off
never
Add/remove Hub datasets
OPENENV_HARBOR_UI_ADD_DATASETS
--add-datasets
on
off
off
Who sees runs
OPENENV_HARBOR_RUN_VISIBILITY
--run-visibility
all
all
own
Keep runs across restarts
OPENENV_HARBOR_RUN_HISTORY
--run-history
on
on
off
Rollouts at once (per visitor)
OPENENV_HARBOR_UI_MAX_RUNS(_PER_VISITOR)
4 (4)
4 (4)
4 (2)
push also changes in five ways:
--llm-url is optional; without it, visitors bring their own model.
UI visitors don't run on the Space's endpoint unless --share-endpoint is given; the Task API and MCP use it either way.
UI flags left out of a push are removed from the Space, so they return to the Space defaults rather than to what an earlier push set.
The bucket is private by default. --public-bucket/--private-bucket sets it, and an existing bucket keeps its visibility unless one is given. Only a 404 counts as a missing bucket.
--hf-login (default on) adds hf_oauth: true with the inference-api scope.
With --add-datasets on a Space, an added dataset is copied into the Space's bucket server side (copy_files, no download) and read through the /data mount, so it survives restarts. Removing it deletes it from the bucket.
Security
This became a large part of the work once the UI could run publicly:
Visitor URLs: a URL, and every redirect it answers with, must resolve to public addresses unless private URLs are allowed. This covers IPv4-mapped/embedded IPv6 forms. The URL is checked again before each rollout.
Keys: a key typed into the page is held in server memory for that page only, never written to disk, run history or back to the page. It is forget-ed from the capture proxy's pool once no run uses it.
The server's environment: a task that uses ${VAR} templating in task.toml/compose, which is where the server's keys live, or reaches the host through compose, runs from the UI only on the server's own endpoint. Reaching the host covers env_file, include, extends, and host paths in mounts, builds, caches and watch rules. It also covers privileged, cap_add, host namespaces, the Docker socket, other containers' volumes, named volumes or networks with settings (the local driver binds any path), and a build's SSH agent. A model a visitor connects could print the sandbox's environment into a trace that visitor reads. Both files are checked as parsed. On --share-endpoint, such a served task does run on the operator's model, and the visitor who starts it reads its trace; the README says not to combine the two on a public Space.
Added datasets: these come from the Hub only (no local paths) and are size-capped; one the Hub reports no size (or 0) for is measured from its file listing, or refused. Their tasks may not read the server's environment at all, and one that contains a symbolic link is refused and removed; the page never reads task.toml or instruction.md through a link. Grouped task discovery doesn't follow symlinks.
Per-visitor limit: counts a signed-in visitor's HF account, otherwise the browser id, so for anonymous visitors it's a fairness limit; OPENENV_HARBOR_UI_MAX_RUNS bounds the total.
Browser requests: every dataset, task and file path the browser sends is checked against what the server serves. Every file the page reads (cards, task view, file viewer, environment check) must resolve inside the dataset's own folder, so a link to anywhere else, at any level, is not followed. Downloads need a per-page grant.
Cross-site requests: a state-changing request to the UI (/web) that a browser sends from another site is refused (Sec-Fetch-Site/Origin), with or without sign-in. Gradio's CORS accepts any origin, so otherwise any page a visitor opens could start rollouts through their browser. The Task API and MCP are not guarded: they hold no visitor state, and browser tools like the MCP Inspector call them cross-origin on purpose. X-Forwarded-Host is never trusted; behind a proxy that rewrites Host, the operator lists the public host in OPENENV_HARBOR_UI_HOSTS.
Sign-in on Docker Spaces: these do not set SYSTEM=spaces, so Gradio's attach_oauth would fall back to its mocked login, signing every visitor in as the operator. serving sets it before attaching OAuth, and OAuth is attached to the parent app because Gradio hardcodes /login/callback, which breaks under the /web mount.
Rendering: everything a model, task or tool produced is escaped; model markdown loads no images.
ALIGNMENT FLAG: behaviour change in the shared capture server
Invariant at risk: none violated, but openenv.core.harness.capture.server changes, and that is core code.
The concern: /healthupstreams is now [] without the admin key (it was public). On a public Space those are other callers' endpoints, private tunnel URLs among them. Also, UpstreamPool.forget is a new public method.
Suggested reviewer: a maintainer of the capture proxy.
No other invariant is touched. The UI is operator/infrastructure side; nothing reaches the agent's MCP surface; no credentials are logged. No RFC is needed, since there are no core API or architecture changes beyond the flag above.
Behaviour changes (release notes)
For anyone upgrading an existing server or Space:
huggingface_hub>=1.29.0 is now required, by both the root package and harbor_env (push changes a bucket's visibility with update_bucket_settings).
serve on 0.0.0.0 (the default) counts as reachable by others, so a visitor's endpoint on localhost or a private address is refused in the UI. Serve with --host 127.0.0.1 when only this machine uses the UI, or pass --private-urls.
Re-pushing an existing Space with --llm-url no longer lets UI visitors run on that endpoint unless --share-endpoint is given (push prints a note). A UI flag left out of a push is removed from the Space.
--hf-login is on by default, so every push writes hf_oauth: true into the Space README's front matter; --no-hf-login leaves it out.
Capture /health returns upstreams only to a caller with the admin key (OPENENV_CAPTURE_ADMIN_KEY); the other fields stay public.
Tasks that template the server's environment (e.g. HF_TOKEN = "${HF_TOKEN}") run from the UI only on the server's own endpoint.
RFC Status
Not required (bug fix, docs, minor refactoring)
Alignment Checklist
I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
I have checked .claude/docs/INVARIANTS.md and no invariants are violated (see the flag above)
I have run the lint pipeline (usort, ruff format, ruff check on src/ tests/) and the tests
Test Plan
PYTHONPATH=src:envs pytest tests/ with CI's ignores and markers: 3190 passed, 103 skipped, 2 failed. Both also fail on main here (macOS): test_start_refuses_a_port_held_by_a_non_capture_listener every time, and test_agentic_harness_process.py::test_read_line_returns_none_on_timeout intermittently (3 of 6 runs on main)
New tests, mostly tests/envs/test_harbor_ui_training_contract.py:
settings defaults per deployment;
the URL guard, including redirects;
run visibility;
the HF account source;
download grants;
Hub-only adds and env templating refusal;
per-visitor limits;
markdown images;
add progress;
run-store robustness;
bucket add/remove;
sign-in and the mocked-login guard on Docker Spaces;
cross-site refusal;
push bucket visibility and README front matter.
Also grouped layouts in test_harbor_tasks_and_dialects.py, and /health gating plus forget in test_harbor_per_session_engine.py.
After review: the Harbor and CLI suites (tests/envs/test_harbor_*.py tests/test_cli) pass, 702 tests, apart from the macOS-only port test above. The review fixes each have a regression test (env reads on a visitor's model, bucket 404s, re-push flag cleanup, cancelled rollouts, the engine refcount race, Hub inspection errors, removal races, empty secrets, size 0, keyword parsing). In a running UI, a served task templating ${HF_TOKEN} is refused on a connected model before anything starts, and with --no-rollouts the card is a read-only note.
usort check / ruff format --check / ruff check clean on src/ tests/. Files compile under 3.11. The wheel ships ui_assets/*. sync_env_docs.py --check passes. The envs/harbor_env lock passes uv sync --frozen.
In a browser, locally:
browse, filter and search tasks; open a task and use the file viewer's full view;
run with the server endpoint, with HF Inference Providers, and with a custom URL; a private URL is refused when exposed;
open runs, compare, download;
add a Hub dataset with progress, then remove it.
On the test Space:
add from the page copies into the private bucket and persists across a restart; remove deletes the bucket files;
/login/huggingface redirects to the real Hub OAuth with scope openid profile inference-api;
a cross-site POST gets 403; /capture/healthupstreams is [].
DNS rebinding: a visitor's URL is resolved and checked when it's probed and again before each rollout, but the capture proxy's live client resolves it once more and doesn't pin the checked addresses. Pinning needs a change in the core capture client.
HF Sandbox rollouts on a Space are billed to the Space's HF_TOKEN, whoever starts them, and the run card says so. Billing per signed-in visitor would need the jobs OAuth scope and sandbox creation on the visitor's token (discussed in review).
Registry datasets (name@version) in "Add from the Hub".
Claude Code Review
Alignment checked by hand against PRINCIPLES.md and INVARIANTS.md; see the flag above.
Note
High Risk
Adds a publicly deployable web surface with credential handling, URL validation, and cross-site guards, plus behavioral changes to the shared capture proxy /health and upstream cache.
Overview Replaces Harbor’s /web experience with a four-tab Gradio UI (Tasks, run card, Runs, Setup): searchable task cards, task detail with lazy file viewer, rollouts on the server model / HF Inference Providers / a probed custom endpoint, run timelines with compare and downloads, and Hub dataset add/remove with progress. Static CSS/JS assets ship via openenv.harbor.ui_assets; serve/push wire UI policy through new flags and env vars (rollouts, endpoint sharing, visitor models, private URLs, run visibility/history, add-datasets, reward key).
CLI and Space deploy behavior changes:--llm-url is optional (visitors bring their own model); push defaults to not sharing the Space endpoint with UI visitors unless --share-endpoint; buckets are private by default with --public-bucket/--private-bucket; --hf-login writes OAuth front matter; omitted UI flags on re-push reset Space variables to defaults; UI-added datasets copy into the mounted bucket.
Hardening for public exposure:SameOrigin middleware on state-changing /web POSTs; path confinement for task/instruction reads and nested grouped tasks/ discovery (no symlink walks); visitor URL public-address checks (documented); tasks that touch host env/compose run only on the server endpoint; capture UpstreamPool.forget and /health hides upstreams without admin key.
Docs (harbor.md, env README) document the UI, deployment matrix, and migration notes; huggingface_hub>=1.29.0, plus authlib/itsdangerous for Space OAuth.
Reviewed by Cursor Bugbot for commit 37f84ed. Bugbot is set up for automated code reviews on this repo. Configure here.
…c /health
- Task discovery now finds datasets that group tasks by field
(`tasks/<field>/<subfield>/<task>/`, e.g. terminal-bench-science). The nested
search runs only when no top-level folder holds a task.toml, so flat datasets
keep exactly the order, and so the indexes, they had.
- `resolve_task_dirs` takes a `tqdm_class` for callers that show download progress.
- `UpstreamPool.forget` drops the cached client for an engine, and with it the key
it holds, for a caller whose key should not outlive their use of it.
- `/health` lists probed per-session engines only to the admin key. On a public
Space those are other callers' endpoints, private tunnel URLs among them.
…icly
The `/web` UI of `openenv harbor serve` becomes a task explorer and rollout viewer
in four tabs (Tasks, Runs, Setup, and a Docs link), built from Gradio with custom
HTML components.
Tasks
- Card grid with search and facets (dataset, category, difficulty, tags).
- A task page with a table of contents: the instruction the agent receives, the
task's files in a viewer with a full-window mode, environment and verifier
settings, and the task's runs. Answer-like metadata is left out of the summary.
- "Add from the Hub": public datasets tagged `harbor`, with task counts, size and
download progress. On a Space with its bucket mounted, an added dataset is
copied into the bucket server side and survives restarts; it can be removed.
Running
- A run card with three model sources: the server's endpoint, Hugging Face
Inference Providers (a searchable model list; this machine's token locally, or
signing in with Hugging Face on a Space), or any OpenAI-compatible or
Anthropic endpoint, probed before use.
- A run page with a timeline of what the agent did, the result and reward, trace
checks, and downloads of the result and training contract; comparing 2-4 runs.
Deployment settings
- `ui_settings` decides what a visitor may do, with defaults by where the server
runs (loopback, network, Space), each overridable by an environment variable,
and by new `serve`/`push` flags: `--share-endpoint`, `--visitor-endpoints`,
`--run-visibility`, `--run-history`, `--add-datasets`, `--rollouts`.
- Runs can be private to the visitor who started them; per-visitor limits.
- `push`: `--llm-url` is optional, the bucket is private by default
(`--public-bucket` to change it), and `--hf-login` (default on) turns on
OAuth with the `inference-api` scope.
Security
- Visitor URLs, and every redirect they answer with, must resolve to public
addresses unless private URLs are allowed; checked again before each rollout.
- Keys typed into the page stay in server memory for that page, are never
written to disk or run history, and are dropped from the capture proxy once
no run uses them.
- Datasets added from the page come from the Hub only and may not template
the server's environment into their config.
- With sign-in on, state-changing requests from another site are refused.
- Everything a model, task or tool produced is escaped; model markdown loads no
images.
- A "The web UI" section: the four tabs, the run card's model sources, and a
table of deployment settings with their variables, flags and defaults, plus
what the UI guarantees whatever the settings.
- `serve` and `push` flag tables: the UI flags, `--llm-url` optional,
`--public-bucket/--private-bucket` and `--hf-login`.
- Spaces: the private bucket, datasets added into it, signing in with Hugging
Face, and who pays for sandboxes.
- Two troubleshooting entries. docs/ regenerated with sync_env_docs.py.
The reason will be displayed to describe this comment to others. Learn more.
Audit (release bot) — tip 7895dad1
Requested via #openenv-release. Keep draft; do not merge into the 0.6.1 cut until the CSRF finding is fixed and a maintainer completes a security pass. This is feature surface, not a patch.
Tier 1
SameOrigin only attaches when hf_login is on (serving._attach_hf_login). On an exposed Space / network serve with OAuth off, Gradio still accepts cross-origin credentialed POSTs, so a third-party page can start rollouts and spend the shared endpoint without needing the victim’s visitor id. Attach SameOrigin whenever the deployment is exposed or rollouts are enabled — not only beside OAuth. The existing SameOrigin unit test is good; add a regression that _attach_hf_login is not the sole install path.
DNS-rebinding TOCTOU (already called out in url_problem’s docstring). The check resolves once; the live capture httpx.AsyncClient resolves again and does not pin the earlier addresses. Probe-path redirect SSRF is handled (_PublicRedirects, mapped/6to4/teredo/NAT64 coverage + tests). Live httpx does not follow redirects by default, which narrows redirect SSRF on the hot path, but rebinding remains a residual for public deployments.
What looks solid
/healthupstreams behind _admin_ok + UpstreamPool.forget, with focused tests — agree with the PR’s alignment flag; capture maintainers should still glance.
Docker Space OAuth mock-login fix (SYSTEM=spaces before attach_oauth) + attach on the parent app — correct; regression present.
Visitor keys stay in memory, are omitted from run records, and are forget-ed when the last live run releases the engine.
Added datasets: Hub-only; ${VAR} / compose env reads blocked for non-served specs; task file reads use resolve() + is_relative_to.
Deployment defaults (private URLs / local token / add-datasets off when exposed; run visibility own on Space) match ui_settings.load().
Tier 2
Process-wide urllib opener when private URLs are refused — intentional policy, but it affects every urlopen in the process.
Size (~7.9k lines) + public deployment surface: stay draft for reviewability (the three commits help).
Recommendation
Fix (1), keep draft, then maintainer security review before ready-for-review. Out of scope for Release OpenEnv 0.6.1.
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Audited at 7895dad1 on Adithya's request. No blocking defect found in what I could verify, and the security claims that matter held up under execution rather than reading. Three things I could not settle are below, plus one scheduling note.
Verified by running the code, not by reading the description
SSRF guard (ui_settings.url_problem). This is the claim I trusted least and it is the strongest part of the PR. Every bypass form I tried is refused, and legitimate public URLs pass:
Refusing when any resolved address is non-public is the right direction to fail, and the docstring is honest about the remaining rebinding window instead of claiming it is closed.
Path containment (ui_data.read_task_file). Built a task dir with a sibling secret file and a symlink pointing at it. task.toml is served; ../SECRET.txt, ../../SECRET.txt, ./../SECRET.txt, subdir/../../SECRET.txt, ..\SECRET.txt, /etc/hostname and the symlink all return not a file in this task with no content. resolve() before is_relative_to is what closes the symlink case.
Visitor keys never reach disk.RunStore.save writes LiveRun.record(), which is built from the _METAallowlist (id, status, timings, dataset, task, harness, sandbox, model, endpoint, purpose, error, owner). An allowlist rather than a denylist is the reason I believe this will stay true. _card likewise projects the engine down to ok/model/host/source/train/level_text/notes/reason, so the key in server-side session state is not echoed to the page.
The flagged /health change is effective, and worth flagging as you did.upstreams is now gated on _admin_ok, which fails open when no admin key is set — but HarborService.__init__ always sets one ($OPENENV_CAPTURE_ADMIN_KEY or a random token_urlsafe(24)), so on any serve/push deployment the gate really applies. UpstreamPool.forget takes the lock and only pops the cache entry. I agree a capture-proxy maintainer should still sign off, since this is core code.
Tests. The four Harbor suites pass (95 tests), and the full CI-equivalent run on Linux is 3192 passed, 103 skipped, 0 failed — so the two failures you saw are macOS-only, as you said.
Three things I could not settle
reads_environment may be narrower for task.toml than for compose._COMPOSE_VAR is \$(\{|[A-Za-z_]), catching both ${VAR} and bare $VAR, but _HARBOR_VAR is just \$\{. If Harbor's own task.toml resolution honours bare $HF_TOKEN, an added dataset could read the server's environment past this guard. I could not find the resolver in src/openenv/ (no expandvars anywhere), so this may well be compose-only and fine — can you confirm where task.toml env values are expanded?
SameOrigin is installed only inside _attach_hf_login. That matches your description ("with sign-in on"), and the reasoning is sound because the cookie is what makes cross-site requests dangerous. Worth stating explicitly in the docs table though: a public Space with sign-in off has no cross-site guard on state-changing POSTs, so it depends entirely on there being no ambient credential.
Run isolation on a Space rests on an unguessable browser-local id, not authentication — owner_of digests the Gradio-encrypted visitor id. visitor_secret's docstring is candid about this, and it is a reasonable trade for a demo Space, but anyone reading run traces as private data should know it is not auth.
Scheduling
This is 7,912 added lines across 31 files, still a draft, and on a fork whose CI cannot run here. The 0.6.1 cut is Thursday 1 Oct and its candidate (#1258) is already TestPyPI-validated, so this should not go into that release — landing it after the tag gives it a full cycle in main, and the flagged core change deserves a human security reviewer first. Nothing here reaches the agent-facing MCP surface, so it does not block the release either.
The same-origin check was installed only next to Hugging Face sign-in. Without
it, a page a visitor opens could still make their browser start rollouts on the
server's endpoint, or reach a server on their own machine. build_app now always
installs it, scoped to the UI (/web): the Task API and MCP carry no visitor
state, and browser tools such as the MCP Inspector call them cross-origin on
purpose. Behind a proxy that rewrites Host, X-Forwarded-Host counts as the
page's own origin; a request marked Sec-Fetch-Site: cross-site is refused
regardless.
fix(harbor): refuse cross-site UI requests whether or not sign-in is on correctly fixes the finding from the prior tip:
SameOrigin is installed in build_app unconditionally (not only inside _attach_hf_login).
Scope is limited to /web so Task API / MCP (e.g. MCP Inspector) stay callable cross-origin — good call; regression covers /mcp.
test_the_cross_site_guard_is_on_without_sign_in locks the install path.
Docs moved the guarantee out of the OAuth section into the always-on list.
Residual (non-blocking): trusting X-Forwarded-Host alongside Host for the Origin check. A client that sends OriginwithoutSec-Fetch-Site: cross-site could spoof X-Forwarded-Host to match a hostile Origin. Modern browsers still refuse via Sec-Fetch-Site, so the Gradio CSRF/resource-abuse case stays closed. Prefer matching Origin only against Host / configured SPACE_HOST (or a trusted forwarded-host allowlist) if you harden further.
Still keep draft and out of 0.6.1 — large feature surface; capture /health alignment flag and DNS-rebinding residual remain for a maintainer security pass. Watching remaining CI on this head.
…un visibility is not sign-in
reads_environment flags any `${` in task.toml but also bare `$VAR` in compose
files. That is right only because Harbor's resolve_env_vars expands a whole
value of `${VAR}` or `${VAR:-default}` and leaves `$VAR` as it is; a test now
pins that, so a Harbor change there fails here first.
The README now says that `own` run visibility keeps visitors' runs apart but
is not authentication.
The UI's cross-site check compared Origin with X-Forwarded-Host as well as Host.
A client sets that header itself, so one that sends Origin without
Sec-Fetch-Site could name any origin as its own. The check now accepts only
Host, a Space's SPACE_HOST, and hosts the operator lists in
OPENENV_HARBOR_UI_HOSTS, for a proxy that rewrites Host.
The reason will be displayed to describe this comment to others. Learn more.
Re-review @ ba53f480
Both open residuals from the prior tips look correctly closed:
X-Forwarded-Host no longer vouches for Origin. Origin is matched against Host + SPACE_HOST + operator-listed OPENENV_HARBOR_UI_HOSTS only. The spoof case (Origin/XFH = evil) is refused; the proxy case uses the allowlist. Good.
reads_environment asymmetry pinned.test_harbor_expands_only_braced_variables_in_task_toml exercises harbor.utils.env.resolve_env_vars and shows bare $HF_TOKEN is left literal while ${HF_TOKEN} expands — so looking for ${ in task.toml matches Harbor’s real contract. Compose still checked for bare $.
Docs now state own visibility is not authentication.
No new Tier-1 from this delta. Still keep draft / out of 0.6.1 pending maintainer security pass (capture /health alignment flag + DNS-rebinding residual remain by design). Exact-head CI still settling on this tip.
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Re-audited at fc8cb4be. All three of my open questions are closed, and one of them turned out not to be a gap at all. You also fixed something I missed.
The ${VAR}-only question is settled, and the guard was already complete
I verified the contract against the real harbor.utils.env.resolve_env_vars rather than the new test's assertions:
{'bare': '$HF_TOKEN', 'braced': 'secret', 'inline': 'a $HF_TOKEN b', 'default': 'fallback'}
Bare and inline $HF_TOKEN stay literal; only ${VAR} and ${VAR:-default} expand. So _HARBOR_VAR = \$\{ was never narrow — my concern was unfounded, and pinning it with a test that calls the upstream function is the right way to keep it that way. Worth knowing that importorskip makes it skip where harbor is absent; it does run here, so the pin is live in CI as long as the harbor extra stays installed.
Cross-site guard: you went further than I asked, and the scoping is better
I suggested documenting that the guard only applied with sign-in on. Instead SameOrigin now installs unconditionally in build_app and is scoped to /web, with the Task API and MCP deliberately left open because browser tools call them cross-origin on purpose. That is the better call. Probed the matcher directly:
request
result
POST /web/…, Sec-Fetch-Site: cross-site
refused
POST /web/…, same-site but Origin https://b.hf.space (sibling Space)
refused
POST /web/…, Origin on a different port
refused
POST /web exactly, cross-site
refused
POST /web/…, own Origin
passes
GET /web/… cross-site
passes
POST /mcp cross-site
passes (intended)
POST /webhook cross-site
passes — no prefix confusion with /web
X-Forwarded-Host — your catch, and the fix holds
I did not flag this and should have: _same_host compared Origin against the Host header, so behind a proxy a client-supplied X-Forwarded-Host could have vouched for its own origin. The new version never consults it and takes operator-listed hosts from OPENENV_HARBOR_UI_HOSTS instead. Verified: host=10.0.0.5:8000 with X-Forwarded-Host: evil.example and Origin: https://evil.example is refused.
One residual, unchanged and reasonable: a request carrying neither Sec-Fetch-Site nor Origin passes. That is deliberate (server-to-server callers send neither) and safe for CSRF, since the attack needs a browser and current browsers send both on cross-origin POSTs.
No regressions in what I verified last time
SSRF guard still refuses metadata IP, IPv4-mapped loopback, NAT64-embedded loopback, decimal IPv4, RFC1918 and metadata.google.internal, and still allows https://api.openai.com/v1.
Path containment still refuses ../, /etc/hostname, a symlink to a sibling file and subdir/../../, and still serves task.toml.
Full CI-equivalent suite on Linux: 3194 passed, 103 skipped, 0 failed (up two tests from 7895dad1). ruff format --check and ruff check clean on src/ tests/.
Where this leaves the PR
From my side the security review is done and I have nothing blocking. It still needs, in this order: a capture-proxy maintainer on the /health gating and UpstreamPool.forget (your own alignment flag — core code, and I agree it should not ride on my review alone), repository CI, which cannot run while this is a fork PR, and a decision to take it out of draft. Unchanged recommendation on timing: land it after the v0.6.1 tag on Thursday rather than in it, so it gets a full cycle in main.
The nested search for `tasks/<field>/<task>/` layouts followed symlinked
folders. One pointing outside the dataset was indexed as a task, and one
pointing back into it recursed until the path got too long, indexing the same
task dozens of times (or, deep enough, killing the thread). Symlinked folders
are now skipped, and the search stops at six levels.
From Cursor's audit of the PR:
- A dataset the Hub reports no size for is measured from its file listing
before it is added, and refused if that fails; the size cap was skipped.
- A task from an added dataset may not reach the host's files through its
compose files either: env_file, include, extends, or a bind mount from outside
the task. Every YAML file under environment/ is checked, not only the top
level, since one compose file can include another.
- The per-visitor limit counts a signed-in visitor's Hugging Face account; a
browser id is free to replace. The README says what it is for anonymous
visitors, and that OPENENV_HARBOR_UI_MAX_RUNS bounds the total.
- The run page escapes the text it puts on a download button when a download
fails.
- The file tree of a task is walked lazily up to its cap, instead of listing
and sorting every file first.
Two commits on top of the already-audited tip. Security-relevant and look correct.
Verified
_nested_task_dirs skips symlinks + depth cap (_MAX_TASK_DEPTH=6). Closes outside-dataset and loop cases; regression test_a_grouped_layout_does_not_follow_symlinks matches.
reads_environment fail-closed for compose host reads. Any env_file / include / extends, plus bind mounts of /, ~, .., and long-form source:; all YAML under environment/ checked (not only *compose*). Relative ./data still allowed. Tests cover the interesting forms.
Per-visitor quota follows HF account when signed in (quota=hf:{username}); anonymous still browser-id (documented honesty that rotating storage bypasses that cap; MAX_RUNS still bounds total).
Unknown Hub size → measure tasks/ or refuse — no unbounded download. HfApi(token=False) stays public-only.
file_tree → os.walk (no followlinks) + top-level-first ordering; read_task_file still resolve() + is_relative_to so symlink-to-outside content stays refused.
Minor XSS hygiene in run_view.js (esc on button text / errors).
Residuals (non-blocking)
Anonymous per-visitor cap remains rotatable by design — docs say so; global MAX_RUNS is the real bound.
file_tree may still list a symlink’s target size via stat(); content read remains confined.
…ndpoint is shared
Anyone with a Space's URL is a visitor, and with the endpoint shared by default
they all ran on the operator's key. On a Space, OPENENV_HARBOR_UI_SERVER_ENDPOINT
now defaults off: visitors connect their own model (signing in with Hugging
Face, a token, or an endpoint), and `push --share-endpoint` opts back in.
Rollouts stay on. Local and network `serve` keep sharing it, and the Task API
and MCP use the endpoint either way.
`push` says so when it deploys with an endpoint but no --share-endpoint, and
refuses --no-visitor-endpoints without --share-endpoint, which would leave
visitors no model at all.
The reason will be displayed to describe this comment to others. Learn more.
Re-review @ a010993d — Space default: don't share endpoint
Good hardening of the default threat model for public Spaces:
ui_settings.load(): server_endpoint defaults to not space (off on Space, on locally/network).
push leaves the var unset when --share-endpoint is omitted → Space default applies; NOTE printed.
--no-visitor-endpoints without --share-endpoint refused (would leave visitors with no model).
Docs/table updated; tests pin Space off / local+lan on, plus the push refusal.
No Tier-1. Complements the CSRF/SSRF work: anonymous visitors can no longer spend the operator LLM key by default. Sandbox billing to Space HF_TOKEN is still called out in docs (unchanged residual for --no-rollouts ops).
…utput
On GitHub Actions Rich colours and boxes Typer's error, so the colour codes
land inside "--share-endpoint" and a plain substring check fails, while it
passes on a terminal without colour. The test now strips the colour codes and
the box, and checks the exit code is the usage error's 2.
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Final audit at ae7c84b6. Ready from my side — approving. Two things changed since my last pass that matter for readiness: repository CI is now green (11/11, so the fork workflow got approved), and the five new commits close gaps beyond the ones I raised.
The new hardening does what it says
file_tree no longer advertises what read_task_file would refuse. With a task containing inside.txt, task.toml, a .hidden file and an escape symlink to a sibling secret, the tree lists exactly ['inside.txt', 'task.toml'] and reading escape still returns no content. Switching to a lazy capped os.walk also removes the old rglob cost on a vendored tree.
reads_environment now catches host reads that use no $VAR at all — this is the gap I did not think to probe, and it is the most valuable of the five commits. Verified each pattern individually: absolute bind mount - /etc:/host → True; parent bind - ../..:/host → True; env_file: ../../.env → True; include: → True; long-form source: /etc → True; a clean image:-only compose → False. Recursing every YAML under environment/ is right, since one compose file can include another.
Per-visitor quota keyed to the signed-in account rather than the browser id is the correct fix (a browser id is free to replace), and keeping quota out of _META means it cannot reach disk — same discipline as the API key.
server_endpoint now defaults off on a Space so visitors bring their own model. That is a materially safer default than what I first audited: the operator's key is no longer spent by default by anyone with the URL.
Nothing I verified earlier has regressed
SSRF guard still refuses the metadata IP, IPv4-mapped and NAT64-embedded loopback, decimal IPv4, RFC1918, 0.0.0.0 and metadata.google.internal, and still allows https://api.openai.com/v1. Path containment still refuses ../, /etc/hostname, symlinks and subdir/../../. Cross-site guard still refuses a sibling Space's same-site Origin, a different port, POST /web exactly and X-Forwarded-Host self-vouching, while POST /mcp and POST /webhook pass. Bare $VAR in task.toml is still correctly not flagged, since Harbor leaves it literal.
Full CI-equivalent suite on Linux: 3199 passed, 103 skipped, 0 failed. ruff format --check and ruff check clean (usort flags only the two pre-existing files, test_grid_world.py and test_julia_env.py, which are unrelated and also flagged on main).
What "ready" means, precisely
Two gates remain and neither is mine to close:
Take it out of draft. It is still a draft, though CLEAN and mergeable.
A capture-proxy maintainer on the core change — the /healthupstreams gating and UpstreamPool.forget. You flagged this yourself and you were right to: it is core code, my approval should not be the only one on it, and it is a two-file read for whoever owns that module.
On timing, unchanged: the v0.6.1 cut is Thursday 1 Oct and its candidate (#1258) is already TestPyPI-validated, so land this after the tag. That is a scheduling preference, not a reservation about the PR — it gives 7,900 lines of new UI a full cycle in main before it ships in a wheel.
…rom the page
The size check measured `tasks/` when the Hub reported no size, but the bucket
copy took the whole repository, so a small tasks/ folder beside a large rest of
the repo passed the cap and still filled the Space's bucket. The copy now takes
only tasks/: all the loader reads, the same folder a download fetches, and
what the size check measures (the Hub's size, when given, is the whole repo's,
an upper bound on it). Reported by Cursor Bugbot.
The reason will be displayed to describe this comment to others. Learn more.
Bugbot medium is real — fix before merge
Bucket add size check mismatch (Bugbot on ae7c84b6):
When dataset_info().used_storage is missing, the add path falls back to _tasks_bytes (sums only tasks/), but _copy_to_bucket does copy_files(hf://datasets/{spec}/, …) of the entire dataset. A repo with a small tasks/ tree and large non-task content can pass OPENENV_HARBOR_UI_MAX_ADD_GB and still fill the Space bucket.
Local download via resolve_task_dirs (tasks-only allow patterns) is closer to the fallback metric; the mismatch is the bucket path.
Ask: measure (or cap) what will actually be copied — e.g. whole-repo tree size when copying to the bucket, or copy only tasks/ to match the check. Add a regression.
Until that lands, I no longer treat this tip as merge-ready from the audit side. Exact-head repo CI remains green otherwise; still out of 0.6.1.
The reason will be displayed to describe this comment to others. Learn more.
Bugbot medium fixed @ 9c98dc7c
_copy_to_bucket now copies only hf://datasets/<spec>/tasks/ → <prefix>/tasks/, matching _tasks_bytes, the Hub download allow-patterns, and what the loader reads. Docs updated; test pins the new paths.
Prior REQUEST_CHANGES on the size mismatch is satisfied. Exact-head CI still settling on this tip.
Merge gate unchanged: still needs independent human APPROVE from the requested reviewers (sergiopaniego / burtenshaw or a capture maintainer for /health+forget). Out of 0.6.1.
…sed, not as text
reads_environment matched patterns in the raw text, so two forms a task could
use to read the server's environment got past it (found by Cursor's audit):
- a TOML escape: `K = "${HF_TOKEN}"` has no `${` on disk, but decodes to
`${HF_TOKEN}`, which Harbor expands;
- a compose pass-through: `environment: [HF_TOKEN]` has no `$` at all, and
Compose copies the host's value in.
task.toml is now read with tomllib and compose files with yaml.safe_load, and
the decoded values are checked: any `${` in task.toml; in compose, any `$`
interpolation, bare names under `environment:` or build `args:`, a secret or
config sourced from the environment, env_file/include/extends, and host paths
in volumes, secret or config files, build contexts and devices. A file that
doesn't parse, or a compose file over 1 MB, is refused. YAML aliases are walked
once, so an alias bomb can't stall the check. The text patterns stay as a
second net.
Alias-bomb walk bounded by seen; oversized compose refused
Regressions for both prior evasions + ordinary compose still passes
Prior REQUEST_CHANGES on these evasions is satisfied. Exact-head CI settling.
Merge gate unchanged: still needs independent human APPROVE (sergiopaniego / burtenshaw or capture maintainer for /health+forget). Out of Thursday v0.7.0.
The reason will be displayed to describe this comment to others. Learn more.
Bugbot mediums on 0fc44903 — two more path-guard gaps
Reproduced both:
additional_contexts list form (name=path): check applies _HOST_PATH to the whole string (assets=/etc → False). Map form works.
Embedded .. traversal: _HOST_PATH only flags a leading //~/.., so foo/../../etc as source/file/context/volume LHS is unflagged.
Please fix + regress before merge. Parsed TOML/list-env work from this commit still stands; these are follow-ons in the same host-path net. Out of v0.7.0.
…orm build contexts
Two gaps Cursor Bugbot found in the parsed compose check:
- a host path was recognised only by how it starts, so `foo/../../etc` in a
bind source, secret file or build context got through. Paths are now
normalised first: absolute, `~`, a Windows drive, or anything that climbs
out of its folder once normalised is refused;
- `additional_contexts` has a list form, `- name=path`, and the whole entry
was read as the path. The part after `=` is checked now, as the map form's
values already were.
The reason will be displayed to describe this comment to others. Learn more.
Re-audit @ 22549f16 — both path-guard mediums fixed
Verified against tip:
additional_contexts list form — path after = is checked (src=/etc, src=foo/../../etc).
_outside — posixpath.normpath then rejects absolute / ~ / Windows drive / .. climb-out (foo/../../etc → ../etc).
Regressions in test_env_reads_are_found_in_what_harbor_and_compose_decode cover both. Local reproduction of the Bugbot cases + allowlist cases: all pass.
Approve for the Harbor /web safety surface. Keep out of v0.7.0 unless you explicitly want it cherry-picked after CI is fully green on this tip (lint already green; test / Docker / Bugbot still running when I looked).
The reason will be displayed to describe this comment to others. Learn more.
thanks!! The new UI looks great :) just a few things from me and an agent reviewing the code:
Inline comments below, roughly the first ones are the important ones.
The PR also touches some core/shared pieces beyond the UI (the capture server's /health gating + UpstreamPool.forget, the grouped task discovery in tasks.py), so it'd be good to get a maintainer's ok on those too.
The reason will be displayed to describe this comment to others. Learn more.
Follow-up after Sergio’s review — withdrawing public-Space approve
Sergio’s pass (APPROVED with comments) surfaces a real residual our earlier audits did not weight enough:
Served datasets skip reads_environment (ui.py ~1169: spec not in served and …). With Space defaults (visitor_endpoints on, operator endpoint off), a visitor BYO model can still get the Space’s ${HF_TOKEN} expanded into a served task and read it from the trace. The added-dataset guard does not cover this.
That needs a fix (or hard startup warn / trace masking) before I’d call public Spaces safe on this surface.
Also agree these are worth landing before merge (not blocking the same way, but real):
sticky Space UI flags on re-push (can’t turn --share-endpoint back off)
_release_engine calling forget outside self._lock
broad suppress(Exception) / inspect-cache treating 429 as “no tasks”
… check
The env check's recursive string walker reused the name `_strings`, which
`task_row` already uses for keywords, so every task row added two generators
and the task list failed to load. Renamed to `_every_string`, with a test.
- serve: `--private-urls/--no-private-urls` for OPENENV_HARBOR_UI_PRIVATE_URLS,
which had no flag; the refusal message and the migration note name it. `push`
neither sets nor removes it: on a Space those addresses are its own network.
- A served task that reads the server's environment still runs on the server's own
endpoint, so the refusal for a visitor's model now says that, and the README no
longer says such tasks can't run from the UI at all.
- Adding a dataset: a size of 0 from the Hub (not measured yet) is measured like an
unknown one instead of passing the size cap.
- Add panel: a Hub error from hb_inspect shows as "couldn't check yet", leaves Add
enabled and is asked again, rather than reading as "not in Harbor's tasks/ layout".
- The remaining getattr on attributes that always exist (DatasetInfo, harness
status and kind, capabilities, OAuth profile, repo and bucket tree entries,
TurnNode) are gone; tree entries are told apart by type.
attach_oauth reads it once and its routes use SPACE_HOST afterwards; the reason to
leave it set is that later get_space() callers agree with the login.
The push tests' fake HfApi defines update_bucket_settings on any version. This
checks both floors exclude 1.28, the last release without it, and that the
installed client has every bucket and Space call push makes.
@burtenshaw the Cursor Release automation's changes-requested review (5363806484, on 22549f16) still blocks the PR. Its later runs only posted comments, so its verdict never updated. Everything that review asked for is fixed on f140979f, and CI passes:
Asked for
Where
Served datasets hand ${HF_TOKEN} to a visitor's model
A task that reads the server's environment or files runs only on the server's own endpoint, whichever dataset it's from (52586156; refusal wording b3fee845). Checked in a running UI with FineEnvs/SmolDataEnvs-harbor-eval.
Sticky Space UI flags on re-push
A push deletes the UI variables it wasn't given (52586156).
forget outside self._lock
It now runs in the same critical section as the refcount reaching zero (52586156).
suppress(Exception) / 429 cached as "no tasks"
Only a 404 means a missing bucket or no tasks. Hub errors aren't cached, the cache is bounded, and the page shows the error (52586156, b3fee845).
Both floors are >=1.29.0 (52586156), with a test that doesn't use the fake (f140979f).
The docs and UX items are in too: the read-only card, sandbox billing copy, migration notes, and serve --private-urls. Each thread has the details, and all are resolved except Sergio's question on visitor-funded sandboxes, which is left for you two. Could you re-run the review, or dismiss it if you're happy?
A final audit found a named volume on the local driver binding a host path
(`driver_opts: {type: none, o: bind, device: /etc}`) that `reads_environment`
let through. The same holds for the rest of that class, now all refused for an
added dataset and run only on the server's own endpoint otherwise:
- named volumes with any setting but labels (driver, driver_opts, external, a
`name` that reuses an existing volume) and networks with a `name` or a driver
other than bridge/overlay;
- privileges and host access: privileged, cap_add, security_opt,
device_cgroup_rules, volumes_from, use_api_socket, provider, `external`;
- host or other-container namespaces: pid, ipc, network_mode, userns_mode, uts,
cgroup, and build `network`, set to `host` or `container:...`;
- a build's SSH agent, entitlements and `type=local` caches, and `develop.watch`
paths outside the task.
An ordinary compose file (plain named volumes, bridge networks, service aliases,
relative mounts, registry caches) still passes, and the README says which tasks
`--share-endpoint` still runs on a public Space.
Follow-up to the final merge-readiness audit: the named-volume hole it found is fixed in 105a08fe. volumes: {h: {driver: local, driver_opts: {type: none, o: bind, device: /etc}}} now counts as reading the host.
I closed the rest of that class in the same commit, so a dataset added from the page can't use any of it, and a served task that does runs only on the server's own endpoint:
named volumes with any setting but labels, and networks with a name or a non-bridge/overlay driver;
a build's ssh, entitlements and type=local caches, and develop.watch paths outside the task.
An ordinary compose file still passes, and there's a test covering every case. I also checked Harbor's own task fields: stdio MCP servers run inside the sandbox, and artifact destinations can't leave the trial folder.
The branch is merged with current main (c16c5e49, no overlapping files). The --share-endpoint residual is now in the README.
Flat task discovery follows a symlinked task folder, and the page reads
`task.toml` and `instruction.md` on its own, so a dataset added from the page
with a link in it could put files from the server on the card grid and task
view. Discovery itself is unchanged, since it is shared with the Task API and
skipping links would renumber tasks in a local dataset that uses them.
- An added dataset with any symbolic link, file or folder, is refused before it
is indexed, and what was copied is removed, so a restart (which lists the
bucket's folders) does not bring it back.
- The page never reads `task.toml` or `instruction.md` through a link.
- `reads_environment` counts a linked task folder, `task.toml`, `environment/`
or compose file as reading the host.
…d adds out
- The page's own reads of `task.toml` and `instruction.md` refuse a task folder
that is itself a link, not only a linked file.
- A refused add whose cleanup fails says so, and the bucket listing at startup
skips a dataset whose `tasks/` or task folders are links, so the copy it left is
not served again after a restart. One directory listing per dataset; links
further down are never followed by the page, the file viewer or
`reads_environment`.
- `_first_link` walks with scandir entries, without a stat per file.
…lder
The last audit found two reads that still followed a link on a dataset the
server was started with: the file viewer resolved a linked task folder and
served files from the host directory behind it, and a linked `tasks/` made the
task folder under it look ordinary to the card reads.
One rule now covers every read the page makes (card text, task view, file tree,
file viewer, environment check): `task_root(spec, task_dir)` is the task folder
resolved, or nothing when it resolves outside the dataset's folder, and each file
is read only if it resolves inside that root. A link to the dataset folder itself
(`--dataset ~/datasets/current`) is still followed. A task behind a link to
anywhere else shows no files and counts as reading the host, so it never runs on
a visitor's model. Discovery and task indexes are unchanged.
…anchor
The last audit found two reads outside the page's rule. `read_instruction`,
which the public Task API returns for every listed task, read `instruction.md`
through any link: a linked task folder or a linked file served host text with no
rollout. And a registry dataset (`name@version`) had no folder to anchor on, so
a link inside Harbor's cache still resolved to the host.
`task_root` and `own_file` move to `tasks.py`, so the Task API and the UI share
one rule. A registry task in Harbor's cache is anchored on the cache, and a link
out of it is not followed; one its registry names outside the cache is already
resolved by Harbor. Only what is read to be shown changes: Harbor still reads a
task for itself when it runs it.
A real rollout from the page (terminus-2 on e2b, the server's model, a
SmolDataEnvs task) solved the task and was still marked failed: the task
reports `correctness` and `submission`, none named `reward`, and the page never
says which is the headline, so `run_rollout` refused. A trainer must choose; a
page's run is for reading.
- `run_rollout(require_reward=False)` keeps such a run, with every reward in
`rewards` and a warning, instead of failing it; the default is unchanged. UI
runs pass it, plus `serve --reward-key` / `push --reward-key`
(`OPENENV_HARBOR_REWARD_KEY`) when the operator names the headline one. The
result shows "Graded" with each reward listed, the stage strip "2 rewards",
and "ungraded" is no longer claimed when the verifier did grade.
- The run card no longer renders the read-only note before its first value: it
shows only when the server says rollouts are off.
- A `#task=` link pasted into an open page opens that task (hashchange).
- An unset setting reads "not set" on the Setup tab.
…lled
Harbor installs only on Python 3.12+ (the `harbor` extra), so the 3.11 job has
no `harbor.constants` to patch. The code already treats a missing Harbor as no
registry cache; the test now skips there instead of failing on the import.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
sergiopaniego
left a comment
Summary
Reworks the
/webUI ofopenenv harbor serve(and of Spaces made withopenenv harbor push) into a task explorer and rollout viewer: browse and filter every served task, read exactly what the agent receives, run an agent with the server's model, a Hugging Face Inference Providers model or any OpenAI-compatible/Anthropic endpoint, then read what it did. It also adds deployment settings, so the same UI is safe to run as a public Space: who may start rollouts, whose endpoint they use, who sees which runs, and whether datasets can be added from the page.Test Space (public, rollouts off, adding on, private bucket): https://huggingface.co/spaces/AdithyaSK/openenv-harbor-ui-test
Type of Change
What changes
The first three commits are the change, reviewable one at a time; the commits after them address review (see Behaviour changes):
fix(harbor): find grouped task folders; keep probed engines off public /health: small changes to shared code.tasks/<field>/<subfield>/<task>/, e.g. terminal-bench-science). The nested search runs only when no top-level folder holds atask.toml, so flat datasets keep exactly the order, and so the indexes, they had.resolve_task_dirs(..., tqdm_class=)for callers that show download progress.UpstreamPool.forget(upstream)drops a cached engine client, and the key it holds./healthlists probed per-session engines only to the admin key.feat(harbor): rework the serve web UI and make it safe to deploy publicly: the UI.docs(harbor): document the web UI, its deployment settings and new flags: README plus the regenerated docs page.The UI
gold_answer,solution, …) is left out of the summary. Add from the Hub lists publicharbor-tagged datasets with task count and size, and adds one in the background with progress.Deployment settings
Defaults depend on where the server runs: local means
--host 127.0.0.1; network means any other bind address; Space is detected fromSPACE_ID. Each setting is an env var, and most have aserve/pushflag.OPENENV_HARBOR_UI_ROLLOUTS--rolloutsOPENENV_HARBOR_UI_SERVER_ENDPOINT--share-endpointOPENENV_HARBOR_UI_VISITOR_ENDPOINTS--visitor-endpointsOPENENV_HARBOR_UI_PRIVATE_URLS--private-urls(serveonly)OPENENV_HARBOR_UI_LOCAL_TOKENOPENENV_HARBOR_UI_ADD_DATASETS--add-datasetsOPENENV_HARBOR_RUN_VISIBILITY--run-visibilityOPENENV_HARBOR_RUN_HISTORY--run-historyOPENENV_HARBOR_UI_MAX_RUNS(_PER_VISITOR)pushalso changes in five ways:--llm-urlis optional; without it, visitors bring their own model.--share-endpointis given; the Task API and MCP use it either way.--public-bucket/--private-bucketsets it, and an existing bucket keeps its visibility unless one is given. Only a 404 counts as a missing bucket.--hf-login(default on) addshf_oauth: truewith theinference-apiscope.With
--add-datasetson a Space, an added dataset is copied into the Space's bucket server side (copy_files, no download) and read through the/datamount, so it survives restarts. Removing it deletes it from the bucket.Security
This became a large part of the work once the UI could run publicly:
forget-ed from the capture proxy's pool once no run uses it.${VAR}templating intask.toml/compose, which is where the server's keys live, or reaches the host through compose, runs from the UI only on the server's own endpoint. Reaching the host coversenv_file,include,extends, and host paths in mounts, builds, caches and watch rules. It also coversprivileged,cap_add, host namespaces, the Docker socket, other containers' volumes, named volumes or networks with settings (the local driver binds any path), and a build's SSH agent. A model a visitor connects could print the sandbox's environment into a trace that visitor reads. Both files are checked as parsed. On--share-endpoint, such a served task does run on the operator's model, and the visitor who starts it reads its trace; the README says not to combine the two on a public Space.task.tomlorinstruction.mdthrough a link. Grouped task discovery doesn't follow symlinks.OPENENV_HARBOR_UI_MAX_RUNSbounds the total./web) that a browser sends from another site is refused (Sec-Fetch-Site/Origin), with or without sign-in. Gradio's CORS accepts any origin, so otherwise any page a visitor opens could start rollouts through their browser. The Task API and MCP are not guarded: they hold no visitor state, and browser tools like the MCP Inspector call them cross-origin on purpose.X-Forwarded-Hostis never trusted; behind a proxy that rewritesHost, the operator lists the public host inOPENENV_HARBOR_UI_HOSTS.SYSTEM=spaces, so Gradio'sattach_oauthwould fall back to its mocked login, signing every visitor in as the operator.servingsets it before attaching OAuth, and OAuth is attached to the parent app because Gradio hardcodes/login/callback, which breaks under the/webmount.ALIGNMENT FLAG: behaviour change in the shared capture server
openenv.core.harness.capture.serverchanges, and that is core code./healthupstreamsis now[]without the admin key (it was public). On a public Space those are other callers' endpoints, private tunnel URLs among them. Also,UpstreamPool.forgetis a new public method.No other invariant is touched. The UI is operator/infrastructure side; nothing reaches the agent's MCP surface; no credentials are logged. No RFC is needed, since there are no core API or architecture changes beyond the flag above.
Behaviour changes (release notes)
For anyone upgrading an existing server or Space:
huggingface_hub>=1.29.0is now required, by both the root package andharbor_env(pushchanges a bucket's visibility withupdate_bucket_settings).serveon0.0.0.0(the default) counts as reachable by others, so a visitor's endpoint onlocalhostor a private address is refused in the UI. Serve with--host 127.0.0.1when only this machine uses the UI, or pass--private-urls.--llm-urlno longer lets UI visitors run on that endpoint unless--share-endpointis given (pushprints a note). A UI flag left out of a push is removed from the Space.--hf-loginis on by default, so everypushwriteshf_oauth: trueinto the Space README's front matter;--no-hf-loginleaves it out./healthreturnsupstreamsonly to a caller with the admin key (OPENENV_CAPTURE_ADMIN_KEY); the other fields stay public.HF_TOKEN = "${HF_TOKEN}") run from the UI only on the server's own endpoint.RFC Status
Alignment Checklist
.claude/docs/PRINCIPLES.mdand this PR aligns with our principles.claude/docs/INVARIANTS.mdand no invariants are violated (see the flag above)usort,ruff format,ruff checkonsrc/ tests/) and the testsTest Plan
PYTHONPATH=src:envs pytest tests/with CI's ignores and markers: 3190 passed, 103 skipped, 2 failed. Both also fail onmainhere (macOS):test_start_refuses_a_port_held_by_a_non_capture_listenerevery time, andtest_agentic_harness_process.py::test_read_line_returns_none_on_timeoutintermittently (3 of 6 runs onmain)tests/envs/test_harbor_ui_training_contract.py:pushbucket visibility and README front matter.Also grouped layouts in
test_harbor_tasks_and_dialects.py, and/healthgating plusforgetintest_harbor_per_session_engine.py.tests/envs/test_harbor_*.py tests/test_cli) pass, 702 tests, apart from the macOS-only port test above. The review fixes each have a regression test (env reads on a visitor's model, bucket 404s, re-push flag cleanup, cancelled rollouts, the engine refcount race, Hub inspection errors, removal races, empty secrets, size 0, keyword parsing). In a running UI, a served task templating${HF_TOKEN}is refused on a connected model before anything starts, and with--no-rolloutsthe card is a read-only note.usort check/ruff format --check/ruff checkclean onsrc/ tests/. Files compile under 3.11. The wheel shipsui_assets/*.sync_env_docs.py --checkpasses. Theenvs/harbor_envlock passesuv sync --frozen./login/huggingfaceredirects to the real Hub OAuth with scopeopenid profile inference-api;/capture/healthupstreamsis[].To try it locally:
Then open http://127.0.0.1:8000/web.
Follow-ups (not in this PR)
HF_TOKEN, whoever starts them, and the run card says so. Billing per signed-in visitor would need thejobsOAuth scope and sandbox creation on the visitor's token (discussed in review).name@version) in "Add from the Hub".Claude Code Review
Alignment checked by hand against
PRINCIPLES.mdandINVARIANTS.md; see the flag above.Note
High Risk
Adds a publicly deployable web surface with credential handling, URL validation, and cross-site guards, plus behavioral changes to the shared capture proxy
/healthand upstream cache.Overview
Replaces Harbor’s
/webexperience with a four-tab Gradio UI (Tasks, run card, Runs, Setup): searchable task cards, task detail with lazy file viewer, rollouts on the server model / HF Inference Providers / a probed custom endpoint, run timelines with compare and downloads, and Hub dataset add/remove with progress. Static CSS/JS assets ship viaopenenv.harbor.ui_assets;serve/pushwire UI policy through new flags and env vars (rollouts, endpoint sharing, visitor models, private URLs, run visibility/history, add-datasets, reward key).CLI and Space deploy behavior changes:
--llm-urlis optional (visitors bring their own model);pushdefaults to not sharing the Space endpoint with UI visitors unless--share-endpoint; buckets are private by default with--public-bucket/--private-bucket;--hf-loginwrites OAuth front matter; omitted UI flags on re-push reset Space variables to defaults; UI-added datasets copy into the mounted bucket.Hardening for public exposure:
SameOriginmiddleware on state-changing/webPOSTs; path confinement for task/instruction reads and nested groupedtasks/discovery (no symlink walks); visitor URL public-address checks (documented); tasks that touch host env/compose run only on the server endpoint; captureUpstreamPool.forgetand/healthhidesupstreamswithout admin key.Docs (
harbor.md, env README) document the UI, deployment matrix, and migration notes;huggingface_hub>=1.29.0, plusauthlib/itsdangerousfor Space OAuth.Reviewed by Cursor Bugbot for commit 37f84ed. Bugbot is set up for automated code reviews on this repo. Configure here.