Skip to content

feat(providers): add DMOJ as a third problem source - #140

Merged
cameroncuster merged 1 commit into
mainfrom
feat/dmoj-provider
Aug 26, 2026
Merged

cameroncuster merged 1 commit into
mainfrom
feat/dmoj-provider

Conversation

@cameroncuster

Copy link
Copy Markdown
Owner

Adds DMOJ as a third problem source, so https://dmoj.ca/problem/ciw26p2 and
its siblings can be listed alongside Codeforces and Kattis.

Why the API and not the page

DMOJ's public problem pages sit behind bot protection — a plain non-browser
request gets 403, so the Kattis-style HTML scrape cannot work:

https://dmoj.ca/problem/ciw26p2        -> 403
https://dmoj.ca/api/v2/problem/ciw26p2 -> 200

Ingestion therefore reads the API v2 single-problem endpoint through
/api/dmoj, an allowlisted proxy that mirrors /api/kattis: caller input is
reduced to a validated problem code before the upstream URL is built, so no
arbitrary-URL proxy is ever exposed. Timeout, abort, and upstream-status
handling are identical to the Kattis route.

Two decisions worth reviewing

Bare codes are rejected. Kattis accepts a bare hello because its slugs are
readable and unambiguous. DMOJ codes are opaque short strings, so accepting
them unqualified would let any stray pasted word classify as a DMOJ problem —
and getProblemSource runs against every stored URL. DMOJ requires a fully
qualified dmoj.ca/problem/<code> URL, which is also what makes the three-way
source classification safe.

Difficulty is left unset. problems.difficulty is an integer on the
Codeforces 800–3500 scale, and Kattis maps its 1–10 range onto it. DMOJ scores
problems in points (typically 3–50), which are not comparable across problem
groups the way a rating is. Rather than invent a curve that would sort DMOJ
problems wrongly against the existing catalog, difficulty stays undefined —
the same state the Kattis fetch-failure path already produces. DMOJ rows render
as unrated, with a tooltip explaining why. Happy to add a mapping if you'd
prefer one; it is a one-line change in the ingestion module.

DMOJ's types field maps directly onto the existing tags array, so that
metadata is preserved.

Changes

New:

  • src/lib/providers/dmoj/ingestion.ts — URL validation, extraction, resolve
  • src/routes/api/dmoj/+server.ts — code-validated upstream proxy
  • src/lib/submit/providers/dmoj.ts — submit adapter
  • static/images/dmoj.svg — an original mark, not DMOJ's logo

Widened from two sources to three: ProviderId, providerOrder,
providerFromUrl, Problem['source'], getProblemSource,
SOURCE_FILTER_STATES (the table filter is now a 4-state cycle), and
ProblemTable's logo lookup and difficulty tooltip. No schema change —
problems is source-agnostic and derives source from the URL.

E2E redirects the DMOJ upstream to the mock server via
PUBLIC_DMOJ_API_BASE, so no live provider is contacted.

Validation

lint, lint:es, check, build, and test:e2e (211 passed, 7 live-layer
skipped) all pass. Unit coverage stays at 100% lines/branches/functions with
test:coverage; 472 unit tests pass.

Note on deployment

The API answered from a residential IP. Vercel egresses from datacenter IPs,
which Cloudflare treats more aggressively — worth confirming the first DMOJ
submit works from the deployed runtime. If it is blocked, the proxy is the only
place that would need to change.

DMOJ problem pages sit behind bot protection, so ingestion reads the
public API v2 single-problem endpoint through an allowlisted proxy that
reduces caller input to a validated problem code before building the
upstream URL.

Unlike Kattis, bare problem codes are rejected: DMOJ codes are opaque
short strings, so accepting them unqualified would let any stray word
classify as a DMOJ problem.

DMOJ scores problems in points, which are not comparable to the stored
Codeforces-style 800-3500 rating range, so difficulty is left unset
rather than mapped through an invented curve.
@cameroncuster
cameroncuster merged commit e109f53 into main Aug 26, 2026
14 checks passed
@cameroncuster
cameroncuster deleted the feat/dmoj-provider branch August 26, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant