Skip to content

fix(GAT-9245): Saving unresolvable linkages - #1748

Merged
calmacx merged 4 commits into
devfrom
fix/GAT-9245
Sep 28, 2026
Merged

calmacx merged 4 commits into
devfrom
fix/GAT-9245

Conversation

@calmacx

@calmacx calmacx commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Note: The automated tests in this PR were written by Claude (AI assistant) and reviewed by the author.

Screenshots (if relevant)

N/A

Describe your changes

Now that the linkage junction tables are the source of truth for GWDM 2.x linkages, afterRead() rebuilds the linkage block from SQL alone. Previously, any reference extraction could not resolve at write time was dropped — so it survived only in the JSON blob and was lost on read:

  • a publication DOI with no matching publications row, or
  • a free-text dataset reference (raw url/pid/title not matching any gateway dataset).

This branch makes the junction tables the complete source of truth so those references are preserved, and adds a manual command to backfill/inspect existing data.

1. Preserve unresolved references (Gwdm2xHandler + migration 2026_06_18_000001)

  • Unresolved dataset refs are stored with dataset_version_target_id = NULL + raw_url/raw_pid/raw_title; unresolved DOIs with publication_id = NULL + raw_doi (both FKs made nullable).
  • afterRead() / getLinkages() reconstruct them (publications via COALESCE(publications.paper_doi, raw_doi)).
  • Empty raw strings are coerced to NULL (blankToNull) on write and read: the GWDM schema allows null url/pid/title but rejects '' (minLength/uri), so this prevents read-time validation failures. (Null itself is valid per the schema.)
  • The unresolved-rows query is a shared raw-SQL helper (resolveUnresolvedDatasetLinkages) reused by afterRead() and getLinkages().

2. app:reconcile-linkages command (manual ops tool — not auto-wired)

  • Re-runs extraction (idempotent, blob → SQL) to backfill references that pre-date the migration.
  • --dry-run prints a drift report: a table of what is in the blob but missing from SQL, with a reason, plus a per-reason summary — and writes nothing. Reasons: no publication row, publication exists but not linked, linkage soft-deleted, unresolvable reference, dataset exists but not linked, target dataset not active.
  • Options: --dataset, --all-versions, --sync (inline via handler), --dry-run, --force, --chunk.

Issue ticket link

https://hdruk.atlassian.net/browse/GAT-9245

Environment / Configuration changes (if applicable)

None. New artisan command is run manually; nothing dispatches it automatically.

Requires migrations being run?

Yes — 2026_06_18_000001_add_unresolved_linkage_columns (nullable target FKs + raw_url/raw_pid/raw_title/raw_doi). Existing rows predate the change; run php artisan app:reconcile-linkages --dry-run to see drift, then without --dry-run to backfill.

If not using the pre-push hook. Confirm tests pass:

  • pest tests/Feature/DatasetVersionLinkageTest.php — 12 passed.
  • Pint clean; PHPStan level 3 clean on changed files.

Tests added (authored by Claude, reviewed by the author):

  • test_unresolved_linkage_roundtrips_title_and_url_via_get_linkages
  • test_extract_linkages_reads_blob_not_stale_sql_overlay
  • test_reconcile_command_backfills_unresolved_publication_doi
  • test_reconcile_dry_run_reports_drift_without_writing

Checklist before requesting a review

  • I have performed a self-review of my code
  • I have added appropriate unit tests
  • I have created mocks for unit tests (where appropriate) — N/A (feature tests hit the DB; TRASER mocked via MockExternalApis)
  • I have added appropriate Behat tests to confirm AC (if applicable) — N/A
  • I have added Swagger annotations for new endpoints (if applicable) — N/A (no new endpoints)
  • I have added audit logs for new operation logic (if applicable) — N/A (reconstruction is read-path; command re-uses LinkageExtraction)
  • I have added new environment variables to the .env.example file (if applicable) — N/A
  • I have added new environment variables to terraform repository (if applicable) — N/A

calmacx and others added 3 commits August 6, 2026 14:03
Make the linkage junction tables the complete source of truth for linkage
data. Unresolvable free-text dataset references (raw url/pid/title) and
publication DOIs are now stored as junction rows with a NULL target FK, so
Gwdm2xHandler::afterRead() can reconstruct the full GWDM linkage section from
SQL instead of a dual-source merge against the JSON blob.

- migration: nullable target FKs + raw_url/raw_pid/raw_title/raw_doi columns
- Gwdm2xHandler: store + reconstruct unresolved references
- junction models: raw_* fillable

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gh-actions-pipelines-app

Copy link
Copy Markdown

🎉 Great job! Your PR title follows the correct format. 🚀

@calmacx
calmacx marked this pull request as ready for review September 28, 2026 08:44
@calmacx
calmacx requested review from a team as code owners September 28, 2026 08:44
@calmacx
calmacx merged commit 769ff02 into dev Sep 28, 2026
3 checks passed
@calmacx
calmacx deleted the fix/GAT-9245 branch September 28, 2026 09:19

This branch was successfully deployed

1 active deployment
dev — cca26b9c Deployed Sep 28, 2026 by calmacx via test #6842
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants