Skip to content

feat(database): add Compatible indexes for Mongo and SQL dialects - #1613

Draft
ChrisPdgn wants to merge 2 commits into
mainfrom
feat/database-dialect-indexes
Draft

feat(database): add Compatible indexes for Mongo and SQL dialects#1613
ChrisPdgn wants to merge 2 commits into
mainfrom
feat/database-dialect-indexes

Conversation

@ChrisPdgn

Copy link
Copy Markdown
Contributor

Description

Rewrite of the approved #643 index work from current origin/main (not a cherry-pick/rebase of the stale indexes branch).

Schema indexes are now first-class on MongoDB, PostgreSQL, MySQL, MariaDB, and SQLite. Platform models (authorization + chat) declare CompatibleIndexType Ascending/Descending so the same indexes exist on SQL too (kkopanidis CHANGES_REQUESTED on #643).

What this does

  • Adds CompatibleIndexType (Ascending / Descending) plus optional unique. Maps to Mongo 1/-1 and SQL BTREE ASC/DESC.
  • Admin create / get / delete / import / export persist declared indexes into _DeclaredSchema via metadata update. No full schema rebuild/sync() for index create/delete.
  • getIndexes uses the live engine as source of truth and overlays declared types (hybrid).
  • Index name is optional; a deterministic name is generated when missing (platform schemas no longer require a name).
  • SQL adapters use originalSchema.collectionName everywhere (no hardcoded cnd_ prefix).
  • Admin create is privileged for unique indexes. Import is not privileged and respects schema owner.
  • Export/import are Admin-only and paginated (skip/limit). Same-name indexes are skipped on import.
  • validateModelOptions accepts indexes and still allows conduit.readPreference.
  • Mongoose dropIndex is awaited (no fire-and-forget).
  • On schema recover, foreign-dialect leftover types are warned and skipped. Admin-bound invalid types throw.

Intentionally not changed / not ported

  • getDatabaseType() still returns 'PostgreSQL' (no 'postgres' rename).
  • No Client API, no BullMQ, no masterkey.
  • Does not conflate with ReBAC ActorIndex / ObjectIndex documents — only their schema index declarations changed to Compatible.
  • Did not port old feat(database)!: add index support for all sql dialects & index refactoring #643 bugs: case 'mysql' || 'mariadb', metadata-only getIndexes, createSchemaFromAdapter rebuild for indexes, unbounded export, required index names, import always treating unique as database-owned, ReverseMongoIndexTypeMap display-name round-trip.

Fixes #643 (implementation rewrite; this PR does not close or merge #643 by itself).

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Unit tests covering T1–T42 plus existing SchemaAdmin PUT/PATCH tests:

cd modules/database
NODE_OPTIONS=--experimental-vm-modules pnpm exec jest --testPathPatterns='admin/__tests__|indexes|validateModelOptions|platform-models|no-old-pr-bugs' --testPathIgnorePatterns=integration

Result: 9 suites, 54 passed.

Live-engine integration tests were not run in this environment (no Mongo/SQL). Recommended follow-up: smoke create/get/delete/import/export of a Compatible unique + descending compound index on Mongo, PostgreSQL, MySQL/MariaDB, and SQLite.

Test Configuration:

  • Unit tests via Jest + ts-jest (ESM)
  • Node 24
  • Workspace packages @conduitplatform/database, @conduitplatform/grpc-sdk

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

Remaining risks

  • showIndex row shapes differ by SQL dialect; unusual engine-specific indexes may normalize oddly.
  • Compound Compatible DESC uses Sequelize { name, order } field objects — worth a live MySQL/MariaDB/SQLite check.
  • Replica instances rely on the shared DB + published modelOptions.indexes; they do not recreate indexes on instanceSync.
Open in Web Open in Cursor 

Make schema indexes first-class on Mongo, PostgreSQL, MySQL, MariaDB, and SQLite. Platform models use Compatible Ascending/Descending so the same declarations create live indexes on every dialect. Admin create/get/delete/import/export persist into _DeclaredSchema without a full schema rebuild.
Repair the ModelOptionsIndexTypes union that failed tsc, stop stuffing
Sequelize field objects into Conduit index types, and move the index
suites into src/__tests__/indexes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants