Skip to content

refactor: Nautilus-style instrument model — broker-first seeding, exp… - #17

Closed
maxkuttner wants to merge 1 commit into
mainfrom
refactor/nautilus-instrument-model
Closed

refactor: Nautilus-style instrument model — broker-first seeding, exp…#17
maxkuttner wants to merge 1 commit into
mainfrom
refactor/nautilus-instrument-model

Conversation

@maxkuttner

Copy link
Copy Markdown
Owner

…licit feed mapping

Instrument seeding, data feeds and broker routing were entangled: instruments were created from Databento dataset "universes", and a single overloaded oms.instrument_xref table mixed market-data mapping with broker routing via a source_type discriminator. Feed-to-instrument wiring was implicit, relying on an instrument-class filter plus the coincidence that a provider's symbol sometimes equalled ours.

Restructured along the lines NautilusTrader uses: one canonical instrument catalog keyed (symbol, venue), with two independent mappings hanging off it.

Instruments are now seeded broker-first. Connecting a broker adapter discovers its tradable catalog, and that is the instrument set — sync-broker creates the master rows and the broker_instrument routing mapping in one pass. Databento demotes to a pure data feed. Dropped instrument_universe, instrument_universe_symbol and oms.instrument_xref.

Data feeds are independent of brokers and map 1:n: map-feed populates feed_instrument, so one feed symbol can price the same pair on several venues (the Bybit feed marks the Binance-seeded BTCUSDT).

Each feed owns its own symbology. FeedSymbology splits "which instruments can this feed price" (a declarative filter pushed into SQL) from "what does it call them" (a pure function), and both live on the feed struct beside the code that speaks that vendor's protocol. This replaces a central match holding every feed's rules as SQL literals, and makes the fiddly cases — Databento's space-padded OSI in particular — unit-testable without a database.

Venue resolution no longer fails silently. alpaca_exchange_to_mic returns Option instead of passing unknown labels through, the unwrap_or("") that manufactured empty-string venues is gone, and the catalog summary reports the distinct venue/currency codes that failed the FK check rather than only counting them.

Also removes the now-unreachable UniverseSource tree (its only consumer was an example), seeds crypto exchange venues that are absent from the ISO MIC registry but required by the venue FK, and splits feeds from broker connections in the cockpit with a new Architecture page documenting the model.

Verified against the dev database: Alpaca sync 15,876 instruments, Binance 1,249, feed mapping 31,488 rows unchanged across the symbology port (identical fingerprint, zero new rows). 92 tests pass.

…licit feed mapping

Instrument seeding, data feeds and broker routing were entangled: instruments
were created from Databento dataset "universes", and a single overloaded
`oms.instrument_xref` table mixed market-data mapping with broker routing via a
`source_type` discriminator. Feed-to-instrument wiring was implicit, relying on
an instrument-class filter plus the coincidence that a provider's symbol
sometimes equalled ours.

Restructured along the lines NautilusTrader uses: one canonical instrument
catalog keyed `(symbol, venue)`, with two independent mappings hanging off it.

Instruments are now seeded broker-first. Connecting a broker adapter discovers
its tradable catalog, and that is the instrument set — `sync-broker` creates the
master rows and the `broker_instrument` routing mapping in one pass. Databento
demotes to a pure data feed. Dropped `instrument_universe`,
`instrument_universe_symbol` and `oms.instrument_xref`.

Data feeds are independent of brokers and map 1:n: `map-feed` populates
`feed_instrument`, so one feed symbol can price the same pair on several venues
(the Bybit feed marks the Binance-seeded BTCUSDT).

Each feed owns its own symbology. `FeedSymbology` splits "which instruments can
this feed price" (a declarative filter pushed into SQL) from "what does it call
them" (a pure function), and both live on the feed struct beside the code that
speaks that vendor's protocol. This replaces a central match holding every
feed's rules as SQL literals, and makes the fiddly cases — Databento's
space-padded OSI in particular — unit-testable without a database.

Venue resolution no longer fails silently. `alpaca_exchange_to_mic` returns
`Option` instead of passing unknown labels through, the `unwrap_or("")` that
manufactured empty-string venues is gone, and the catalog summary reports the
distinct venue/currency codes that failed the FK check rather than only counting
them.

Also removes the now-unreachable `UniverseSource` tree (its only consumer was an
example), seeds crypto exchange venues that are absent from the ISO MIC registry
but required by the venue FK, and splits feeds from broker connections in the
cockpit with a new Architecture page documenting the model.

Verified against the dev database: Alpaca sync 15,876 instruments, Binance 1,249,
feed mapping 31,488 rows unchanged across the symbology port (identical
fingerprint, zero new rows). 92 tests pass.
@maxkuttner

Copy link
Copy Markdown
Owner Author

Superseded by #18 — same commit on a renamed branch.

@maxkuttner maxkuttner closed this Jul 19, 2026
@maxkuttner
maxkuttner deleted the refactor/nautilus-instrument-model branch July 19, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant