Skip to content

taprpc+tapdb: add GenesisInfo to AssetGroupBalance in ListBalances - #2130

Open
kaldun-tech wants to merge 1 commit into
lightninglabs:mainfrom
kaldun-tech:taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse
Open

kaldun-tech wants to merge 1 commit into
lightninglabs:mainfrom
kaldun-tech:taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse

Conversation

@kaldun-tech

@kaldun-tech kaldun-tech commented May 13, 2026 •

Copy link
Copy Markdown
Contributor

Add GenesisInfo and DecimalDisplay fields to AssetGroupBalance in the ListBalances RPC response when using group_by=group_key mode.

Previously, AssetGroupBalance only contained group_key and balance, making it impossible for clients to display asset metadata (name, type, decimal display) alongside grouped balances without additional RPC calls.

Changes

  • Proto: Added asset_genesis (field 3) and decimal_display (field 4) to AssetGroupBalance message
  • SQL: Modified QueryAssetBalancesByGroup to join genesis_info_view and return genesis data for a representative issuance per group
  • DB Store: Extended AssetGroupBalance struct with genesis fields and updated query result mapping
  • RPC Server: Populated new fields in listBalancesByGroupKey, including DecimalDisplay lookup via DecDisplayForAssetID

Design Notes

  • The representative genesis per group is selected using MIN(gen_asset_id), which gives the earliest-inserted genesis. For locally-minted assets, this is the group anchor. For fungible assets, all tranches share the same name/type, so any genesis provides correct metadata.
  • ErrAssetMetaNotFound leaves DecimalDisplay unset (asset has no meta); other errors are escalated to the caller.
  • Ungrouped assets are intentionally excluded from QueryAssetBalancesByGroup results (use group_by=asset_id mode for those).

Test Coverage Gaps

  1. Error handling paths in RPC layer: The ErrAssetMetaNotFound and error escalation paths in listBalancesByGroupKey are not directly unit tested. This would require mocking AddrBook.DecDisplayForAssetID.

  2. Received grouped assets without local anchor: Edge case where we receive a grouped asset but don't have the anchor locally. The new query should still work (joins through asset_groups.genesis_point_id), but this isn't explicitly tested.

Test Plan

  • make unit pkg=tapdb - verify DB layer changes
  • make unit pkg=rpcserver - verify RPC marshaling
  • Manual test run via make itest icase=minting_multi_asset_groups
  • tapcli assets balance --by-group returns genesis info
  • Manual test: JSON/REST response includes new fields

Fixes issue #2066

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the 'ListBalances' RPC functionality by including comprehensive metadata for grouped assets. By adding genesis information and decimal display settings to the 'AssetGroupBalance' response, clients can now display asset details directly without requiring additional API calls. The changes involve updates to the protocol buffers, database query logic, and the RPC server implementation to populate these new fields efficiently.

Highlights

  • RPC Response Enhancement: Added 'asset_genesis' and 'decimal_display' fields to the 'AssetGroupBalance' message in the 'ListBalances' RPC response to provide necessary asset metadata.
  • Database Query Optimization: Updated 'QueryAssetBalancesByGroup' to join with 'genesis_info_view', allowing the retrieval of representative genesis information for each asset group.
  • Data Struct Updates: Extended the 'AssetGroupBalance' struct in the database layer to include genesis fields, ensuring consistent data mapping from the SQL query results.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the ListBalances RPC by including representative genesis information and decimal display metadata for asset groups. The changes involve updating the database queries to select anchor genesis data, extending the AssetGroupBalance structures in both the database and RPC layers, and populating these new fields in the RPC server. Feedback was provided regarding a potential N+1 query performance issue when fetching decimal displays and a requirement to use structured logging for debug messages as per the repository style guide.

Comment thread rpcserver/rpcserver.go Resolved
Comment thread rpcserver/rpcserver.go Resolved
@kaldun-tech
kaldun-tech force-pushed the taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse branch 2 times, most recently from 2b0f3f4 to 48f4267 Compare May 13, 2026 15:47
@kaldun-tech

Copy link
Copy Markdown
Contributor Author

I believe these CI failures are flakes. @claude please code review and investigate whether the failed CI is due to code changes or flakes

@kaldun-tech
kaldun-tech marked this pull request as ready for review May 13, 2026 19:13
@kaldun-tech

Copy link
Copy Markdown
Contributor Author

@jtobin @darioAnongba Please re-run the failed CI jobs. My investigation is that the failures were due to flakes but let me know if you want me to look into anything on my end

@darioAnongba
darioAnongba self-requested a review May 22, 2026 13:55
Comment thread rpcserver/rpcserver.go Outdated
Comment on lines 1510 to 1523
if err == nil {
groupBalance.DecimalDisplay = fn.MapOptionZ(
decDisplay,
func(d uint32) *taprpc.DecimalDisplay {
return &taprpc.DecimalDisplay{
DecimalDisplay: d,
}
},
)
} else {
rpcsLog.DebugS(ctx, "Failed to fetch decimal display",
btclog.Fmt("asset_id", "%x", balance.ID[:]),
"error", err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can avoid silently logging debug and escalate instead.

Suggested change
if err == nil {
groupBalance.DecimalDisplay = fn.MapOptionZ(
decDisplay,
func(d uint32) *taprpc.DecimalDisplay {
return &taprpc.DecimalDisplay{
DecimalDisplay: d,
}
},
)
} else {
rpcsLog.DebugS(ctx, "Failed to fetch decimal display",
btclog.Fmt("asset_id", "%x", balance.ID[:]),
"error", err)
}
switch {
case errors.Is(err, address.ErrAssetMetaNotFound):
// Asset legitimately has no meta — leave DecimalDisplay unset.
case err != nil:
return nil, fmt.Errorf("unable to fetch decimal display "+
"for asset %x: %w", balance.ID[:], err)
default:
groupBalance.DecimalDisplay = fn.MapOptionZ(
decDisplay,
func(d uint32) *taprpc.DecimalDisplay {
return &taprpc.DecimalDisplay{DecimalDisplay: d}
},
)
}

Comment thread tapdb/sqlc/queries/assets.sql Outdated
-- works for locally-minted assets where the anchor is inserted first.
SELECT
tweaked_group_key,
MIN(gen_asset_id) as anchor_gen_id

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gen_asset_id is a local-DB primary key reflecting local insertion order, not protocol-level ordering or any canonical anchor. The canonical anchor is stored explicitly in asset_groups.genesis_point_id

A fix is to join through asset_groups.genesis_point_id to resolve the actual anchor gen_asset_id. The existing FetchGroupByGroupKey pattern (ORDER BY witness_id LIMIT 1) already does this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that gen_asset_id order was the wrong signal. I first switched to joining through asset_groups.genesis_point_id as you suggested, but George found that pointer can move to a tranche this node has no witness for (e.g. after creating an address for a new tranche), and the inner join then dropped the group from the response. The representative is now chosen from witnessed rows with MIN(witness_id), the same ordering as the FetchGroupByGroupKey query you pointed to. Details are in George's thread on this query.

@github-project-automation github-project-automation Bot moved this from 🆕 New to 👀 In review in Taproot-Assets Project Board May 26, 2026
@kaldun-tech

Copy link
Copy Markdown
Contributor Author

Hey @darioAnongba thanks for the feedback. I'm making a few changes:

  1. SQL Query (assets.sql): Changing from MIN(gen_asset_id) to joining through asset_groups.genesis_point_id for canonical anchor selection.
  2. Error Handling (rpcserver.go): Changing from silent debug logging to proper error handling - ErrAssetMetaNotFound leaves DecimalDisplay unset, other errors are escalated.
  3. Unit Tests (assets_store_test.go): Adding assertions to verify genesis info fields match expected anchor genesis for each group.

@kaldun-tech
kaldun-tech force-pushed the taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse branch from 48f4267 to 44d12a0 Compare May 27, 2026 15:48
@kaldun-tech

Copy link
Copy Markdown
Contributor Author

@darioAnongba Pushed my changes

  1. SQL query now joins through asset_groups.genesis_point_id for canonical anchor selection
  2. Error handling escalates non-ErrAssetMetaNotFound errors instead of silent logging
  3. Added DB layer test assertions verifying genesis info matches the expected anchor

@kaldun-tech

kaldun-tech commented May 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Investigated the CI failure. Copilot & Claude pointed me at PR #2133 but this is not yet merged which indicates the failure is due to flakiness. Sergey's changes in PR #2060 may correct this.

EDIT: Second failure today that seems to be caused by issue #2120 and fixed by PR #2132. Will investigate

@kaldun-tech

Copy link
Copy Markdown
Contributor Author

@GeorgeTsagk @jtobin Tagging for review

@darioAnongba
darioAnongba removed their request for review July 9, 2026 22:45
@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

3 similar comments
@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@darioAnongba: review reminder
@kaldun-tech, remember to re-request review from reviewers when ready

@kaldun-tech
kaldun-tech force-pushed the taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse branch from 44d12a0 to 13630f6 Compare September 3, 2026 04:07
@darioAnongba
darioAnongba removed their request for review September 7, 2026 11:24

@GeorgeTsagk GeorgeTsagk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sticking with this one. A few housekeeping items first, they make the PR much easier to pick up for whoever reviews next:

  1. Rebase on main. The branch is 18 commits behind and GitHub shows it as conflicting.
  2. Move the release note to release-notes-0.9.0.md, 0.8.x is already out.
  3. Rewrite the commit message so it matches the code. It still describes the MIN(gen_asset_id) approach and mentions merge conflict fixing.
  4. Drop the UpsertAssetGroupKey ON CONFLICT change. It is not part of what this PR advertises and it changes behavior for every group upsert path.

Inline I found two correctness issues, one of which drops groups from the response. Details on the lines.

Comment thread rpcserver/rpcserver.go
ctx, balance.ID,
)
switch {
case errors.Is(err, address.ErrAssetMetaNotFound):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This never matches. DecDisplayForAssetID wraps the store error with %v, so errors.Is fails and we fall into the escalation branch. Any grouped asset whose anchor has no meta makes the whole by-group ListBalances call error out. The itest assets all carry meta, which is why it is not caught.

Fix is %w in address/book.go DecDisplayForAssetID plus a test for a group without meta. Also note that a cache miss there goes through FetchAssetMetaForAsset, which with the universe syncer wired in will try a federation sync on every call for such assets.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. DecDisplayForAssetID now wraps with %w, so the ErrAssetMetaNotFound branch matches and a group whose representative has no meta leaves decimal_display unset instead of failing the whole call. Added TestBook_DecDisplayForAssetID_NoMeta to assert the errors.Is chain holds.

On the federation sync: agreed. A miss isn't cached, so with the syncer wired in, every call for a meta-less asset retries SyncAssetInfo. That's existing DecDisplayForAssetID behavior shared by the other rpcserver call sites (ListAssets etc.), so I'd suggest a follow-up that caches misses in the address book rather than growing this PR. Can fold it in here if you'd prefer

Comment thread tapdb/sqlc/queries/assets.sql Outdated

-- name: QueryAssetBalancesByGroup :many
WITH group_anchor AS (
-- Select the canonical anchor genesis for each group by joining through

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two problems with this CTE.

  1. It is an inner join through asset_groups.genesis_point_id and asset_group_witnesses. When the group row was first created via an address for tranche A (no witness) and we later receive tranche B, there is no witness for A and the group disappears from the by-group balances entirely. Main returns the group in that scenario, I reproduced both. Freezing genesis_point_id in the upsert below makes this permanent.

  2. When the anchor and another tranche share a genesis point (same mint batch, exactly what the multi asset group itest does) the CTE returns one row per tranche. Balance stays correct but the representative genesis is whichever row the DB emits last, so it differs between runs and backends.

Suggestion: pick the representative from asset_group_witnesses joined to key_group_info_view, collapse to one row per group with MIN over the witness gen ids, and LEFT JOIN so a missing anchor never hides a balance. Add tests for the no-witness anchor and shared genesis point cases, and assert the exact representative.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworked along the lines you suggested. From Opus:

  1. group_anchor now picks the representative from key_group_info_view, so only witnessed tranches are considered, collapsed to one row per group with MIN(witness_id). It no longer goes through asset_groups.genesis_point_id, so repointing the anchor at an unreceived tranche can't drop the group. It's also LEFT JOINed, so a missing representative never hides a balance.
  2. MIN(witness_id) makes the pick deterministic when tranches share a genesis point, independent of row order or backend.

Tests:

  • TestQueryAssetBalancesByGroupNoWitnessAnchor reproduces the address-then-receive case and asserts the balance and representative come from the witnessed tranche.
  • TestQueryAssetBalancesByGroupSharedGenesisPoint mints two tranches on one genesis point, asserts the first-witnessed tranche is the representative, and checks the pick is stable across calls. It passes on both SQLite and Postgres in CI.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the UpsertAssetGroupKey: Reverted. this is now identical to main, and the fix is entirely in QueryAssetBalancesByGroup. The scenario you described is covered by TestQueryAssetBalancesByGroupNoWitnessAnchor, which repoints genesis_point_id using main's existing ON CONFLICT DO UPDATE

Comment thread tapdb/sqlc/queries/assets.sql Outdated
-- Preserve the original genesis_point_id (the anchor's genesis point).
-- We update version to ensure RETURNING works, but genesis_point_id
-- must remain unchanged as it identifies the canonical group anchor.
DO UPDATE SET version = EXCLUDED.version

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to the PR and untested. This changes what every group upsert does (mint, proof import, universe, address book) and now also overwrites version on conflict. Please revert and fix the query instead.

burns.

- [PR#2130](https://github.com/lightninglabs/taproot-assets/pull/2130)
adds `asset_genesis` and `decimal_display` fields to `AssetGroupBalance`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Belongs in release-notes-0.9.0.md.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to 0.9

@kaldun-tech
kaldun-tech force-pushed the taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse branch from 13630f6 to e66ad6d Compare September 12, 2026 04:55
Add GenesisInfo and DecimalDisplay fields to the AssetGroupBalance
message, so the group_by=group_key mode of ListBalances returns asset
metadata alongside balances without a follow-up call per group.

QueryAssetBalancesByGroup gains a group_anchor CTE that selects a
representative genesis per group from key_group_info_view, picking the
row with MIN(witness_id) for deterministic results when multiple
tranches share a genesis point. The CTE is LEFT JOINed so a group whose
anchor tranche has no local witness yet still returns its balance, just
without genesis fields, instead of being dropped from the response.

DecDisplayForAssetID in address/book.go now wraps its store error with
%w instead of %v, so errors.Is(err, ErrAssetMetaNotFound) correctly
detects assets with no known meta instead of falling through to a hard
error.

Add tapdb tests for the no-witness-anchor and shared-genesis-point
cases, and an address book test for DecDisplayForAssetID on an asset
with no meta.

Fixes lightninglabs#2066.

Signed-off-by: kaldun-tech <tsmereka@protonmail.com>
@kaldun-tech
kaldun-tech force-pushed the taprpc-add-GenesisInfo-AssetGroupBalance-ListBalancesResponse branch from e66ad6d to e137e0c Compare September 12, 2026 21:37
@kaldun-tech

kaldun-tech commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor Author

@GeorgeTsagk @darioAnongba thanks for the thorough reviews. Everything should now be addressed

Side note, I'm working on an AI startup contract which keeps me pretty busy. Aiming to get more contributions rolling when I figure out time management.

Housekeeping:

  1. Rebased on main
  2. Release note moved to release-notes-0.9.0.md
  3. Commit message rewritten to describe the final design
  4. UpsertAssetGroupKey change dropped

Correctness:

  • DecDisplayForAssetID wraps with %w, so the missing-meta branch actually matches, with a test
  • The group_anchor CTE picks the representative from witnessed rows using MIN(witness_id) and is LEFT JOINed, with tests for the no-witness-anchor and shared-genesis-point cases
  • Dario's error-handling suggestion is in. His genesis_point_id suggestion was replaced by the approach above, explained inline.

One open question inline: misses in DecDisplayForAssetID aren't cached, so each call re-runs the federation sync. I'd suggest handling that in a follow-up PR. Let me know if you'd rather it land here.

@kaldun-tech

Copy link
Copy Markdown
Contributor Author

@GeorgeTsagk @jtobin could one of you re-run the failed unit-cover job? Looks like a flake
https://github.com/lightninglabs/taproot-assets/actions/runs/34720486028/job/103625321106

It failed on TestWatcherShrinkingReorg in tapreorg, which this PR doesn't touch. The failure was a require.Eventually timeout at service_test.go:1446, where it waits for the anchoring to go back to
Unwitnessed after the watcher restarts. The same code passed unit-cover on the previous push; the only change since was a one-line release-note fix.

I couldn't reproduce it locally:

  • make unit pkg=tapreorg case=TestWatcherShrinkingReorg nocache=1: 11/11
    passes, ~1s each (the CI run took 34s before timing out)
  • make unit-race pkg=tapreorg nocache=1: pass

Looks like a load-related flake under unit-cover's parallel run.

@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

3 similar comments
@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@kaldun-tech, remember to re-request review from reviewers when ready

@darioAnongba
darioAnongba requested review from darioAnongba and removed request for darioAnongba October 12, 2026 16:13
@litbot-9000

Copy link
Copy Markdown
Collaborator

@GeorgeTsagk: review reminder
@darioAnongba: review reminder
@kaldun-tech, remember to re-request review from reviewers when ready

3 similar comments
@litbot-9000

Copy link
Copy Markdown
Collaborator

@GeorgeTsagk: review reminder
@darioAnongba: review reminder
@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@GeorgeTsagk: review reminder
@darioAnongba: review reminder
@kaldun-tech, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@GeorgeTsagk: review reminder
@darioAnongba: review reminder
@kaldun-tech, remember to re-request review from reviewers when ready

@darioAnongba
darioAnongba removed their request for review November 9, 2026 13:35
@litbot-9000

Copy link
Copy Markdown
Collaborator

@GeorgeTsagk: review reminder
@kaldun-tech, remember to re-request review from reviewers when ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 👀 In review

Development

Successfully merging this pull request may close these issues.

4 participants