SPIKE: determine current state (token balances) retrieval strategy

[aristidesstaffieri]

Wallets need a way to index token balances and potentially other contract data for custom tokens.
We recently did a spike on this for Freighter and came to the conclusion that a good solution would be to have balances be indexed and to be able to track the TTL in order to display the appropriate UI during/after a balance data entry has expired.

Should wallet backend index token balances?
Should it also index things like decimals and symbol?
Should it track the TTL for the entries that are being indexed?

[JakeUrban]

Should wallet backend index token balances?

Given that the only alternative is querying the ledger state via RPC and the entry might be archived, I think the answer to this is yes.

However, something to note is that some tokens' balances are not the sum of all credits and debits of the token. Some tokens use a rebasing approach, where a user's balance increases or decreases based on a multiple stored and updated in the contract's state. In this case, the indexer would need to add support for ingesting changes to the multiplier, which would imply changes to all balances of the token. We can decide not to support this type of token in the wallet backend, at least initially, and default to reading ledger state within freighter on a special case basis.

Should it also index things like decimals and symbol?

This is tricky because the number of decimals and the token's symbol is a value returned from calling a function, and that function's implementation could change. Maybe we can account for this by monitoring for token contract's wasm being updated and calling these functions to fetch the potentially updated values.

Alternatives are assuming the number of decimals and the symbol is static, or working with the ecosystem to have these attributes in the contract's meta as static values.

Should it track the TTL for the entries that are being indexed?

I don't see why the indexer would need to track TTLs unless we wanted to create some kind of notification service that notified users that relevant entires are expiring soon. I'm not sure if its worth doing that vs. just letting them restore entries as needed.

[aristidesstaffieri]

However, something to note is that some tokens' balances are not the sum of all credits and debits of the token. Some tokens use a rebasing approach, where a user's balance increases or decreases based on a multiple stored and updated in the contract's state. In this case, the indexer would need to add support for ingesting changes to the multiplier, which would imply changes to all balances of the token. We can decide not to support this type of token in the wallet backend, at least initially, and default to reading ledger state within freighter on a special case basis.

It seems like this won't be the only variant of a token that we could decide to support, but I think even tokens that behave like your typical SEP-41 asset could have very different storage layouts for users balances. How would we approach indexing the data generically?

This is tricky because the number of decimals and the token's symbol is a value returned from calling a function, and that function's implementation could change. Maybe we can account for this by monitoring for token contract's wasm being updated and calling these functions to fetch the potentially updated values.

Could valid tokens implement a decimals or asymbol that changes without a wasm upgrade? This seems possible but maybe so edge case that we would not want to support it or care about when it happens.

I don't see why the indexer would need to track TTLs unless we wanted to create some kind of notification service that notified users that relevant entires are expiring soon. I'm not sure if its worth doing that vs. just letting them restore entries as needed.

The only reason is to be able to show a UI state when an entry related to an asset has been archived, but yeah it sounds like this isn't a huge win for the work it would take.

[JakeUrban]

How would we approach indexing the data generically?

Generally speaking off chain applications are expected to index a contract's state using the events the contract emits. SEP-41 defines when tokens should emit events and what their format should be. In fact, maybe the rebasing token could still emit transfer events when the multiplier is updated, which would remove the need for us to treat it as a special case.

Could valid tokens implement a decimals or a symbol that changes without a wasm upgrade?

Yes, the function could be implemented to return different values based on an arbitrary set of variables, its up to them. Thats partly why I think decimals and symbol should have been required meta attributes rather than functions.

The only reason is to be able to show a UI state when an entry related to an asset has been archived, but yeah it sounds like this isn't a huge win for the work it would take.

Yea I view this sort of functionality as an optimization, not a requirement. The default approach should be supporting restoration when necessary, and we can improve the experience opportunistically from there.

[JakeUrban]

I just spoke with the core team, and they said that in Protocol 23, transactions that require archived entires will automatically restore them prior to executing the transaction.

That means we'd be able to simulate calls to a token contract's balance() function and get the result even if the balance is archived.

Given that, we may not need to index token balances, or any ledger state for that matter. The wallet backend could index state transitions (i.e. token transfers), but when Freighter needs a user's current balance, it could use RPC for this.

cc @gouthamp-stellar