[febyeji]

This PR adds a new compact block filter chain source (via kyoto) to ldk-node. Let's go through its features.

Design notes

Contrary to other chain sources, we use a push-based model: we don't proactively request data, but rather listen
for events and process them.

Kyoto

Kyoto is a standalone node which comes in the form of a library. It manages peers and provides users with compact
block filters as well as the ability to request blocks on demand.

Getting data from kyoto is done via a channel which emits events (e.g. IndexedFilter which means we received a new
compact block filter).

Operation

In broad terms we operate in the following way:

1. On start, build and run the kyoto node
2. Listen for kyoto events
3. Process kyoto events and respectively apply needed operations (e.g. download full block to apply it).

There is no concept of "active syncing": when the chain source runs, it always processes new events and applies them.
Method sync_wallet waits until the application of all blocks up to the known tip.

Node aliveness

As we are responsible for always having an operational node, we have its runtime status, and if it goes down by any
reason, we try to restart the kyoto node. If something went wrong and after MAX_RESTART_RETRIES=5 the node still died,
we give up.

Scripts

CBFChainSource tracks watched scripts provided from the onchain wallet as well as the lightning wallet.

We need to track two types of scripts: ones provided from the onchain wallet (e.g. to fund the lightning node) and
"LDK-scripts", exposed and needed to be tracked during channel operation (e.g. channel force-close transactions). The
former scripts are revealed via bdk wallet. We don't maintain a CBFChainSource-related copy of them, rather we just ask
bdk wallet each time we need them. This is done to have a single source of truth.

Thus, registered_scripts are LDK-revealed scripts.

Block applicator

This entity applies blocks to the state of the node. (Perhaps its title can be changed to something more concise.)

It runs as its own background task, decoupled from kyoto event processing via an unbounded channel of ChainOps:

• ConnectFull — filter matched, apply the downloaded full block (block_connected)
• ConnectFiltered — filter didn't match, apply header only (filtered_block_connected with no transactions)
• Disconnect — reorg, disconnect back to the fork point
• Synced — kyoto reported FiltersSynced, i.e. we caught up to the network tip
• Failed — an unrecoverable error occurred while producing ops (surfaces to sync_wallets waiters)

For applying blocks we reuse bitcoind's ChainListener, which fans each connect/disconnect out to the onchain
wallet, channel manager, chain monitor and output sweeper.

process_kyoto_events is a single sequential loop: for a matched filter it awaits the full block fetch (with
retries/timeout) inline, and for an unmatched one it awaits the header fetch inline, before moving on to the next
kyoto event — this is what ensures ops are produced in order. The block applicator then just consumes those ops
in order and applies them. As a safety net it also tracks the next expected height itself and skips any
out-of-sequence op (e.g. duplicates, or stale ops produced before a reorg); a Disconnect rewinds the expected
height to the fork point.

Besides applying blocks, the applicator is the single writer of CbfSyncState: it advances applied_tip /
synced_to_tip (which sync_wallets waits on) and persists the node's wallet-sync timestamps once the tip is
reached. When the native kyoto fee source is used, it also opportunistically caches the coinbase-derived fee rate
of every full block it applies, so the fee estimator doesn't have to download those blocks again.

There is room for improvement: we could download full blocks in parallel (e.g. using
https://docs.rs/futures/latest/futures/stream/struct.FuturesOrdered.html), which would speed up initial sync.

We could do without this additional entity, by just processing ops in place, but we stick to this design for
clarity.

Fee source

As we don't have mempool with CBF chain source, we cannot properly estimate the transaction fees.
There are three options for a fee source provider (FeeSource):

• native kyoto estimation via extrapolation from recent blocks' fees
• esplora
• electrum

Using esplora or electrum just for the fees is privacy-preserving and acceptable.
Using native extrapolation is a fallback method, because it has two flaws:

• we need to download all blocks, which defeats kyoto's bandwidth advantage
• the estimation is still rough, as we have no mempool

Also, recently fetched blocks (when we need to download a full block anyway) are cached so they won't be downloaded
twice during a fee update.

Reorganizations

On reorgs kyoto emits events about disconnected blocks and we disconnect them accordingly.

CbfSyncState

We need that struct to distinguish between last block that kyoto has and the last block we have applied locally, these
may be different blocks. Also used to understand whether we processed everything up to kyoto's tip.

Testing

We tried to adhere to already existing tests, as they cover the vital node operation scenarios. Though some of the tests
are not meant for kyoto (for example tests related to mempool, 0-conf channels, RBF).

As we want to test exclusively the new chain source in this PR, we've added an environment variable overriding the
random chain source choice, it can be used on the following way:

LDK_TEST_CHAIN_SOURCE=cbf cargo test --test integration_tests_rust

Other available options: esplora, electrum, bitcoind-rpc, bitcoind-rest.

Also please note sometimes in test scenarios kyoto bans bitcoind because of this bug:
bitcoin/bitcoin#35354.

Previously tests assumed that the chain source and the mining node are the same entity, i.e. there is no gossip delay.
This is not the case with the kyoto node, that's why one might experience a race condition due to gossip propagation
(we do something, like mine a block with a needed transaction, then it is correctly shown in esplora/electrum/bitcoind
and we try to invoke some action, but the new block has not yet propagated to the kyoto node); the situation is
worsened by the fact that GitHub's CI runners are slower than local machines, and a simple time-based wait does not
work well to circumvent the propagation delay. That's why many functions in the tests have additional waiting for
propagation.

Tests not comptible with the compact block filter node:

• channel_full_cycle_0conf,
• channel_full_cycle_0conf_0reserve
• onchain_send_receive
• onchain_fee_bump_rbf,
• onchain_fee_bump_rbf_respects_anchor_reserve

History

We switched to bitcoind's sync model (poll_and_update_listener vs sync_onchain_wallet and sync_lightning_wallet).

We tried to follow the patterns for already existing chain sources, more precisely we had distinct sync_onchain_wallet
and sync_lightning_wallet (dictated by transaction-based syncing sources) functions however it is suboptimal, because
we might fetch the same block twice.

[ldk-reviews-bot]

👋 I see @tankyleo was un-assigned.
If you'd like another reviewer assignment, please click here.

[ldk-reviews-bot]

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

[tnull]

@febyeji @randomlogin Please let me know when you consider merging of the two PRs completed and this is ready for a first round of review!

[rustaceanrob]

bip157 version 0.5.0 includes a Package type that may be used here.

@tnull do I understand correctly that Vec<Transaction> is used so the traits are forward compatible for arbitrary package lengths? My crate only supports packages of size 1 and 2, as per the current Bitcoin Core policy.

[tnull]

Hmm, well, kinda. LDK's trait uses a &[&Transaction] I believe. Generally we prefer to leave any transaction/package validation to Core via. the chain source. Given bip157 is the chain source, it should be fine to use from_vec and log an error if that fails for whatever reason.

[randomlogin]

I've started to test it in the wild (on mutinynet with ldk-server) and encountered this problem:

3. Different listener heights: Replaying from the lowest height sends old blocks to listeners already ahead, triggering a chain-order assertion. Each listener needs synchronization from its own tip.

Working on that.

I'll update the PR with new commits with all the suggestions.