Skip to content

loopd: track lnd connection ownership, don't close borrowed conns - #1272

Open
litbot-9000 wants to merge 2 commits into
lightninglabs:masterfrom
litbot-9000:fix/lnd-conn-ownership
Open

litbot-9000 wants to merge 2 commits into
lightninglabs:masterfrom
litbot-9000:fix/lnd-conn-ownership

Conversation

@litbot-9000

@litbot-9000 litbot-9000 commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

The bug

When the loop daemon runs as an integrated subserver of lightning-terminal (LiT), it receives lnd services backed by a connection that LiT creates once and shares among all its subservers (loop, pool, faraday, tapd) via StartAsSubserver.

On shutdown the daemon unconditionally closed that borrowed connection:

  • d.stop() (both the graceful Stop() path and the runtime-error path via internalErrChan) called d.lnd.Close() on the shared connection.

Once that borrowed connection is closed, everything else in the LiT process that uses it fails with grpc: the client connection is closing — the LiT macaroon service (signrpc.DeriveSharedKey), tapd, faraday and accounts — while the remote lnd stays perfectly healthy. This is the root cause behind the remote-lnd startup failure reported in lightninglabs/lightning-terminal#1381 (comment) (litd 0.17.8 + remote lnd 0.21.4-beta failing 12ms after "Starting LiT macaroon service"), as directed in lightninglabs/lightning-terminal#1329 (comment).

Pool had the identical bug, fixed in the sister PR lightninglabs/pool#528. Faraday already tracked connection ownership correctly (lndOwned), and tapd never receives the GrpcLndServices wrapper, so neither needed a fix. LiT itself already closes the shared client exactly once in shutdownSubServers (terminal.go), after stopping all subservers, so no lightning-terminal change is needed for the ownership model. The ownership model is now documented (with diagrams) in lightninglabs/lightning-terminal#1458.

The fix (two layers)

Layer 1 — ownership tracking (commit 1). Mirror faraday's model: Start() (standalone) marks the connection as owned, StartAsSubserver() marks it borrowed, and stop() only closes owned connections.

Layer 2 — type-level prevention (commit 2). Make borrowed-ness unrepresentable. StartAsSubserver now accepts the non-closable *lndclient.LndServices view instead of the *lndclient.GrpcLndServices wrapper, so a subserver cannot call Close on the shared connection even if it tries. The closable wrapper is only held in the new lndOwner field, which is set only when the daemon creates the connection itself (standalone Start, and the view command whose local was renamed to ownedLnd for greppability). Judge: the type checker.

Note for integrators: this changes the exported StartAsSubserver signature. LiT's call site (subservers/loop.go) must pass &g.lndClient.LndServices instead of g.lndClient when bumping the loop dependency.

Verification (each layer demonstrated to fail pre-fix)

  • loopd/daemon_ownership_test.go starts a real (bufconn, TLS) mock lnd, builds a real GrpcLndServices with a real injected grpc.ClientConn exactly like LiT does, starts an embedded daemon with the borrowed services, and asserts the injected conn does NOT enter connectivity.Shutdown — for both a graceful Stop() and a runtime-error stop (internalErrChan). TestOwnedLndConnClosedOnStop asserts an owned connection is still closed (no standalone leak). Both subserver subtests fail on master ("subserver closed the shared lnd connection") and pass with the fix.
  • loopd/ownership_guard_test.go is a static guard (AST scan of the package, stdlib-only) that fails CI if any .Close() targets an lnd connection handle outside the whitelisted owner sites, or if StartAsSubserver ever accepts the closable wrapper again. Demonstrated on master: it flags the original bug site (daemon.go:1173 d.lnd.Close()), the wrapper-typed signature, and the view.go close.
  • Full loopd package + root package tests pass; golangci-lint clean.

Skipped as low-value-after-type-level: a formal TLA+-style spec — the lifecycle has four states and one invariant ("a subserver stopping never closes the shared conn"), which is now enforced by the type checker, the regression tests and the static guard; the state machine is sketched in the litd docs PR instead.

The reconnect/restart work from lightning-terminal#644 is intentionally kept separate and out of scope.


View the OpenClaw team session

When running as a subserver inside lightning-terminal, the loop daemon
receives an lnd connection that is created and owned by the parent
process and shared among all subservers. On shutdown, both graceful
and after a runtime error, the daemon unconditionally closed that
connection, tearing down lnd access for LiT itself and all other
integrated subservers (pool, faraday, tapd, accounts and the LiT
macaroon service).

Track the connection ownership like faraday already does: the
standalone Start path marks the connection as owned, StartAsSubserver
marks it as borrowed and stop only closes owned connections.

Add a regression test that starts an embedded daemon against a mock
lnd with a real injected grpc.ClientConn and asserts the connection
survives both a graceful Stop and a runtime-error shutdown, plus a
test that an owned connection is still closed on stop.

Refs lightninglabs/lightning-terminal#1381 (comment 6530026672).
@litbot-9000
litbot-9000 force-pushed the fix/lnd-conn-ownership branch from ee36cc5 to 5b570d8 Compare October 30, 2026 19:27
Strengthen the ownership fix from a runtime flag to type-level
prevention: StartAsSubserver now accepts the non-closable
*lndclient.LndServices view instead of the *lndclient.GrpcLndServices
wrapper, so a subserver cannot call Close on the shared connection
even if it tries. The closable wrapper is only held in the new
lndOwner field, which is only set when the daemon creates the
connection itself (standalone Start and the view command).

Add a mechanically enforced guard test that fails CI if any code in
the package closes an lnd connection handle outside the whitelisted
owner sites, or if StartAsSubserver ever accepts the closable wrapper
again. The guard was verified to fail against master (flagging the
original d.lnd.Close() bug site and the wrapper-typed signature) and
to pass with this change.
@litbot-9000

Copy link
Copy Markdown
Collaborator Author

/gateway review

@lightninglabs-gateway lightninglabs-gateway Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gateway review — 2 findings

🔴 0 Blocker · 🟠 0 Major · 🟡 2 Minor · 🔵 0 Nit

Summary

The PR stops the embedded loop daemon from closing the lnd connection that LiT shares with all its subservers. It does this in two steps. Daemon now keeps a closable lndOwner handle only when it created the connection itself. StartAsSubserver (and the rest of the daemon) now get only the non-closable *lndclient.LndServices view. The core change is correct: stop() closes only d.lndOwner (loopd/daemon.go:1178). Start() sets both fields from the one connection it creates (loopd/daemon.go:170-171). All former &d.lnd.LndServices / d.lnd.LndServices call sites were converted the same way. The regression test drives a real StartAsSubserver against a bufconn/TLS mock lnd, and both shutdown paths (Stop() and internalErrChan) are covered.

The remaining concerns are about API scope and test coverage, not correctness. This PR changes the signature of a second exported function that the PR description and release notes don't mention. The standalone-ownership test also builds its own Daemon state instead of going through Start(), so the line that hands ownership to the daemon has no test.

Lower-severity findings (2)

🟡 F1 (Minor) — Exported loop.Resume signature change is undocumented · loopout.go:1569

The exported loop.Resume in the root loop package now takes *lndclient.LndServices instead of *lndclient.GrpcLndServices, which breaks external library callers at compile time. The PR description only warns integrators about StartAsSubserver, and the "Breaking Changes" section of docs/release-notes/release-notes-next.md is still empty. Add a Breaking Changes entry that names both Daemon.StartAsSubserver and loop.Resume, with the &g.lndClient.LndServices migration hint.

🟡 F2 (Minor) — Owned-connection test skips Start's ownership assignment · loopd/daemon_ownership_test.go:407

TestOwnedLndConnClosedOnStop sets d.lnd/d.lndOwner by hand and then calls d.stop(), so it never runs the d.lndOwner = lndGrpc assignment in Start() (loopd/daemon.go:170). If someone drops that line, standalone mode silently leaks the lnd connection, and neither this test nor the AST guard would catch it. Consider calling Start() with a ListenerCfg.getLnd that returns the mock services, or at least asserting d.lndOwner != nil after a standalone start.

Reviewed at medium effort.

Bot commands
  • /gateway re-review — re-run after pushing changes (maintainers)
  • /gateway dismiss <id> — silence a finding (maintainers)
  • /gateway explain <id> — elaborate on a finding (anyone)

@lightninglabs-gateway

Copy link
Copy Markdown

🤖 gateway audit metadata for this PR — auto-generated, please don't edit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant