Skip to content

NETWORK_TRUST.md

Latest commit

 

History

History
262 lines (202 loc) · 9.36 KB

NETWORK_TRUST.md

File metadata and controls

262 lines (202 loc) · 9.36 KB
Raw

Network-Scoped Federation Trust (Locked Decision #12)

Status: active

How SDWAN routing integrates into federation trust, how pessimistic grants gate cross-peer access, and what headers the reverse proxy MUST forward to make the auth chain work.

Plan reference: Decentralized Federation §K + Locked Decision #12.

The trust shape

Federation v1 (pre-K) treated SDWAN as an optional transport. A peer could federate over public WAN with endpoints: [{scope: wan, ...}] and never join the overlay mesh. Grants were scoped to (peer, remote_subject, resource_kind, resource_id, scopes).

Locked Decision #12 makes SDWAN a first-class participant in trust:

  1. Federation handshake includes a network-bridge negotiation step recorded in system_federation_network_bridges (peer × sdwan_network with state machine).
  2. FederationGrant gains three pessimistic-scope allowlists — node_instance_ids, sdwan_network_ids, source_cidrs — that the auth chain enforces.
  3. The reverse proxy (Traefik) forwards trust metadata (calling instance, SDWAN network, source IP) that the auth chain consumes.

Together: a request is denied unless the calling NodeInstance, the SDWAN network the request arrived over, AND the source IP all match the grant's allowlists. Every axis is stated explicitly: concrete entries, or ["*"] (ANY) for no restriction on that axis. A blank allowlist denies, and validation refuses to save one.

What the reverse proxy must forward

Every federation_api request that crosses the proxy → backend hop needs three headers set by the proxy (Traefik). The backend trusts these headers because the proxy → backend hop is itself mTLS- authenticated against the platform's internal CA.

X-Calling-Instance

The NodeInstance.id of the calling peer's process. Extracted from the client cert's URI: SAN encoded as powernode://instance/<uuid>.

Traefik config (sketch):

http:
  middlewares:
    extract-instance-from-cert:
      passTLSClientCert:
        info:
          subject:
            sans: true

A subsequent middleware (or a Lua plugin) parses the URI SAN and sets X-Calling-Instance. Future hardening: validate the SAN format and reject any cert without one for federation_api endpoints.

X-Sdwan-Network

The SDWAN network ID the request arrived through. Traefik binds distinct listeners to distinct SDWAN VIPs at deploy time; each listener is annotated with the corresponding Sdwan::Network.id. The listener writes the static value as the header.

http:
  routers:
    federation-api-on-trusted-overlay:
      rule: "Host(`fd00:trusted::100`) && PathPrefix(`/api/v1/system/federation_api`)"
      service: federation-api-backend
      middlewares:
        - inject-sdwan-network-trusted

  middlewares:
    inject-sdwan-network-trusted:
      headers:
        customRequestHeaders:
          X-Sdwan-Network: "019fab12-3456-7890-abcd-ef0123456789"

The backend additionally validates that the supplied network ID corresponds to an active FederationNetworkBridge for the calling peer. Without an active bridge, the network value has no meaning and the request is denied even if the header is set.

X-Forwarded-For

Standard. The platform reads request.remote_ip after Rails resolves trusted proxies. Configure config.action_dispatch.trusted_proxies to include the proxy's internal address.

Pessimistic-grant matching algorithm

For each allowlist on a FederationGrant:

return forbidden unless grant.applies_to_instance?(request.headers["X-Calling-Instance"])
return forbidden unless grant.applies_to_network?(request.headers["X-Sdwan-Network"])
return forbidden unless grant.applies_to_source_ip?(request.remote_ip)

Each predicate:

  • Returns true when its allowlist is exactly ["*"] (ANY — no restriction on this axis, even when the header is absent).
  • Returns true when the supplied value is in a concrete allowlist.
  • Returns false when the allowlist is concrete but the supplied value is missing or doesn't match.
  • Returns false when the allowlist is blank. Blank means deny, never "unrestricted".

All three axes are AND-combined, so every axis must pass. FederationGrant#unrestricted? is true only when all three are ANY.

Validation refuses a blank axis, and refuses "*" mixed with concrete entries. A blank row can therefore only exist if it was written around validation, and such a row denies every call.

How operators compose pessimistic grants

A typical "very pessimistic" grant for migration of sensitive data:

System::FederationGrant.create!(
  account: alice_account,
  federation_peer: bob_peer,
  grantor_user: alice,
  remote_subject: "bob@peer-b",
  resource_kind: "skill",
  resource_id: skill_x.id,
  permission_scopes: %w[read migrate],

  # Pessimistic axes:
  node_instance_ids: [ bob_api_node.id ],     # only this one node may use the grant
  sdwan_network_ids: [ overlay_trusted.id ],  # only over this network
  source_cidrs: %w[fd00:abcd:1234::/48],      # only from this prefix

  expires_at: 30.days.from_now
)

A request from bob_api_node on overlay_trusted from a fd00:abcd:1234:* address: allowed.

A request from any of the following: denied.

  • A different node on bob's peer
  • Over a different SDWAN network
  • From a source IP outside the declared prefix
  • Missing any of the required headers

Sovereign Auth Handshake

sequenceDiagram
    actor PeerB as Peer B<br/>(calling)
    participant Proxy as Traefik<br/>(at peer A)
    participant Auth as Federation auth chain<br/>(backend)
    participant Grant as FederationGrant<br/>predicates
    participant Bridge as FederationNetwork<br/>Bridge

    PeerB->>Proxy: HTTPS /federation_api/...<br/>(mTLS client cert with URI SAN)
    Proxy->>Proxy: passTLSClientCert + URI SAN parse<br/>→ X-Calling-Instance: instance-uuid
    Proxy->>Proxy: listener bound to SDWAN VIP<br/>→ X-Sdwan-Network: network-uuid
    Proxy->>Proxy: standard X-Forwarded-For
    Proxy->>Auth: request + headers + remote_ip
    Auth->>Bridge: validate active bridge<br/>for (peer, network)
    Bridge-->>Auth: bridge active
    Auth->>Grant: applies_to_instance? + applies_to_network? + applies_to_source_ip?
    alt all three allowlists match (or are ANY ["*"])
        Grant-->>Auth: allowed
        Auth-->>Proxy: 200 + response payload
        Proxy-->>PeerB: 200
    else any allowlist mismatches (or is blank)
        Grant-->>Auth: forbidden
        Auth-->>Proxy: 403 with allowlist contents in body
        Proxy-->>PeerB: 403 ("calling instance not in grant allowlist" etc.)
    end
Loading

Bridge state machine

stateDiagram-v2
    [*] --> proposed: federation handshake creates bridge
    proposed --> active: accept!
    proposed --> revoked: operator revoke before accept
    active --> suspended: suspend! (temporary disable)
    suspended --> active: resume! (suspend was temporary)
    active --> revoked: operator revoke (terminal)
    suspended --> revoked: operator revoke (terminal)
    revoked --> [*]
Loading

A bridge is created at federation handshake time in proposed state. Accepting the federation invitation transitions it to active. Operator may suspend! to temporarily disable traffic over that bridge without losing the configuration; resume by transitioning back to active. Revocation is terminal — the bridge must be recreated.

Diagnostic / debugging

If a federation_api call returns 403 with "calling instance not in grant allowlist," inspect:

grant = System::FederationGrant.find_by_bearer_token("fgs.<id>.<signature>")
grant.node_instance_ids   # what's allowed (["*"] = any, [] = deny)
request.headers["X-Calling-Instance"]   # what was supplied

Repeat for network and source IP. The error message includes the allowlist contents to make this triage straightforward.

Explicit ANY, no implicit unrestricted

There is no back-compat reading of an empty allowlist. The columns default to [], and a blank axis is refused at validation, so every creator states each axis: the operator Grants editor and POST /api/v1/system/platform/peers/:peer_id/grants, the service-catalog subscription grants (both the issuing side and the receiving side's local record), the managed-child accept cascade, and the edge-exposure smoke seed. The system-issued grants write ["*"] on every axis. The axis check runs when a grant is created or an axis changes, so a row written around validation with a blank axis denies but can still be revoked and archived.

The FederationManager AI Skill surfaces grants carrying admin or migrate permission scope as broad_scope_grants findings, marking grants that warrant tightening. Pair them with #unrestricted?, which means ANY on every pessimistic axis, to spot a broad scope that is also unscoped.

See also

  • SOCIAL_CONTRACT.md — operator commitments (#3 truthful capability, #6 no-undermining)
  • ../FEDERATION_MULTI_SITE_GUIDE.md §4b + ../runbooks/expose-service.md — Traefik / reverse-proxy configuration (this doc supplements them)
  • app/models/system/federation_grant.rb — #applies_to_*? and #unrestricted? predicates
  • app/models/system/federation_network_bridge.rb — bridge model + state machine
  • app/controllers/api/v1/system/federation_api/base_controller.rb — full auth chain

Last verified: 2026-06-03