Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions docs/guest-api-auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,10 @@ public_keys = ["2f8d...c1"] # hex Ed25519 public keys, 32 bytes each
```

`name` is the exact `iss` claim a token must carry; `public_keys` is a list so a
key can be rotated by publishing the new one beside the old, waiting out the
longest token lifetime (one hour), then dropping the old one.
key can be rotated by publishing the new one beside the old, waiting until the
tokens it signed have expired, then dropping the old one. The agent does not
cap token lifetimes; there is no revocation list, so `exp` is the issuer's
only exposure control.

Tokens must be Ed25519. `alg: none`, HMAC, and headers carrying key material
(`jku`, `x5u`, `jwk`) are not rejected by policy -- the header is a constant,
Expand All @@ -108,7 +110,7 @@ so they cannot be expressed.
### Where the internal issuer comes from

The gateway does not hold the app root key; the CVM mints its token and hands
it over on registration, bound to the app id and short-lived:
it over on registration, bound to the app id:

1. The agent and `dstack-util`, both inside the CVM, derive the same Ed25519
key: `derive_app_key(app_root_key, "dstack:guest-api:issuer:v1", Ed25519)`.
Expand All @@ -117,8 +119,8 @@ it over on registration, bound to the app id and short-lived:
never versioned again.
2. `dstack-util` signs a token with `iss = "dstack:guest-api:<app_id>"`,
`aud = <app_id>`, only the `guest:health` scope, `iat` rounded down to the
half hour and `exp` one hour later. Every registration within a half hour
carries the same token, and it never has less than 30 minutes left.
year and `exp` ten years later. Every registration within a year carries
the same token, and it never lapses while registration is down.
3. Registration carries it to the gateway, which presents it on the v1 `Health`
poll.

Expand Down
25 changes: 2 additions & 23 deletions dstack/crates/api-auth/src/jwt.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,6 @@ use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine as _};
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use serde::{Deserialize, Serialize};

/// Longest lifetime a token may claim between `iat` and `exp`.
///
/// There is no revocation list: a token is trusted until it expires, so the
/// lifetime is the whole of the exposure control. The CVM re-registers with
/// the gateway every three minutes and mints a fresh token every half hour, so
/// this costs it nothing.
pub const MAX_TOKEN_LIFETIME_SECS: u64 = 3600;

/// Tolerance for clock skew in either direction.
pub const CLOCK_LEEWAY_SECS: u64 = 30;

Expand Down Expand Up @@ -201,7 +193,7 @@ pub fn sign(signing_key: &[u8; 32], claims: &Claims) -> Result<String> {
/// The public keys one issuer may sign with.
///
/// A list so a key can be rotated without a flag day: publish the new key
/// alongside the old, wait out [`MAX_TOKEN_LIFETIME_SECS`], drop the old one.
/// alongside the old, wait out the tokens signed by the old one, drop it.
#[derive(Clone, Debug)]
pub struct IssuerKeys {
/// The exact `iss` this issuer's tokens must carry.
Expand Down Expand Up @@ -263,9 +255,6 @@ impl Verifier {
if claims.exp < now.saturating_sub(CLOCK_LEEWAY_SECS) {
bail!("token has expired");
}
if claims.exp.saturating_sub(claims.iat) > MAX_TOKEN_LIFETIME_SECS {
bail!("token lifetime exceeds {MAX_TOKEN_LIFETIME_SECS}s");
}
if claims.iat > now + CLOCK_LEEWAY_SECS {
bail!("token was issued in the future");
}
Expand Down Expand Up @@ -363,7 +352,7 @@ mod tests {
}

#[test]
fn rejects_expired_overlong_and_future_tokens() {
fn rejects_expired_and_future_tokens() {
let expired = mint(
1,
&internal_issuer(APP_ID),
Expand All @@ -374,16 +363,6 @@ mod tests {
);
assert!(verifier(1).verify(&expired, APP_ID, 1200).is_err());

let overlong = mint(
1,
&internal_issuer(APP_ID),
APP_ID,
1000,
1000 + 7200,
"guest:info",
);
assert!(verifier(1).verify(&overlong, APP_ID, 1100).is_err());

let future = mint(
1,
&internal_issuer(APP_ID),
Expand Down
11 changes: 6 additions & 5 deletions dstack/dstack-util/src/system_setup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -539,10 +539,11 @@ const GATEWAY_AGENT_SCOPES: &[&str] = &[jwt::scope::HEALTH];

/// How often the gateway's token changes. `iat` is rounded down to this, and
/// Ed25519 signatures are deterministic, so every registration within one
/// period carries the same token: the stored record stays unchanged instead of
/// changing every three minutes. With `exp` an hour after `iat`, a token always
/// has at least half an hour left, which outlasts many failed refreshes.
const GATEWAY_AGENT_TOKEN_PERIOD_SECS: u64 = 1800;
/// period carries the same token. With `exp` ten years after `iat`, a token
/// never lapses while registration is down.
const GATEWAY_AGENT_TOKEN_PERIOD_SECS: u64 = 365 * 24 * 3600;

const GATEWAY_AGENT_TOKEN_LIFETIME_SECS: u64 = 10 * GATEWAY_AGENT_TOKEN_PERIOD_SECS;

/// Mint the token the gateway presents to this CVM's guest agent, signed with
/// the key the agent derives the same way from the app root key.
Expand All @@ -561,7 +562,7 @@ fn gateway_agent_token(app_id: &[u8], app_root_key: &[u8]) -> Result<String> {
iss: jwt::internal_issuer(&app_id),
aud: app_id,
iat,
exp: iat + jwt::MAX_TOKEN_LIFETIME_SECS,
exp: iat + GATEWAY_AGENT_TOKEN_LIFETIME_SECS,
scope: GATEWAY_AGENT_SCOPES.join(" "),
};
jwt::sign(&secret, &claims)
Expand Down
Loading