diff --git a/CHANGELOG.md b/CHANGELOG.md index 338ca6a7b7..d6e033e856 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Security +- Prebid Server receives only allowlisted consent cookies in `both` and `cookies_only`, and no Cookie header in `openrtb_only` (#763). Cookie selection matches consent extraction, including last-duplicate selection and GPP section whitespace normalization. If an extracted consent cookie cannot be forwarded safely, `cookies_only` retains only the affected signal in the OpenRTB body; GPP and its section IDs fall back together. Publisher session and identity cookies are excluded in every mode. **Breaking** for deployments that relied on other cookies reaching PBS: Prebid Server's auction endpoint reads its `uids` user-sync cookie and configured host cookie from the request, so when that cookie domain covers the publisher site, those buyer UIDs no longer arrive through Trusted Server. - Didomi SDK requests now derive `X-Geo-Country`, `X-Geo-Region`, and `CloudFront-Viewer-Country` only from trusted platform geo (#1151). Caller-supplied `FastlyGeo-CountryCode` and `FastlyGeo-Region` headers previously let callers choose the geography sent to Didomi. **Breaking** for deployments that injected those headers: configure CDN-fronted client IP instead. Without a complete trusted country/region pair (currently always on Cloudflare, Axum, and Spin), the geo header group is omitted. Noncanonical loaders and SDK POSTs bypass the outbound cache and return `no-store, private`; other SDK GETs retain Didomi's cache policy. Purge cached SDK entries before rollout. Geo-less loaders risk Didomi throttling and disablement; production Fastly deployments should validate geo coverage and enable `geo_query_parameters = true`. Other adapters currently have no compliant loader-caching mode through this proxy; use direct SDK delivery instead. Audit-generated configurations leave detected Didomi disabled for manual review. - `ts prebid client` now keeps the config file's mode bits when it writes `external_bundle_sha256` and `external_bundle_sri`; previously a `0600` config was reset to the umask default (usually `0644`). Configs already rewritten by an earlier version keep the widened mode, so restrict them again (for example `chmod 600 trusted-server.toml`). The rewrite is fsynced and leaves no temporary copy behind on failure. TOML parse errors from `ts prebid client` and `ts audit generate` now name the file, line and column without quoting the offending config line (#1202). - Reserve the application-visible `/_ts/trace*` prefix locally on every adapter, including when mobile tracing is disabled. Existing authentication runs first; disabled or unknown trace routes then return a local `404` instead of forwarding to the publisher. diff --git a/crates/trusted-server-core/src/auction/openrtb.rs b/crates/trusted-server-core/src/auction/openrtb.rs index 74553c9bce..12c885d88e 100644 --- a/crates/trusted-server-core/src/auction/openrtb.rs +++ b/crates/trusted-server-core/src/auction/openrtb.rs @@ -17,7 +17,6 @@ use super::routing::{ PrebidTransportHeaders, ProviderAuctionInput, ProviderSlotInput, RoutedAuction, }; use super::types::{AdFormat, AdSlot, AuctionResponse, Bid}; -use crate::consent::ConsentSource; use crate::error::TrustedServerError; use crate::openrtb::{ Banner, ConsentedProvidersSettings, Device, Format, Geo, Imp, OpenRtbRequest, Publisher, Regs, @@ -347,13 +346,14 @@ fn build_common_request( ProfilePolicy::Standard(_) => common.publisher.page_url.clone(), }; let consent = common.user.consent.as_ref(); - let body_consent = match policy { - ProfilePolicy::Prebid(plan) => consent.filter(|value| { - plan.consent_forwarding.includes_body_consent() - || !matches!(value.source, ConsentSource::Cookie) - }), - _ => consent, + let selected_consent = match policy { + ProfilePolicy::Prebid(plan) => routed + .prebid_transport_headers() + .consent_cookies() + .body_consent(consent, plan.consent_forwarding), + _ => consent.map(std::borrow::Cow::Borrowed), }; + let body_consent = selected_consent.as_deref(); let raw_tc = body_consent.and_then(|value| value.raw_tc_string.clone()); let user = Some(User { id: common.user.id.clone(), diff --git a/crates/trusted-server-core/src/auction/orchestrator.rs b/crates/trusted-server-core/src/auction/orchestrator.rs index 95b8549863..0d00722149 100644 --- a/crates/trusted-server-core/src/auction/orchestrator.rs +++ b/crates/trusted-server-core/src/auction/orchestrator.rs @@ -6981,15 +6981,348 @@ mod tests { .iter() .find(|(name, _)| name == "cookie") .map(|(_, value)| value.as_str()); - assert_eq!(first_cookie, Some("consent=keep; other=value")); + assert_eq!(first_cookie, None, "should omit cookies in openrtb_only"); let second_cookie = headers[1] .iter() .find(|(name, _)| name == "cookie") .map(|(_, value)| value.as_str()); - assert_eq!( - second_cookie, - Some("consent=keep; euconsent-v2=drop; other=value") - ); + assert_eq!(second_cookie, Some("euconsent-v2=drop")); + } + + #[tokio::test] + async fn planned_prebid_cookie_modes_preserve_body_consent_policy() { + for (mode, forwards_body, forwards_cookies) in [ + ("both", true, true), + ("cookies_only", false, true), + ("openrtb_only", true, false), + ] { + for source in [ + crate::consent::ConsentSource::Cookie, + crate::consent::ConsentSource::PolicyDefault, + ] { + for (cookie, allowed_cookie) in [ + ( + Some(http::HeaderValue::from_static( + "euconsent-v2=fictional-tcf; __gpp=fictional-gpp; __gpp_sid=2,6; us_privacy=1YNN; ts-ec=secret; ts-eids=secret; sharedId=secret; session=secret", + )), + Some( + "euconsent-v2=fictional-tcf; __gpp=fictional-gpp; __gpp_sid=2,6; us_privacy=1YNN", + ), + ), + ( + Some( + http::HeaderValue::from_bytes( + b"euconsent-v2=fictional-tcf; session=secret;\xffbroken", + ) + .expect("should build malformed header"), + ), + None, + ), + ( + Some(http::HeaderValue::from_static( + "session=secret; ts-ec=secret", + )), + None, + ), + (None, None), + ] { + let unreadable_cookie = + cookie.as_ref().is_some_and(|value| value.to_str().is_err()); + let http = Arc::new(StubHttpClient::new()); + http.push_response(204, Vec::new()); + let services = build_services_with_http_client(Arc::clone(&http) as Arc<_>); + let plan = AuctionPlan::compile(planned_prebid_config(&[( + "pbs", + serde_json::json!({"consent_forwarding":mode}), + NotificationConfig::default(), + )])) + .expect("should compile PBS plan"); + let orchestrator = AuctionOrchestratorHarness::new(plan, None); + let mut request = planned_prebid_request(); + request.user.consent = Some(crate::consent::ConsentContext { + source, + gdpr_applies: true, + raw_tc_string: Some("fictional-tcf".to_string()), + raw_gpp_string: Some("fictional-gpp".to_string()), + gpp_section_ids: Some(vec![2, 6]), + raw_us_privacy: Some("1YNN".to_string()), + raw_ac_string: Some("fictional-ac".to_string()), + ..Default::default() + }); + let settings = create_test_settings(); + let mut inbound = http::Request::new(edgezero_core::body::Body::empty()); + if let Some(cookie) = cookie { + inbound.headers_mut().append(http::header::COOKIE, cookie); + // A second Cookie field cannot bypass the selected-header policy. + inbound.headers_mut().append( + http::header::COOKIE, + http::HeaderValue::from_static("session=second-secret"), + ); + } + let context = AuctionContext { + settings: &settings, + request: &inbound, + timeout_ms: 777, + transport_timeout_ms: 777, + provider_responses: None, + services: &services, + }; + + orchestrator + .run_auction(&request, &context) + .await + .expect("should run PBS auction"); + + let headers = http.recorded_request_header_bytes(); + assert_eq!(headers.len(), 1, "should dispatch one PBS request"); + let cookies = headers[0] + .iter() + .filter(|(name, _)| name.eq_ignore_ascii_case("cookie")) + .map(|(_, value)| value.as_slice()) + .collect::>(); + let expected = if forwards_cookies { + allowed_cookie + } else { + None + }; + assert_eq!( + cookies, + expected + .map(|value| vec![value.as_bytes()]) + .unwrap_or_default(), + "should apply {mode} cookie policy" + ); + let bodies = http.recorded_request_bodies(); + let body: serde_json::Value = + serde_json::from_slice(&bodies[0]).expect("should parse PBS body"); + let includes_body = + forwards_body || source == crate::consent::ConsentSource::PolicyDefault; + let includes_fallback = !includes_body && unreadable_cookie; + for (pointer, value) in [ + ("/user/consent", serde_json::json!("fictional-tcf")), + ("/user/ext/consent", serde_json::json!("fictional-tcf")), + ( + "/user/ext/ConsentedProvidersSettings/consented_providers", + serde_json::json!("fictional-ac"), + ), + ("/regs/gdpr", serde_json::json!(1)), + ("/regs/gpp", serde_json::json!("fictional-gpp")), + ("/regs/gpp_sid", serde_json::json!([2, 6])), + ("/regs/us_privacy", serde_json::json!("1YNN")), + ("/regs/ext/gdpr", serde_json::json!(1)), + ("/regs/ext/gpp", serde_json::json!("fictional-gpp")), + ("/regs/ext/gpp_sid", serde_json::json!([2, 6])), + ("/regs/ext/us_privacy", serde_json::json!("1YNN")), + ] { + assert_eq!( + body.pointer(pointer), + (includes_body + || (includes_fallback + && pointer != "/user/ext/ConsentedProvidersSettings/consented_providers")) + .then_some(&value), + "should preserve {mode} body policy at {pointer} for {source:?}" + ); + } + } + } + } + } + + #[tokio::test] + async fn planned_prebid_cookie_consent_survives_lenient_extraction() { + for mode in ["both", "cookies_only", "openrtb_only"] { + for (cookie, forwarded, body_fallback) in [ + ( + "__gpp_sid=2, 6; us_privacy = 1YYN; session=secret", + "__gpp_sid=2,6; us_privacy=1YYN", + false, + ), + ( + "us_privacy=1YNN; us_privacy=1YYN; session=secret", + "us_privacy=1YYN", + false, + ), + ( + "us_privacy=1YYN; __gpp=bad\\value; session=secret", + "us_privacy=1YYN", + true, + ), + ] { + let http = Arc::new(StubHttpClient::new()); + http.push_response(204, Vec::new()); + let services = build_services_with_http_client(Arc::clone(&http) as Arc<_>); + let plan = AuctionPlan::compile(planned_prebid_config(&[( + "pbs", + serde_json::json!({"consent_forwarding":mode}), + NotificationConfig::default(), + )])) + .expect("should compile PBS plan"); + let orchestrator = AuctionOrchestratorHarness::new(plan, None); + let inbound = http::Request::builder() + .header(http::header::COOKIE, cookie) + .body(edgezero_core::body::Body::empty()) + .expect("should build browser request"); + let jar = crate::cookies::handle_request_cookies(&inbound) + .expect("should parse browser cookies"); + let signals = crate::consent::extract_consent_signals(jar.as_ref(), &inbound); + let consent = crate::consent::build_context_from_signals(&signals); + assert!( + consent.us_privacy.as_ref().is_some_and(|value| { + value.opt_out_sale == crate::consent::PrivacyFlag::Yes + }), + "should decode the browser opt-out before dispatch" + ); + let mut request = planned_prebid_request(); + request.user.consent = Some(consent); + let settings = create_test_settings(); + let context = AuctionContext { + settings: &settings, + request: &inbound, + timeout_ms: 777, + transport_timeout_ms: 777, + provider_responses: None, + services: &services, + }; + orchestrator + .run_auction(&request, &context) + .await + .expect("should dispatch PBS auction"); + let headers = http.recorded_request_headers(); + assert_eq!( + headers[0] + .iter() + .find(|(name, _)| name == "cookie") + .map(|(_, value)| value.as_str()), + (mode != "openrtb_only").then_some(forwarded), + "should preserve only the selected consent values for {cookie:?} in {mode}" + ); + let body: serde_json::Value = + serde_json::from_slice(&http.recorded_request_bodies()[0]) + .expect("should parse PBS body"); + let includes_body = mode != "cookies_only"; + assert_eq!( + body.pointer("/regs/us_privacy"), + includes_body.then_some(&serde_json::json!("1YYN")), + "should preserve opt-out through at least one channel" + ); + if cookie.starts_with("__gpp_sid") { + assert_eq!( + body.pointer("/regs/gpp_sid"), + includes_body.then_some(&serde_json::json!([2, 6])), + "should preserve decoded GPP sections" + ); + } + if body_fallback { + assert_eq!( + body.pointer("/regs/gpp"), + Some(&serde_json::json!("bad\\value")), + "should carry unsafe cookie framing only as a JSON string" + ); + } + } + } + } + + #[tokio::test] + async fn planned_prebid_cookies_only_falls_back_only_for_unsafe_signal() { + for (cookie, forwarded, expected_tc, expected_gpp, expected_sid, expected_us) in [ + ( + r"euconsent-v2=bad\value; __gpp=GPP; __gpp_sid=2,6; us_privacy=1YYN", + "__gpp=GPP; __gpp_sid=2,6; us_privacy=1YYN", + Some(r"bad\value"), + None, + None, + None, + ), + ( + "euconsent-v2=TC; __gpp=GPP; __gpp_sid=2, 6,; us_privacy=1YYN", + "euconsent-v2=TC; __gpp=GPP; us_privacy=1YYN", + None, + Some("GPP"), + Some(serde_json::json!([2, 6])), + None, + ), + ( + r"euconsent-v2=TC; __gpp=GPP; __gpp_sid=2,6; us_privacy=bad\value", + "euconsent-v2=TC; __gpp=GPP; __gpp_sid=2,6", + None, + None, + None, + Some(r"bad\value"), + ), + ] { + let http = Arc::new(StubHttpClient::new()); + http.push_response(204, Vec::new()); + let services = build_services_with_http_client(Arc::clone(&http) as Arc<_>); + let plan = AuctionPlan::compile(planned_prebid_config(&[( + "pbs", + serde_json::json!({"consent_forwarding":"cookies_only"}), + NotificationConfig::default(), + )])) + .expect("should compile PBS plan"); + let orchestrator = AuctionOrchestratorHarness::new(plan, None); + let inbound = http::Request::builder() + .header(http::header::COOKIE, cookie) + .body(edgezero_core::body::Body::empty()) + .expect("should build browser request"); + let jar = crate::cookies::handle_request_cookies(&inbound) + .expect("should parse browser cookies"); + let signals = crate::consent::extract_consent_signals(jar.as_ref(), &inbound); + let mut request = planned_prebid_request(); + request.user.consent = Some(crate::consent::build_context_from_signals(&signals)); + let settings = create_test_settings(); + let context = AuctionContext { + settings: &settings, + request: &inbound, + timeout_ms: 777, + transport_timeout_ms: 777, + provider_responses: None, + services: &services, + }; + + orchestrator + .run_auction(&request, &context) + .await + .expect("should dispatch PBS auction"); + + let headers = http.recorded_request_headers(); + assert_eq!(headers.len(), 1, "should dispatch exactly one PBS request"); + assert_eq!( + headers[0] + .iter() + .find(|(name, _)| name == "cookie") + .map(|(_, value)| value.as_str()), + Some(forwarded), + "should keep unaffected safe consent cookies" + ); + let bodies = http.recorded_request_bodies(); + let body: serde_json::Value = + serde_json::from_slice(&bodies[0]).expect("should parse PBS body"); + for path in ["/user/consent", "/user/ext/consent"] { + assert_eq!( + body.pointer(path), + expected_tc.map(serde_json::Value::from).as_ref(), + "should fall back only for affected TC" + ); + } + for prefix in ["/regs", "/regs/ext"] { + assert_eq!( + body.pointer(&format!("{prefix}/gpp")), + expected_gpp.map(serde_json::Value::from).as_ref(), + "should fall back for the coupled GPP string" + ); + assert_eq!( + body.pointer(&format!("{prefix}/gpp_sid")), + expected_sid.as_ref(), + "should fall back for coupled GPP sections" + ); + assert_eq!( + body.pointer(&format!("{prefix}/us_privacy")), + expected_us.map(serde_json::Value::from).as_ref(), + "should fall back only for affected US Privacy" + ); + } + } } #[tokio::test] diff --git a/crates/trusted-server-core/src/auction/routing.rs b/crates/trusted-server-core/src/auction/routing.rs index 2b8135da3c..4134ae6a7f 100644 --- a/crates/trusted-server-core/src/auction/routing.rs +++ b/crates/trusted-server-core/src/auction/routing.rs @@ -9,6 +9,7 @@ use serde_json::Value; use super::plan::{AuctionPlan, BidderId, ProviderId, RoutingMode}; use super::types::{AdSlot, AuctionRequest, MediaType}; +use crate::integrations::prebid::PrebidConsentCookies; const TRUSTED_SERVER_ENVELOPE: &str = "trustedServer"; const BIDDER_PARAMS_FIELD: &str = "bidderParams"; @@ -199,19 +200,21 @@ impl ProviderSlotInput { /// Request headers approved for later Prebid transport forwarding. /// -/// Values remain as raw [`HeaderValue`] instances so non-ASCII bytes retain -/// the same legacy handling. Client-supplied `X-Forwarded-For` is never read. +/// The inbound `Cookie` header is reduced to [`PrebidConsentCookies`] when the +/// snapshot is taken, so publisher session and identity cookies are never +/// retained for the auction. Other captured headers retain their bytes as +/// [`HeaderValue`] instances. Client-supplied `X-Forwarded-For` is never read. #[derive(Debug, Clone, Default)] pub(crate) struct PrebidTransportHeaders { - cookie: Option, + consent_cookies: PrebidConsentCookies, user_agent: Option, referer: Option, accept_language: Option, } impl PrebidTransportHeaders { - pub(crate) fn cookie(&self) -> Option<&HeaderValue> { - self.cookie.as_ref() + pub(crate) fn consent_cookies(&self) -> &PrebidConsentCookies { + &self.consent_cookies } pub(crate) fn user_agent(&self) -> Option<&HeaderValue> { @@ -228,7 +231,9 @@ impl PrebidTransportHeaders { fn snapshot(request: &Request) -> Self { Self { - cookie: request.headers().get(header::COOKIE).cloned(), + consent_cookies: PrebidConsentCookies::from_header( + request.headers().get(header::COOKIE), + ), user_agent: request.headers().get(header::USER_AGENT).cloned(), referer: request.headers().get(header::REFERER).cloned(), accept_language: request.headers().get(header::ACCEPT_LANGUAGE).cloned(), @@ -1303,12 +1308,15 @@ mod tests { #[test] fn snapshots_first_headers_retains_raw_bytes_and_ignores_inbound_xff() { let mut inbound = inbound(); + inbound.headers_mut().append( + header::COOKIE, + HeaderValue::from_static( + "session_id=example-session; ts-ec=example-ec; euconsent-v2=TC", + ), + ); inbound .headers_mut() - .append(header::COOKIE, HeaderValue::from_static("first=1")); - inbound - .headers_mut() - .append(header::COOKIE, HeaderValue::from_static("second=2")); + .append(header::COOKIE, HeaderValue::from_static("us_privacy=1YNN")); inbound.headers_mut().append( header::USER_AGENT, HeaderValue::from_bytes(b"agent-\x80").expect("should accept raw header"), @@ -1336,7 +1344,11 @@ mod tests { Some(attested), ); let headers = routed.prebid_transport_headers(); - assert_eq!(headers.cookie(), Some(&HeaderValue::from_static("first=1"))); + assert_eq!( + headers.consent_cookies().header(), + Some(&HeaderValue::from_static("euconsent-v2=TC")), + "should retain only consent cookies from the first Cookie header" + ); assert_eq!( headers.user_agent().expect("should retain UA").as_bytes(), b"agent-\x80" diff --git a/crates/trusted-server-core/src/consent_config.rs b/crates/trusted-server-core/src/consent_config.rs index fbef7efe82..b9165d562d 100644 --- a/crates/trusted-server-core/src/consent_config.rs +++ b/crates/trusted-server-core/src/consent_config.rs @@ -125,14 +125,16 @@ pub enum ConsentMode { /// How consent signals are forwarded to a specific partner integration. /// -/// Controls whether consent travels through the `OpenRTB` body, raw `Cookie` -/// headers, or both. The default (`Both`) preserves backward compatibility. +/// Controls whether consent travels through the `OpenRTB` body, consent +/// cookies, or both. PBS permits only allowlisted consent cookies and omits +/// the entire `Cookie` header in [`Self::OpenrtbOnly`]. #[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)] #[serde(rename_all = "snake_case")] pub enum ConsentForwardingMode { - /// Forward consent in the `OpenRTB` body only; strip consent cookies. + /// Forward consent in the `OpenRTB` body only; PBS sends no `Cookie` header. OpenrtbOnly, - /// Forward consent cookies only; omit consent fields from the body. + /// Prefer consent cookies; omit cookie-sourced body consent when cookies + /// can carry it safely. Policy-sourced signals retain a body fallback. CookiesOnly, /// Forward consent in both cookies and body (default). #[default] @@ -140,15 +142,6 @@ pub enum ConsentForwardingMode { } impl ConsentForwardingMode { - /// Whether consent cookies should be stripped from forwarded requests. - /// - /// Returns `true` for [`OpenrtbOnly`](Self::OpenrtbOnly) since consent - /// travels exclusively through the request body in that mode. - #[must_use] - pub const fn strips_consent_cookies(self) -> bool { - matches!(self, Self::OpenrtbOnly) - } - /// Whether consent fields should be included in the request body. /// /// Returns `true` for [`OpenrtbOnly`](Self::OpenrtbOnly) and @@ -421,24 +414,6 @@ mod tests { assert_eq!(config.mode, ConsentMode::Proxy, "should parse proxy mode"); } - #[test] - fn consent_forwarding_mode_strips_cookies_only_for_openrtb() { - use super::ConsentForwardingMode; - - assert!( - ConsentForwardingMode::OpenrtbOnly.strips_consent_cookies(), - "openrtb_only should strip consent cookies" - ); - assert!( - !ConsentForwardingMode::CookiesOnly.strips_consent_cookies(), - "cookies_only should not strip consent cookies" - ); - assert!( - !ConsentForwardingMode::Both.strips_consent_cookies(), - "both should not strip consent cookies" - ); - } - #[test] fn consent_forwarding_mode_includes_body_consent_except_cookies_only() { use super::ConsentForwardingMode; diff --git a/crates/trusted-server-core/src/cookies.rs b/crates/trusted-server-core/src/cookies.rs index 3fe6719ef7..6cd7e913bb 100644 --- a/crates/trusted-server-core/src/cookies.rs +++ b/crates/trusted-server-core/src/cookies.rs @@ -1,6 +1,6 @@ //! Cookie handling utilities. //! -//! This module provides functionality for parsing, stripping, and forwarding cookies used in the +//! This module provides functionality for parsing and extracting cookies used in the //! trusted server system. pub(crate) mod template_cache_policy; @@ -16,9 +16,8 @@ use crate::error::TrustedServerError; /// Cookie names carrying privacy consent signals. /// -/// Used by [`strip_cookies`] to remove consent signals from a `Cookie` header -/// before forwarding requests to partners that receive consent through the -/// `OpenRTB` body instead. +/// Used as the PBS consent-cookie allowlist. PBS omits the entire `Cookie` +/// header when consent is sent exclusively through the `OpenRTB` body. pub const CONSENT_COOKIE_NAMES: &[&str] = &[ COOKIE_EUCONSENT_V2, COOKIE_GPP, @@ -81,67 +80,6 @@ pub fn extract_cookie_value(req: &Request, name: &str) -> Option { }) } -/// Strips named cookies from a `Cookie` header value string. -/// -/// Parses the semicolon-separated cookie pairs, filters out any whose name -/// matches one of `cookie_names`, and reconstructs the header string. -/// -/// Returns an empty string if all cookies were stripped or the input was empty. -#[must_use] -pub fn strip_cookies(cookie_header: &str, cookie_names: &[&str]) -> String { - cookie_header - .split(';') - .map(str::trim) - .filter(|pair| { - if let Some(name) = pair.split('=').next() { - !cookie_names.contains(&name.trim()) - } else { - true - } - }) - .filter(|s| !s.is_empty()) - .collect::>() - .join("; ") -} - -/// Copies the `Cookie` header from one request to another, optionally -/// stripping consent cookies. -/// -/// When `strip_consent` is `true`, cookies listed in [`CONSENT_COOKIE_NAMES`] -/// are removed before forwarding. If stripping leaves no cookies or yields an -/// invalid header value, the stripped header is omitted. Non-UTF-8 cookie -/// headers are forwarded unchanged. Existing `Cookie` headers on `to` are -/// preserved; source values are appended after them. -pub fn forward_cookie_header( - from: &Request, - to: &mut Request, - strip_consent: bool, -) { - for cookie_value in from.headers().get_all(header::COOKIE) { - if !strip_consent { - to.headers_mut() - .append(header::COOKIE, cookie_value.clone()); - continue; - } - - match cookie_value.to_str() { - Ok(s) => { - let stripped = strip_cookies(s, CONSENT_COOKIE_NAMES); - if !stripped.is_empty() - && let Ok(value) = http::HeaderValue::from_str(&stripped) - { - to.headers_mut().append(header::COOKIE, value); - } - } - Err(_) => { - // Non-UTF-8 Cookie header — forward as-is. - to.headers_mut() - .append(header::COOKIE, cookie_value.clone()); - } - } - } -} - #[cfg(test)] mod tests { use http::HeaderValue; @@ -287,168 +225,4 @@ mod tests { assert_eq!(extract_cookie_value(&req, "session"), None); } - - // --------------------------------------------------------------- - // forward_cookie_header tests - // --------------------------------------------------------------- - - #[test] - fn test_forward_cookie_header_strips_consent() { - let from = build_request(Some("euconsent-v2=BOE; session=abc123; us_privacy=1YNN")); - let mut to = build_request(None); - - forward_cookie_header(&from, &mut to, true); - - let forwarded = to - .headers() - .get(header::COOKIE) - .and_then(|v| v.to_str().ok()) - .unwrap_or(""); - assert!( - !forwarded.contains("euconsent-v2"), - "should strip consent cookie" - ); - assert!( - forwarded.contains("session=abc123"), - "should keep non-consent cookie" - ); - } - - #[test] - fn test_forward_cookie_header_strip_all_leaves_header_absent() { - let from = build_request(Some("euconsent-v2=BOE; __gpp=DBAC")); - let mut to = build_request(None); - - forward_cookie_header(&from, &mut to, true); - - assert!( - to.headers().get(header::COOKIE).is_none(), - "should omit Cookie header when all cookies are stripped" - ); - } - - #[test] - fn test_forward_cookie_header_no_strip_passes_all() { - let from = build_request(Some("euconsent-v2=BOE; session=abc123")); - let mut to = build_request(None); - - forward_cookie_header(&from, &mut to, false); - - let forwarded = to - .headers() - .get(header::COOKIE) - .and_then(|v| v.to_str().ok()) - .unwrap_or(""); - assert!( - forwarded.contains("euconsent-v2"), - "should forward consent cookie when not stripping" - ); - assert!( - forwarded.contains("session=abc123"), - "should forward non-consent cookie" - ); - } - - #[test] - fn test_forward_cookie_header_non_utf8_forwarded_unchanged() { - let non_utf8 = http::HeaderValue::from_bytes(b"\xff\xfe=value") - .expect("should build non-UTF-8 header value"); - let mut from = build_request(None); - from.headers_mut().append(header::COOKIE, non_utf8); - let mut to = build_request(None); - - forward_cookie_header(&from, &mut to, true); - - let forwarded = to.headers().get(header::COOKIE); - assert!( - forwarded.is_some(), - "should forward non-UTF-8 Cookie header unchanged" - ); - assert_eq!( - forwarded.expect("should have cookie header").as_bytes(), - b"\xff\xfe=value", - "should preserve raw bytes for non-UTF-8 cookie" - ); - } - - #[test] - fn test_forward_cookie_header_multiple_cookie_headers_appended() { - let mut from = build_request(Some("session=abc123")); - from.headers_mut().append( - header::COOKIE, - "theme=dark".parse().expect("should parse header value"), - ); - let mut to = build_request(None); - - forward_cookie_header(&from, &mut to, false); - - let all_cookies: Vec<_> = to - .headers() - .get_all(header::COOKIE) - .iter() - .filter_map(|v| v.to_str().ok()) - .collect(); - assert_eq!(all_cookies.len(), 2, "should append all Cookie headers"); - assert!(all_cookies.iter().any(|v| v.contains("session=abc123"))); - assert!(all_cookies.iter().any(|v| v.contains("theme=dark"))); - } - - #[test] - fn test_forward_cookie_header_appends_to_existing_target_cookies() { - let from = build_request(Some("session=abc123")); - let mut to = build_request(Some("template=existing")); - - forward_cookie_header(&from, &mut to, false); - - let all_cookies: Vec<_> = to - .headers() - .get_all(header::COOKIE) - .iter() - .filter_map(|v| v.to_str().ok()) - .collect(); - assert_eq!( - all_cookies, - vec!["template=existing", "session=abc123"], - "should preserve existing target cookies and append source cookies after them" - ); - } - - // --------------------------------------------------------------- - // strip_cookies tests - // --------------------------------------------------------------- - - #[test] - fn test_strip_cookies_removes_consent() { - let header = "euconsent-v2=BOE; __gpp=DBAC; session=abc123; us_privacy=1YNN"; - let stripped = strip_cookies(header, CONSENT_COOKIE_NAMES); - assert_eq!(stripped, "session=abc123"); - } - - #[test] - fn test_strip_cookies_preserves_non_consent() { - let header = "session=abc123; theme=dark"; - let stripped = strip_cookies(header, CONSENT_COOKIE_NAMES); - assert_eq!(stripped, "session=abc123; theme=dark"); - } - - #[test] - fn test_strip_cookies_empty_input() { - let stripped = strip_cookies("", CONSENT_COOKIE_NAMES); - assert_eq!(stripped, ""); - } - - #[test] - fn test_strip_cookies_all_stripped() { - let header = "euconsent-v2=BOE; __gpp=DBAC; __gpp_sid=2,6; us_privacy=1YNN"; - let stripped = strip_cookies(header, CONSENT_COOKIE_NAMES); - assert_eq!(stripped, ""); - } - - #[test] - fn test_strip_cookies_with_complex_values() { - // Cookie values can contain '=' characters. - let header = "euconsent-v2=BOE=xyz; session=abc=123=def"; - let stripped = strip_cookies(header, CONSENT_COOKIE_NAMES); - assert_eq!(stripped, "session=abc=123=def"); - } } diff --git a/crates/trusted-server-core/src/integrations/prebid.rs b/crates/trusted-server-core/src/integrations/prebid.rs index 82c5d808f3..cc39c29cd6 100644 --- a/crates/trusted-server-core/src/integrations/prebid.rs +++ b/crates/trusted-server-core/src/integrations/prebid.rs @@ -1,3 +1,4 @@ +use std::borrow::Cow; use std::collections::HashMap; use std::collections::HashSet; use std::sync::{Arc, LazyLock}; @@ -36,7 +37,8 @@ use crate::auction::types::{AuctionContext, AuctionRequest, MediaType}; use crate::auction::types::{AuctionResponse, Bid as AuctionBid}; use crate::cache_policy::{CacheControlPolicy, EdgeCacheHeader}; use crate::consent_config::ConsentForwardingMode; -use crate::cookies::{CONSENT_COOKIE_NAMES, strip_cookies}; +use crate::constants::{COOKIE_EUCONSENT_V2, COOKIE_GPP, COOKIE_GPP_SID, COOKIE_US_PRIVACY}; +use crate::cookies::{CONSENT_COOKIE_NAMES, parse_cookies_to_jar}; use crate::error::TrustedServerError; #[cfg(test)] use crate::http_util::RequestInfo; @@ -462,9 +464,9 @@ pub struct LegacyPrebidServerConfig { pub bid_param_override_rules: Vec, /// How consent signals are forwarded to Prebid Server. /// - /// - `openrtb_only` — consent in `OpenRTB` body only, consent cookies stripped - /// - `cookies_only` — consent cookies forwarded, body consent fields omitted - /// - `both` — consent in both cookies and body (default) + /// - `openrtb_only` — consent in `OpenRTB` body only, no `Cookie` header + /// - `cookies_only` — allowlisted consent cookies, with body fallback for unsafe values + /// - `both` — allowlisted consent cookies and body consent (default) #[serde(default)] pub consent_forwarding: ConsentForwardingMode, /// Strip `nurl` and `burl` from PBS bids before they reach `window.tsjs.bids`. @@ -2016,9 +2018,8 @@ fn non_empty_override_object( /// synthesized from `client_ip` — the platform-attested client address — /// matching the trusted IP already sent in `OpenRTB` `device.ip`. /// -/// In [`ConsentForwardingMode::OpenrtbOnly`] mode, consent cookies are -/// stripped from the `Cookie` header since consent travels exclusively -/// through the `OpenRTB` body. +/// Only allowlisted consent cookies can reach PBS. In +/// [`ConsentForwardingMode::OpenrtbOnly`] mode, the `Cookie` header is omitted. #[cfg(test)] fn copy_request_headers( from: &http::Request, @@ -2027,7 +2028,7 @@ fn copy_request_headers( client_ip: Option, ) { apply_prebid_header_values( - from.headers().get(header::COOKIE), + &PrebidConsentCookies::from_header(from.headers().get(header::COOKIE)), from.headers().get(header::USER_AGENT), from.headers().get(header::REFERER), from.headers().get(header::ACCEPT_LANGUAGE), @@ -2037,7 +2038,7 @@ fn copy_request_headers( ); } -/// Apply the common raw-header transport policy for a planned PBS request. +/// Applies the common browser-header and consent-cookie policy for PBS. pub(crate) fn apply_prebid_transport_headers( from: &PrebidTransportHeaders, to: &mut http::Request, @@ -2045,7 +2046,7 @@ pub(crate) fn apply_prebid_transport_headers( client_ip: Option, ) { apply_prebid_header_values( - from.cookie(), + from.consent_cookies(), from.user_agent(), from.referer(), from.accept_language(), @@ -2057,10 +2058,10 @@ pub(crate) fn apply_prebid_transport_headers( #[allow( clippy::too_many_arguments, - reason = "the helper preserves four independently optional raw headers plus transport policy" + reason = "the helper preserves selected consent cookies, three independently optional raw headers, and transport policy" )] fn apply_prebid_header_values( - cookie: Option<&HeaderValue>, + consent_cookies: &PrebidConsentCookies, user_agent: Option<&HeaderValue>, referer: Option<&HeaderValue>, accept_language: Option<&HeaderValue>, @@ -2085,30 +2086,179 @@ fn apply_prebid_header_values( .insert(header::HeaderName::from_static("x-forwarded-for"), value); } - let Some(cookie_value) = cookie else { - return; - }; - if !consent_forwarding.strips_consent_cookies() { - to.headers_mut() - .insert(header::COOKIE, cookie_value.clone()); + // Rebuild the destination header so no stale or arbitrary cookies survive. + to.headers_mut().remove(header::COOKIE); + if consent_forwarding == ConsentForwardingMode::OpenrtbOnly { return; } - match cookie_value.to_str() { - Ok(value) => { - let stripped = strip_cookies(value, CONSENT_COOKIE_NAMES); - if !stripped.is_empty() - && let Ok(cookie_header) = HeaderValue::from_str(&stripped) - { - to.headers_mut().insert(header::COOKIE, cookie_header); + if let Some(value) = consent_cookies.header() { + to.headers_mut().insert(header::COOKIE, value.clone()); + } +} + +/// Consent cookies selected with the same parser used during consent extraction. +/// +/// Unsafe values remain available only through the JSON body fallback. No raw +/// cookie bytes are forwarded on a parsing or framing failure. +#[derive(Debug, Clone, Default)] +pub(crate) struct PrebidConsentCookies { + header: Option, + unsafe_tc: bool, + unsafe_gpp: bool, + unsafe_us_privacy: bool, +} + +impl PrebidConsentCookies { + pub(crate) fn from_header(header: Option<&HeaderValue>) -> Self { + let mut result = Self::default(); + let Some(header) = header else { + return result; + }; + let Ok(header) = header.to_str() else { + result.unsafe_tc = true; + result.unsafe_gpp = true; + result.unsafe_us_privacy = true; + return result; + }; + let jar = parse_cookies_to_jar(header); + let mut pairs = Vec::new(); + for name in CONSENT_COOKIE_NAMES { + let Some(cookie) = jar.get(name) else { + continue; + }; + if let Some(value) = prebid_cookie_value(name, cookie.value()) { + pairs.push(format!("{name}={value}")); + } else { + match *name { + COOKIE_EUCONSENT_V2 => result.unsafe_tc = true, + COOKIE_GPP | COOKIE_GPP_SID => result.unsafe_gpp = true, + COOKIE_US_PRIVACY => result.unsafe_us_privacy = true, + _ => { + log::warn!( + "Preserves body consent for unrecognized allowlisted cookie {name}" + ); + result.unsafe_tc = true; + result.unsafe_gpp = true; + result.unsafe_us_privacy = true; + } + } } } - Err(_) => { - to.headers_mut() - .insert(header::COOKIE, cookie_value.clone()); + if !pairs.is_empty() { + result.header = HeaderValue::from_str(&pairs.join("; ")).ok(); + if result.header.is_none() { + result.unsafe_tc = true; + result.unsafe_gpp = true; + result.unsafe_us_privacy = true; + } + } + result + } + + /// The allowlisted consent `Cookie` header, if any consent cookie is safe to forward. + pub(crate) fn header(&self) -> Option<&HeaderValue> { + self.header.as_ref() + } + + fn requires_body(&self) -> bool { + self.unsafe_tc || self.unsafe_gpp || self.unsafe_us_privacy + } + + /// Selects only the consent signals whose cookies need a JSON fallback. + /// + /// GPP and its section IDs travel together. This subset is used only for + /// serialization; the original context remains intact for consent gating. + pub(crate) fn body_consent<'a>( + &self, + context: Option<&'a crate::consent::ConsentContext>, + mode: ConsentForwardingMode, + ) -> Option> { + let context = context?; + if mode.includes_body_consent() + || !matches!(context.source, crate::consent::ConsentSource::Cookie) + { + return Some(Cow::Borrowed(context)); } + if !self.requires_body() { + return None; + } + let carries_gdpr = self.unsafe_tc || self.unsafe_gpp; + Some(Cow::Owned(crate::consent::ConsentContext { + raw_tc_string: self + .unsafe_tc + .then(|| context.raw_tc_string.clone()) + .flatten(), + raw_gpp_string: self + .unsafe_gpp + .then(|| context.raw_gpp_string.clone()) + .flatten(), + gpp_section_ids: self + .unsafe_gpp + .then(|| context.gpp_section_ids.clone()) + .flatten(), + raw_us_privacy: self + .unsafe_us_privacy + .then(|| context.raw_us_privacy.clone()) + .flatten(), + gdpr_applies: carries_gdpr && context.gdpr_applies, + jurisdiction: if carries_gdpr { + context.jurisdiction.clone() + } else { + crate::consent::jurisdiction::Jurisdiction::Unknown + }, + source: crate::consent::ConsentSource::Cookie, + ..Default::default() + })) } } +/// Preserves safe values and normalizes decimal GPP section whitespace. +fn prebid_cookie_value<'a>(name: &str, value: &'a str) -> Option> { + if valid_prebid_cookie_value(name, value) { + return Some(Cow::Borrowed(value)); + } + if name != COOKIE_GPP_SID { + return None; + } + let quoted = value.len() >= 2 && value.starts_with('"') && value.ends_with('"'); + let unquoted = if quoted { + &value[1..value.len() - 1] + } else { + value + }; + let sections = unquoted.split(',').map(str::trim).collect::>(); + if sections + .iter() + .any(|section| section.is_empty() || !section.bytes().all(|byte| byte.is_ascii_digit())) + { + return None; + } + let normalized = sections.join(","); + Some(Cow::Owned(if quoted { + format!("\"{normalized}\"") + } else { + normalized + })) +} + +/// Validates cookie framing without interpreting or modifying consent signals. +fn valid_prebid_cookie_value(name: &str, value: &str) -> bool { + let value = value + .strip_prefix('"') + .and_then(|value| value.strip_suffix('"')) + .unwrap_or(value); + // GPP section IDs use comma-separated decimals, already accepted by consent extraction. + if name == COOKIE_GPP_SID && value.contains(',') { + return value.split(',').all(|section| { + !section.is_empty() && section.bytes().all(|byte| byte.is_ascii_digit()) + }); + } + // RFC 6265 cookie-octet excludes whitespace, quotes, commas, semicolons and backslashes. + value + .bytes() + .all(|byte| matches!(byte, 0x21 | 0x23..=0x2b | 0x2d..=0x3a | 0x3c..=0x5b | 0x5d..=0x7e)) +} + /// Appends query parameters to a URL, handling both URLs with and without existing query strings. /// Returns the original URL unchanged if params are empty or already present. #[cfg(test)] @@ -2704,15 +2854,16 @@ impl PrebidAuctionProvider { } }); - // Build user object — populate consent at both OpenRTB 2.6 top-level - // and Prebid ext-based locations (dual placement). - // In cookies_only mode, cookie-sourced consent travels through the - // forwarded Cookie header. Policy-sourced consent has no inbound cookie - // to forward, so carry it in the OpenRTB body instead. - let consent_ctx = request.user.consent.as_ref().filter(|ctx| { - self.config.consent_forwarding.includes_body_consent() - || !matches!(ctx.source, crate::consent::ConsentSource::Cookie) - }); + // Select consent independently per signal: safely forwarded cookies + // remain omitted in cookies_only, while unsafe GPP and its IDs fall + // back together. Identity gating still uses the original context. + let body_consent = + PrebidConsentCookies::from_header(context.request.headers().get(header::COOKIE)) + .body_consent( + request.user.consent.as_ref(), + self.config.consent_forwarding, + ); + let consent_ctx = body_consent.as_deref(); let raw_tc = consent_ctx.and_then(|c| c.raw_tc_string.clone()); let user = Some(User { id: request.user.id.clone(), @@ -9733,7 +9884,7 @@ set = { networkId = 42 } } #[test] - fn planned_transport_preserves_malformed_cookie_bytes_and_attested_xff() { + fn planned_transport_omits_malformed_cookie_bytes_and_preserves_attested_xff() { let inbound = http::Request::builder() .uri("https://publisher.example/auction") .header( @@ -9777,8 +9928,8 @@ set = { networkId = 42 } .headers() .get(header::COOKIE) .map(HeaderValue::as_bytes), - Some(b"session=fictional;\xffbroken".as_slice()), - "should forward malformed cookie bytes unchanged" + None, + "should omit malformed cookie bytes" ); assert_eq!( outbound @@ -9790,6 +9941,201 @@ set = { networkId = 42 } ); } + #[test] + fn prebid_cookie_policy_rebuilds_only_valid_consent_pairs() { + for mode in [ + ConsentForwardingMode::Both, + ConsentForwardingMode::CookiesOnly, + ConsentForwardingMode::OpenrtbOnly, + ] { + for (input, allowed) in [ + (None, None), + (Some(""), None), + ( + Some("ts-ec=secret; ts-eids=secret; sharedId=secret; session=secret"), + None, + ), + ( + Some( + "EUCONSENT-V2=secret; x__gpp=secret; __gpp_sid_extra=secret; us_privacy_extra=secret", + ), + None, + ), + ( + Some( + "euconsent-v2; =secret; __gpp=bad value; __gpp_sid=2,session=secret; us_privacy=bad,session=secret; __gpp_sid=\"unterminated", + ), + None, + ), + ( + Some("euconsent-v2=bad\\value; __gpp=bad\"quote; us_privacy=bad\tvalue"), + None, + ), + ( + Some( + " session=secret ; euconsent-v2=TC==; __gpp=GPP; __gpp_sid=2,6; us_privacy=1YNN", + ), + Some("euconsent-v2=TC==; __gpp=GPP; __gpp_sid=2,6; us_privacy=1YNN"), + ), + ( + Some("euconsent-v2=\"TC==\"; __gpp=; __gpp_sid=2; us_privacy=1YNN"), + Some("euconsent-v2=\"TC==\"; __gpp=; __gpp_sid=2; us_privacy=1YNN"), + ), + ( + Some("__gpp=first; session=secret; __gpp=second"), + Some("__gpp=second"), + ), + ] { + let mut inbound = http::Request::new(EdgeBody::empty()); + if let Some(input) = input { + inbound.headers_mut().insert( + header::COOKIE, + HeaderValue::from_str(input).expect("should build cookie header"), + ); + } + let mut outbound = http::Request::new(EdgeBody::empty()); + outbound + .headers_mut() + .append(header::COOKIE, HeaderValue::from_static("stale=secret")); + outbound + .headers_mut() + .append(header::COOKIE, HeaderValue::from_static("another=secret")); + + copy_request_headers(&inbound, &mut outbound, mode, None); + + let expected = if mode == ConsentForwardingMode::OpenrtbOnly { + None + } else { + allowed + }; + assert_eq!( + outbound + .headers() + .get(header::COOKIE) + .and_then(|value| value.to_str().ok()), + expected, + "should rebuild only valid allowed pairs for {mode:?} with {input:?}" + ); + assert_eq!( + outbound.headers().get_all(header::COOKIE).iter().count(), + usize::from(expected.is_some()), + "should remove stale destination cookies" + ); + } + for bytes in [ + b"euconsent-v2=TC; session=secret;\xffbroken".as_slice(), + b"__gpp=GPP;\xc3\xa9=secret".as_slice(), + ] { + let mut inbound = http::Request::new(EdgeBody::empty()); + inbound.headers_mut().insert( + header::COOKIE, + HeaderValue::from_bytes(bytes).expect("should build non-ASCII header"), + ); + let mut outbound = http::Request::new(EdgeBody::empty()); + + copy_request_headers(&inbound, &mut outbound, mode, None); + + assert!( + !outbound.headers().contains_key(header::COOKIE), + "should omit unreadable headers in {mode:?}" + ); + } + } + } + + #[test] + fn prebid_cookie_selection_matches_extraction_and_preserves_safe_framing() { + for (input, expected, requires_body) in [ + ("us_privacy = 1YYN", Some("us_privacy=1YYN"), false), + ( + "euconsent-v2=\"TC==\"; __gpp_sid=\"2, 6\"", + Some("euconsent-v2=\"TC==\"; __gpp_sid=\"2,6\""), + false, + ), + ( + "__gpp_sid=2, 6; __gpp_sid=7, 8", + Some("__gpp_sid=7,8"), + false, + ), + ( + "__gpp=first; __gpp=bad\\value; us_privacy=1YYN", + Some("us_privacy=1YYN"), + true, + ), + ("__gpp_sid=\"", None, true), + ( + "__gpp_sid=2, session=secret; us_privacy=1YYN", + Some("us_privacy=1YYN"), + true, + ), + ( + "__gpp_sid=2,; us_privacy=1YYN", + Some("us_privacy=1YYN"), + true, + ), + ( + "__gpp=bad\"quote; us_privacy=1YYN", + Some("us_privacy=1YYN"), + true, + ), + ] { + let header = HeaderValue::from_str(input).expect("should build cookie header"); + let selected = PrebidConsentCookies::from_header(Some(&header)); + assert_eq!( + selected + .header + .as_ref() + .and_then(|value| value.to_str().ok()), + expected, + "should select safe last consent values from {input:?}" + ); + assert_eq!( + selected.requires_body(), + requires_body, + "should retain unrepresentable consent in the body for {input:?}" + ); + } + } + + #[test] + fn prebid_cookie_fallback_carries_gdpr_applicability_only_with_tc_or_gpp() { + let context = crate::consent::ConsentContext { + raw_tc_string: Some("fictional-tcf".to_string()), + raw_gpp_string: Some("fictional-gpp".to_string()), + gpp_section_ids: Some(vec![2]), + raw_us_privacy: Some("1YNN".to_string()), + gdpr_applies: true, + jurisdiction: crate::consent::jurisdiction::Jurisdiction::Gdpr, + source: crate::consent::ConsentSource::Cookie, + ..Default::default() + }; + for (input, carries_gdpr) in [ + ("euconsent-v2=bad\\value; __gpp=GPP; us_privacy=1YNN", true), + ("euconsent-v2=TC; __gpp=bad\\value; us_privacy=1YNN", true), + ("euconsent-v2=TC; __gpp_sid=2,; us_privacy=1YNN", true), + ("euconsent-v2=TC; __gpp=GPP; us_privacy=bad\\value", false), + ] { + let header = HeaderValue::from_str(input).expect("should build cookie header"); + let selected = PrebidConsentCookies::from_header(Some(&header)) + .body_consent(Some(&context), ConsentForwardingMode::CookiesOnly) + .expect("should fall back for the unsafe signal"); + + assert_eq!( + selected.gdpr_applies, carries_gdpr, + "should carry GDPR applicability only with TC or GPP for {input:?}" + ); + assert_eq!( + selected.jurisdiction, + if carries_gdpr { + crate::consent::jurisdiction::Jurisdiction::Gdpr + } else { + crate::consent::jurisdiction::Jurisdiction::Unknown + }, + "should carry jurisdiction only with TC or GPP for {input:?}" + ); + } + } + #[test] fn copy_request_headers_replaces_client_supplied_xff_with_attested_ip() { let from = http::Request::builder() @@ -9859,17 +10205,17 @@ set = { networkId = 42 } ( ConsentForwardingMode::Both, HeaderValue::from_static("session=fictional; euconsent-v2=fictional-tcf"), - Some(b"session=fictional; euconsent-v2=fictional-tcf".as_slice()), + Some(b"euconsent-v2=fictional-tcf".as_slice()), ), ( ConsentForwardingMode::CookiesOnly, HeaderValue::from_static("session=fictional; euconsent-v2=fictional-tcf"), - Some(b"session=fictional; euconsent-v2=fictional-tcf".as_slice()), + Some(b"euconsent-v2=fictional-tcf".as_slice()), ), ( ConsentForwardingMode::OpenrtbOnly, HeaderValue::from_static("session=fictional; euconsent-v2=fictional-tcf"), - Some(b"session=fictional".as_slice()), + None, ), ( ConsentForwardingMode::OpenrtbOnly, @@ -9880,7 +10226,7 @@ set = { networkId = 42 } ConsentForwardingMode::OpenrtbOnly, HeaderValue::from_bytes(b"session=fictional;\xffbroken") .expect("should build malformed cookie header"), - Some(b"session=fictional;\xffbroken".as_slice()), + None, ), ] { let stub = Arc::new(StubHttpClient::new()); diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index 4220851797..13c3c3c984 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -149,6 +149,10 @@ export default withMermaid( link: '/guide/integrations-overview', }, { text: 'Development Guide', link: '/guide/integration-guide' }, + { + text: 'Cookie Forwarding', + link: '/guide/integrations/cookie-forwarding', + }, { text: 'Ad Server Mock', link: '/guide/integrations/adserver_mock', diff --git a/docs/guide/integrations/cookie-forwarding.md b/docs/guide/integrations/cookie-forwarding.md new file mode 100644 index 0000000000..a45afbbd36 --- /dev/null +++ b/docs/guide/integrations/cookie-forwarding.md @@ -0,0 +1,91 @@ +# Integration Cookie Forwarding + +Browser requests to a first-party integration route can carry the publisher's session, authentication, identity, and consent cookies. Trusted Server limits which cookies reach third-party upstreams. The publisher-origin request has a separate purpose: it preserves the browser cookies needed by the publisher application. + +## Prebid Server + +Prebid Server (PBS) requests allow only these exact, case-sensitive consent cookie names: + +- `euconsent-v2` +- `__gpp` +- `__gpp_sid` +- `us_privacy` + +The `consent_forwarding` setting selects how consent travels: + +| Mode | Outbound `Cookie` header | OpenRTB body consent | +| ---------------- | ----------------------------------- | -------------------------------------------------------------------------------------------- | +| `both` (default) | Only the four consent cookies above | Included | +| `cookies_only` | Only the four consent cookies above | Only affected unsafe signals fall back to the body; other cookie-sourced signals are omitted | +| `openrtb_only` | Omitted | Included | + +Unrelated cookies are excluded in every mode. This includes publisher session and authentication cookies, `ts-ec`, `ts-eids`, and `sharedId`. An unreadable, non-ASCII Cookie header is omitted rather than forwarded unchanged. Cookie parsing uses the same parser as consent extraction: names and values are trimmed, and the last occurrence of each consent cookie wins. Values with unsafe outbound framing are omitted. In `cookies_only`, only the signal whose cookie cannot be forwarded safely is included in the OpenRTB body, so it cannot disappear from both channels. The GPP string and its section IDs fall back together if either GPP cookie is unsafe; safely forwarded TC and US Privacy signals remain omitted. Already valid values retain their framing, including quoted values. Decimal GPP section IDs with whitespace, such as `2, 6`, are normalized to `2,6`. When filtering leaves no allowed cookies, the outbound Cookie header is omitted. + +For example, in `both` mode this inbound header: + +```http +Cookie: session_id=example-session; euconsent-v2=example-consent; theme=dark +``` + +produces this PBS header: + +```http +Cookie: euconsent-v2=example-consent +``` + +Cookie forwarding and identity forwarding are separate decisions. Excluding identity cookies from the raw header does not remove the server's identity-resolution path. Trusted Server can read the relevant first-party cookies, resolve identity, and populate `user.id` and `user.ext.eids` under the applicable consent rules. Forwarding a consent cookie also does not authorize identity forwarding by itself. + +See [Prebid integration](./prebid) and [GDPR compliance](/guide/gdpr-compliance) for configuration and consent behavior. + +## Reviewed Upstream Surfaces + +| Surface | Cookie behavior | +| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| PBS auction provider | Uses the mode-dependent consent cookie allowlist above | +| Standard OpenRTB and APS auction providers | Construct fresh requests without copying the browser Cookie header | +| Lockr | Omits Cookie; forwards selected standard headers and client `x-*` headers except TS-internal names; the SDK supplies selected identity data in its `firstPartyCookies` body | +| Permutive | Copies selected standard headers and client `x-*` headers except TS-internal names; excludes Cookie | +| Didomi, DataDome SDK and browser API proxy | Copy curated request headers that exclude Cookie | +| GTM, GPT, Testlight, `/first-party/proxy`, JS and asset proxies | Forward fixed header allowlists that exclude Cookie | +| Sourcepoint | Uses its own partner-specific allowlist, described below | +| Ad server mock mediator and EC partner pull sync | Construct fresh requests without copying the browser Cookie header | +| Publisher origin | Preserves browser cookies for the publisher application | + +### Identity Values Outside Cookie Headers + +The table describes raw `Cookie` headers. Some upstream contracts also carry selected identity values separately: + +- The `/first-party/proxy` creative proxy adds the request's EC ID, read from the `x-ts-ec` header or else the `ts-ec` cookie, as a `ts-ec` query parameter. GTM, GPT, Testlight, the JS asset proxy, and the Prebid bundle fetch turn this off in code; it is not an operator setting. +- Click redirects add the request's EC ID as a `ts-ec` query parameter on the target URL. +- Testlight puts the `ts-ec` cookie value into its rewritten request body as `user.id`. +- EC partner pull sync sends the EC ID to the partner as an `ec_id` query parameter. + +These selected-value contracts are independent of the PBS cookie allowlist; excluding an identity cookie from a raw header does not remove its other uses. + +### Sourcepoint + +Sourcepoint's proxy allows its established cookie names: + +- `consentUUID` +- `euconsent-v2` +- `dnsDisplayed` +- `ccpaApplies` +- `signedLspa` +- `_sp_su` +- `consentDate` +- `usnatUUID` +- `consentDateUsnat` +- `globalcmpUUID` +- `consentDateGlobalcmp` + +Operators can additionally configure one custom Sourcepoint authentication cookie through `auth_cookie_name`. Unrelated publisher cookies are excluded, and an unreadable Cookie header is omitted. This partner contract is independent of the PBS consent cookie allowlist. See [Sourcepoint integration](./sourcepoint#cookie-forwarding-and-caching). + +### DataDome Protection + +The server-side DataDome protection request extracts the `datadome` cookie as a client identifier when an explicit client-ID header is absent. It also sends the length of the selected inbound Cookie header value as `CookiesLen`. It does not send the full browser Cookie header or unrelated cookie values through those fields. + +### Publisher Origin and Browser Requests + +Requests to the configured publisher origin intentionally retain browser cookies so sessions and authenticated application behavior continue to work. Third-party integration filtering must not remove those cookies from the publisher-origin request. + +Browser calls to Trusted Server's own page-bids, click, and signing routes can also include same-origin credentials. The upstream integration policy applies when Trusted Server constructs a subsequent third-party request. diff --git a/docs/guide/integrations/prebid.md b/docs/guide/integrations/prebid.md index 18db20e237..b7a05ac7be 100644 --- a/docs/guide/integrations/prebid.md +++ b/docs/guide/integrations/prebid.md @@ -135,6 +135,22 @@ The typed `profile_config` fields are: | `bid_param_override_rules` | `[]` | Ordered exact-match rules; later matching rules win | | `consent_forwarding` | `both` | `openrtb_only`, `cookies_only`, or `both` | +PBS never receives the browser's full `Cookie` header. `both` and +`cookies_only` forward only `euconsent-v2`, `__gpp`, `__gpp_sid`, and +`us_privacy`; `openrtb_only` sends no Cookie header. Identity and publisher +session cookies are excluded in all modes. Unreadable headers and malformed +cookie pairs are omitted, and no header is sent when no allowed pair remains. +Cookie parsing matches consent extraction, including trimmed names/values and +last-duplicate selection. Decimal GPP section whitespace is normalized. +Cookie-sourced consent is omitted from the OpenRTB body in `cookies_only` when +the extracted cookies can be forwarded safely. Only an affected unsafe signal +falls back to the body; the GPP string and section IDs fall back together if +either GPP cookie is unsafe. Policy-sourced consent retains its existing body +fallback. + +See [Integration cookie forwarding](./cookie-forwarding) for the complete +outbound audit and intentional partner-specific exceptions. + `notifications.suppress_all` replaces the old global notification switch. `notifications.suppress_seats` removes `nurl` and `burl` only for exact returned `seatbid.seat` values. It does not match bidder route IDs. See @@ -983,7 +999,7 @@ The downstream Prebid Server request includes: - `user.id` when EC forwarding is allowed - `user.ext.eids` containing the merged, deduplicated EID set -- forwarded browser cookies (subject to consent-forwarding mode) +- only allowlisted consent cookies in `both` and `cookies_only`; no Cookie header in `openrtb_only` In practice, this gives operators both: diff --git a/docs/superpowers/plans/2026-10-22-prebid-consent-cookie-allowlist.md b/docs/superpowers/plans/2026-10-22-prebid-consent-cookie-allowlist.md new file mode 100644 index 0000000000..b98600f75d --- /dev/null +++ b/docs/superpowers/plans/2026-10-22-prebid-consent-cookie-allowlist.md @@ -0,0 +1,51 @@ +# Prebid consent cookie allowlist implementation plan + +> **For agentic workers:** Execute centrally with the superpowers debugging, test-driven-development, and verification workflows; use independent agents for the outbound audit and final PR review. + +**Goal:** Resolve #763 by preventing arbitrary browser cookies from reaching Prebid Server, while preserving OpenRTB consent behavior. + +**Architecture:** Rebuild the PBS Cookie header in the shared transport helper from exact consent-cookie names only. Omit Cookie entirely for openrtb_only, non-ASCII headers, and inputs without valid allowed pairs. Keep Sourcepoint's intentional partner allowlist and publisher-origin forwarding intact. No new configuration or dependencies. + +**Tech Stack:** Rust, http HeaderValue, existing consent constants, native and Viceroy tests. + +## Scope and decisions + +The issue already specifies the desired policy. Filtering at the common PBS transport boundary covers planned provider dispatch and the legacy test wrapper. Copying then stripping more names is unsafe because unknown publisher cookies remain; the unused generic forwarding helper is removed so the old unrestricted forwarding path cannot be reused accidentally. + +Consent-cookie selection uses the same CookieJar parser as extraction, with case-sensitive allowlisted names and last-duplicate selection. A safe outbound value uses RFC 6265 cookie-octets (optionally enclosed in double quotes). Invalid pairs are omitted; an unreadable header is omitted entirely. Values may contain `=`. Empty values are valid. Comma-separated decimal GPP section IDs are normalized to remove section whitespace, preserving extraction semantics. If an extracted consent cookie cannot be framed safely, cookies_only carries only the affected signal in the body instead, with the GPP string and section IDs treated as one coupled signal. No cookie values are logged. Repeated inbound headers retain the existing selected-header behavior. + +The existing cookies_only exception for policy-sourced body consent is preserved. + +## Tasks + +- [x] Establish the current transport test baseline in the isolated worktree. +- [x] Audit all outbound integration/provider Cookie paths and record intentional exceptions. +- [x] Add a failing mixed-cookie dispatch matrix for all modes, asserting both captured headers and OpenRTB consent fields through production AuctionPlan/PlannedOpenRtbProvider dispatch as well as legacy coverage. +- [x] Add failing boundary regressions for malformed/non-ASCII input, exact-name matching, quoted values, missing/empty headers, repeated headers, and stale destination cookies; preserve attested XFF. +- [x] Implement a minimal PBS-specific allowlist filter in the shared transport helper; remove any destination Cookie before applying policy. +- [x] Run the regressions and adapter-matched tests; update stale expectations without weakening coverage. +- [x] Document operator-visible PBS behavior and outbound audit results, keeping partner-specific exceptions explicit. +- [x] Run all AGENTS.md CI gates and record command outcomes, including environmental limitations. +- [x] Independently review transport, consent extraction, serialization, and outbound audit behavior. + +## Review and regression evidence + +An independent plan review confirmed the common transport boundary and requested explicit production planned-provider coverage plus updated snapshot/wrapper documentation; both are included. + +The original transport baseline passed. New regressions then failed on the original implementation for raw cookie leakage, malformed-byte fallback, and stale destination cookies. After the fix, all 197 cookie-focused tests passed. The production matrix covers all three modes, cookie- and policy-sourced consent, mixed and malformed headers, no allowed cookies, absent headers, repeated Cookie fields, and top-level/ext OpenRTB consent placement. + +## Verification + +- Native core: 2,982 unit tests passed; both native integration suites passed (10 tests each). +- Fastly: 221 adapter and 2,976 core tests passed, with six existing wasm-only skips; reusable-sandbox suite: 221 passed. +- Axum, Cloudflare, and Spin adapter tests passed; native parity: 17 passed. +- All eight target-matched Clippy aliases passed, including CLI and codegen; native build-digest lint passed. +- Rust formatting, JavaScript formatting/build and 1,211 JavaScript tests passed. +- Documentation and outside-doc Markdown formatting, wasm-clock lint, and all documentation-proposal checks passed. +- Optimized Fastly WASM build passed. + +Initial validation used Rust 1.98.1, the locally available Node 24.12.0 executable, and Viceroy 0.21.0; the repository pins Node 24.20.0. The initially installed Viceroy 0.17.0 could not load the current SDK imports; the pinned version was downloaded separately for verification. Axum's loopback tests and Viceroy's macOS certificate access required sandbox-enabled retries. Those retries passed. The existing proc-macro-error2 future-incompatibility notice remains. No manual publisher deployment was performed. + +## Consent extraction alignment follow-up + +A planned-dispatch regression reproduced loss of both `__gpp_sid=2, 6` and `us_privacy = 1YYN`: consent extraction accepted the values but transport dropped them, while cookies_only omitted the body. Transport now uses the same CookieJar parser, with last-duplicate selection and safe decimal GPP whitespace normalization. A framing failure enables the JSON-body fallback for only the affected consent signal (coupling GPP and its section IDs) without forwarding unsafe bytes or unrelated cookies. Regressions cover all modes, decoded sale opt-out, GPP section IDs, duplicate values, quoted framing, and injected delimiters. diff --git a/docs/superpowers/specs/2026-08-10-config-first-auction-provider-architecture-design.md b/docs/superpowers/specs/2026-08-10-config-first-auction-provider-architecture-design.md index ca13e13e08..74765da3c4 100644 --- a/docs/superpowers/specs/2026-08-10-config-first-auction-provider-architecture-design.md +++ b/docs/superpowers/specs/2026-08-10-config-first-auction-provider-architecture-design.md @@ -452,17 +452,18 @@ A profile receives only: A profile does not receive the raw downstream HTTP request or unrestricted runtime services. Browser values needed by OpenRTB are normalized into canonical auction data before profile execution. -Request admission also retains a transport-owned Prebid header snapshot containing exactly the first values selected by the current HTTP header API for `Cookie`, `User-Agent`, `Referer`, and `Accept-Language`. It preserves accepted header bytes without introducing a second truncation rule; the existing inbound request/header limit remains authoritative. The snapshot is not exposed to profiles or other unrestricted runtime code. Common Prebid transport forwards the current `User-Agent`, `Referer`, and `Accept-Language` values and synthesizes `X-Forwarded-For` only from the platform-attested client IP, never from a client-supplied `X-Forwarded-For` value. The Prebid field policy may also use the raw accepted `Referer` for its existing `site.ref` behavior; the canonical `site.page` remains sanitized separately. APS and `standard` do not receive these raw browser headers. +Request admission also retains a transport-owned Prebid header snapshot containing exactly the first values selected by the current HTTP header API for `User-Agent`, `Referer`, and `Accept-Language`, plus the consent-cookie selection derived from the first `Cookie` value. The raw `Cookie` value is not retained, so later readers cannot reach publisher session or identity cookies. The snapshot preserves accepted `User-Agent`, `Referer`, and `Accept-Language` bytes without introducing a second truncation rule; the existing inbound request/header limit remains authoritative. The snapshot is not exposed to profiles or other unrestricted runtime code. Common Prebid transport forwards the current `User-Agent`, `Referer`, and `Accept-Language` values and synthesizes `X-Forwarded-For` only from the platform-attested client IP, never from a client-supplied `X-Forwarded-For` value. The Prebid field policy may also use the raw accepted `Referer` for its existing `site.ref` behavior; the canonical `site.page` remains sanitized separately. APS and `standard` do not receive these raw browser headers. -To preserve current Prebid consent-forwarding behavior, the compiled Prebid profile selects `openrtb_only`, `cookies_only`, or `both`, and common transport applies the existing behavior to the snapshotted `Cookie` value: +The compiled Prebid profile selects `openrtb_only`, `cookies_only`, or `both`. The consent-cookie minimization follow-up (#763) supersedes the original raw-header forwarding contract: -- `both` and `cookies_only` forward the complete selected `Cookie` header value unchanged. -- `openrtb_only` removes the existing allowlisted consent-cookie names and forwards the remaining cookies. -- When the header cannot be parsed by the existing stripping path, current fallback behavior is preserved, including forwarding the original non-UTF-8 value. -- If stripping removes every cookie, the upstream `Cookie` header is omitted. -- Consent originating from KV or policy state remains in the OpenRTB body when no browser consent cookie can carry it, including in `cookies_only` mode. +- `both` and `cookies_only` forward only `euconsent-v2`, `__gpp`, `__gpp_sid`, and `us_privacy`, using the same cookie parser and last-duplicate selection as consent extraction. +- `openrtb_only` omits the entire `Cookie` header. +- Unreadable headers and unsafe cookie values are omitted; raw headers are never forwarded as a fallback. +- Decimal GPP section IDs are normalized to remove surrounding whitespace in each section. +- `cookies_only` retains body consent for KV/policy-sourced signals. Cookie-sourced fallback includes only the unsafe affected signal, with the GPP string and section IDs coupled if either GPP cookie cannot be forwarded safely. +- No other publisher session, authentication, or identity cookie reaches PBS. -No other first-version profile receives a browser `Cookie` header. Reducing Prebid forwarding to consent cookies only, changing malformed-header handling, or otherwise redesigning these modes requires a separate consent-focused specification. +No other first-version profile receives a browser `Cookie` header. See [Integration Cookie Forwarding](../../guide/integrations/cookie-forwarding.md) for the current consent-focused transport contract. ## Canonical Auction Model @@ -1020,7 +1021,7 @@ No request with zero eligible impressions should be sent upstream. - [ ] Central privacy enforcement defines the maximum permitted data, and profile policies may only omit from that approved view. - [ ] The `prebid-server` and `aps` field policies preserve their current request differences, including the complete consent-field matrix. - [ ] The `standard` profile uses the documented shared PBS and APS baseline, consent placements, and omissions. -- [ ] Prebid transport preserves current `User-Agent`, raw `Referer`, `Accept-Language`, platform-attested `X-Forwarded-For`, and exact selected `Cookie` header behavior without exposing the raw request to profiles. +- [ ] Prebid transport preserves current `User-Agent`, raw `Referer`, `Accept-Language`, platform-attested `X-Forwarded-For`, and the selected-header consent-cookie allowlist without exposing the raw request to profiles. - [ ] The common driver preserves existing consent, identity, floor, timeout, currency, and signing wire semantics while intentionally expanding enabled signing coverage to every OpenRTB provider. - [ ] Tests prove that standard, Prebid Server, and APS requests receive the version 1.1 signing extension after profile request augmentation when global signing is enabled. - [ ] Tests prove that global signing disabled removes all signature-bearing fields, preserves the existing PBS-only host/scheme object, and omits `ext.trusted_server` from APS and `standard`. @@ -1031,7 +1032,7 @@ No request with zero eligible impressions should be sent upstream. - [ ] Client bidder parameters are not assigned an invented generic wire location. - [ ] The Prebid profile preserves current bidder parameters, stored requests, cache handling, diagnostics, and notification behavior. - [ ] Common notification suppression preserves `nurl` and `burl` by default, supports provider-wide suppression, and matches per-seat suppression against exact returned seat IDs independently of bidder routes. -- [ ] Prebid parity tests cover the exact existing `openrtb_only`, `cookies_only`, and `both` Cookie-header behavior, including KV/policy-sourced body-consent fallback. +- [ ] Prebid parity tests cover the `openrtb_only`, `cookies_only`, and `both` consent-cookie allowlist behavior, including KV/policy-sourced body-consent fallback. - [ ] The APS profile preserves current account extensions, inventory identity, response validation, renderer, script policy, diagnostics, and deterministic highest-price-per-impression reduction with its bid-ID tie-breaker. - [ ] Prebid and APS no longer register singleton auction-provider instances. @@ -1062,7 +1063,7 @@ The following require separate requirements before implementation: - New money or currency-conversion models. - New signing protocol versions. - New privacy or data-sharing controls. -- Consent-cookie forwarding minimization or changes to existing malformed-header behavior. +- Further consent-cookie transport changes beyond the #763 follow-up documented above. - New diagnostics and telemetry schemas. - General endpoint authentication configuration. - Arbitrary standard-field mappings.