Skip to content

replayer: add bounded Rust dry-run foundation - #210

Closed
bb7133 wants to merge 4 commits into
mainfrom
codex/cp-replayer-152
Closed

bb7133 wants to merge 4 commits into
mainfrom
codex/cp-replayer-152

Conversation

@bb7133

@bb7133 bb7133 commented Sep 3, 2026 •

Copy link
Copy Markdown
Owner

What is changed

This draft advances the standalone Rust owner for CP-REPLAYER (#152) without changing the SQL-serving binary or claiming production cutover.

  • freeze the Rust offline-replayer ownership, compatibility, restart, and safety contract
  • add a standalone tiproxy-replayer composition root which currently accepts dry-run only
  • add bounded OpenDAL input ownership for fs, s3, gcs, azblob, oss, and cos
  • decode TiProxy native captures plus audit plugin v1/v2 records with deterministic ordering
  • preserve Go-compatible prepared-statement and read-only-filter behavior, including binary KindBytes
  • support gzip + AES-256-CTR native inputs with bounded stored, expanded, metadata, and record sizes
  • add content/config-bound, full-frontier schema-v2 atomic local checkpoints
  • add an immutable Go/Rust semantic corpus, differential comparator, and mutation kill to CI

Review hardening

The second exact head addresses the independent semantic review:

  • make bare checkpoint filenames sync the current directory correctly
  • preserve binary plugin parameters as bytes instead of rejecting non-UTF-8 payloads
  • add command_ordinal to ordering, resume frontiers, checkpoint schema, and input identity
  • reject unknown, duplicate, root, bucket, and container object-store query overrides
  • match the Go audit-extension command-start/end frontier mapping
  • fail closed on start-time grace-period overflow
  • narrow the evidence, retry, zeroization, and reorder-buffer claims to what this foundation proves
  • shorten all temporary OpenDAL RustSec exception expiries to 2026-12-01

Honest scope boundary

This is a foundation-only draft and does not close #152. The following remain intentionally fail-closed or deferred:

  • backend MySQL/TLS/authentication, response draining, pacing, reconnects, and error policy
  • replay-only HTTP service mode and lifecycle
  • dynamic input / wait-on-EOF ownership and remote checkpoint publication
  • SQL/report output, signals, supervised restart, and release packaging
  • bounded storage retry plus MinIO/provider, fault, and restart evidence
  • bounded merge/scheduling that honors the configured reorder window
  • final bridge/Go-owner removal and production cutover

tiproxy-replayer rejects service mode, backend execution, dynamic input, wait-on-EOF, and SQL output in this slice.

Supply-chain exception disclosure

The workspace is pinned to Rust 1.89 and Tokio 1.47.1. OpenDAL 0.53.3 is therefore used for the required provider set; its signer graph currently carries RUSTSEC-2023-0071, RUSTSEC-2026-0194, and RUSTSEC-2026-0195. OpenDAL 0.58 resolves the quick-xml advisories but requires Rust 1.91.

The exceptions are explicitly synchronized across cargo-audit, cargo-deny, and the repository-owned exception ledger with owner @bb7133, rationale, and expiry 2026-12-01. Exposure in this draft is bounded to an operator-run dry-run binary with trusted storage endpoints; backend and service modes fail closed. The OpenDAL/toolchain upgrade is required before enabling either deferred mode.

Verification

  • make rust-test
  • make rust-lint
  • cargo test -p replayer-core --locked
  • tests/replayer/differential/run.sh (Go/Rust equality + mutation detected)
  • python3 rust/ci/check_vulnerability_exceptions.py
  • hosted Audit dependencies and policy (cargo-audit + cargo-deny + exception ledger)
  • git diff --check

Issue: #152

@bb7133

bb7133 commented Oct 16, 2026

Copy link
Copy Markdown
Owner Author

Migrated to pingcap#1388: pingcap#1388

This source PR is closed as part of the upstream migration; its historical timeline and checks remain here.

@bb7133 bb7133 closed this Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CP-REPLAYER] Offline replayer CLI (M2, deferrable)

1 participant