Skip to content

feat(stovepipe): apply queue transitions and materialize history - #856

Merged
mnoah1 merged 7 commits into
mnoah1/stovepipe-queue-state-apifrom
mnoah1/stovepipe-queue-state-transitions
Oct 16, 2026
Merged

mnoah1 merged 7 commits into
mnoah1/stovepipe-queue-state-apifrom
mnoah1/stovepipe-queue-state-transitions

Conversation

@mnoah1

@mnoah1 mnoah1 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Why?

Queue enablement changes must preserve coordination fields and remain recoverable when history or summary writes fail.

What?

Add an internal TransitionController with Initialize, Prepare and Apply, plus queuelog.Materializer.PersistLog. Retain the previous occurrence before replacing internal enablement and materialize the committed winner afterward. Prepared retries preserve the accepted boundary; optimistic locking handles operational races. Startup registers configured queues as disabled without overwriting registered enablement.

Test Plan

Unit, service integration and end-to-end tests cover competing decisions, operational races, projection races and failed writes. Run make local-stovepipe-start for local startup with schemas applied before registration. Changing enablement uses the internal transition controller; this layer adds no mutation RPC or CLI.

Revert Plan

Revert this layer together with any dependent layers in the stack.

Issue Links

None — this is a standalone queue-status proposal.

Stack

Review in this order; each PR targets the preceding branch.

  1. API and RFC
  2. Storage
  3. Transitions and materialization (this PR)
  4. Lookup

@mnoah1
mnoah1 added this pull request to stack #858 October 14, 2026 15:45
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch from 354212a to 07918a3 Compare October 14, 2026 15:56
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch from 07918a3 to 17a0f86 Compare October 14, 2026 17:27
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch 2 times, most recently from 017ff3d to c7526a0 Compare October 14, 2026 17:51

@behinddwalls behinddwalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I traced the CAS and reconcile paths and didn't find a lost update. Version arithmetic stays in the controller, and the read-model repair is idempotent. My only comment is about error context.

Comment thread stovepipe/core/queuestate/queuestate.go Outdated
Comment thread stovepipe/core/queuelog/materializer.go Outdated
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch 2 times, most recently from c127e85 to f6a12a1 Compare October 14, 2026 22:35
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch from f6a12a1 to 61a852f Compare October 14, 2026 22:49
@mnoah1 mnoah1 changed the title feat(stovepipe): [DRAFT] apply queue transitions and materialize history feat(stovepipe): apply queue transitions and materialize history Oct 14, 2026
@mnoah1
mnoah1 marked this pull request as ready for review October 14, 2026 23:40
@mnoah1
mnoah1 requested review from a team and sbalabanov as code owners October 14, 2026 23:40
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch from df46cdb to 8926671 Compare October 16, 2026 14:26
Summary:
Queue enablement needs durable internal state and separate read models so orchestration updates do not alter enablement history.

Extend the existing queue snapshot, add immutable QueueLog and derived QueueSummary records, and implement key-oriented stores with conditional writes. Include the one-time migration for existing queue tables and backend tests for byte-exact queue identity and independent projection versions.

Part 2 of 4, built on the API and RFC. This layer supplies storage for transitions and lookup.

Test Plan:
Run the Stovepipe storage unit and MySQL integration tests. Migration coverage verifies existing operational values survive. Before deploying to an existing database, apply queue_enablement.sql once and create the queue log and summary tables.
Summary:
Queue enablement changes must preserve coordination fields and remain recoverable when history or summary writes fail.

Add an internal TransitionController with Initialize, Prepare and Apply, plus queuelog.Materializer.PersistLog. Retain the previous occurrence before replacing internal enablement and materialize the committed winner afterward. Prepared retries preserve the accepted boundary; optimistic locking handles operational races. Startup registers configured queues as disabled without overwriting registered enablement.

Part 3 of 4, built on state and read-model storage. Configuration observation, administration, pause and execution gating remain deferred.

Test Plan:
Unit, service integration and end-to-end tests cover competing decisions, operational races, projection races and failed writes. Run make local-stovepipe-start for local startup with schemas applied before registration. Changing enablement uses the internal transition controller; this layer adds no mutation RPC or CLI.
Apply schemas and fixtures before starting services, and cover startup order and cleanup in the runner test. Add queue and failure context to enablement and history errors while preserving sentinel matching.
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-queue-state-transitions branch from 8926671 to 971a152 Compare October 16, 2026 17:24
@mnoah1
mnoah1 added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit e90d383 Oct 16, 2026
19 checks passed
Base automatically changed from mnoah1/stovepipe-queue-state-storage to mnoah1/stovepipe-queue-state-api October 16, 2026 17:43
@mnoah1
mnoah1 deleted the mnoah1/stovepipe-queue-state-transitions branch October 16, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants