Repository navigation
Conversation
The bridge links the crate behind a streams feature that is on by default, so the plugin can build the Redis worker extension by name. The plugin buffers each Workflow Activation's Stream calls and hands them to the crate at the end of the Activation, and it gives Activities and clients their writers.
Activity keys come from a per-run counter, so the plugin leaves the Workflow's random values alone. A retried attempt withdraws without a create, the last run always closes the default Stream, and chain_continues covers retries, cron and the execution timeout. The streams feature is now off by default and on in the official builds.
The crate hardened how it tracks closes in flight, which the Worker shutdown drain depends on.
Python closes an evicted run's coroutine outside the Workflow's loop. The interceptor then called the runtime, and each eviction printed an Exception ignored traceback.
Owner
Author
|
Closing as stale. This layer is folded into the v5 tip, #365, and the upstream review uses a fresh series instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
This PR adds
temporalio.contrib.streams.redis.RedisStreamPlugin, and links thetemporalio-streamscrate into the bridge behind astreamsfeature. The feature is off by default and on in the official builds (poe build-develop, the cibuildwheel wheels andfips). The bridge builds it as thetemporalio-streams-redisworker extension.The plugin:
extensionoption and gives the facade aStreamBackend;CreateStream,AppendRecordsandCloseStreamin a System Nexus outbound interceptor and returns a done handle;on_workflow_activation_complete, merges adjacent appends, and callsBufferWorkflowCalls;auto_closeStreams it wrote, withchain_continuesfor a retry (attempts and time left), a cron run, or nothing for continue-as-new;withdraw_runon every call of a retry run;workflow.random()stays the same, and builds the Activity, Local Activity and Standalone Activity Producers and epochs;temporalio.contrib.streamsthrough the sandbox.Why?
One plugin serves Workflow code, Activities and the client, as in the proposal. The crate must be in the same native library as Core, so it links into the bridge.
How did you test it?
pytest tests/contrib/streams/test_redis.py tests/contrib/streams/test_chain.pyagainst the dev server and a localredis-serveron a free port. It covers the default Stream, scoped Streams, a client Stream that a Workflow writes to, Activity retries, a failed Workflow Task, the Workflow's random values, a retried Activity with no Streams, a close after continue-as-new, a cancelledwithblock, a sync Activity, temporal-only sandbox passthrough, and eachchain_continuescase.Please note that an offline build of the sdist still fails with the feature off, since Cargo resolves the optional git dependencies.