Skip to content

feat(dgw): run AI analysis of a recording as a task - #2044

Open
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 58 commits into
masterfrom
feat/recording-ai-analysis
Open

irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 58 commits into
masterfrom
feat/recording-ai-analysis

Conversation

@irvingoujAtDevolution

@irvingoujAtDevolution irvingouj@Devolutions (irvingoujAtDevolution) commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Gateway can now have an AI model analyze a recording and add the result to the recording as the log ai-analysis-0.slog.
A provisioner such as DVLS starts an analysis with POST /jet/jrec/{session_id}/ai-analysis, which needs the scope gateway.tasks.recording-ai-analysis.start, and follows it with GET /jet/jrec/{session_id}/ai-analysis/{task_id}, which needs the scope gateway.tasks.recording-ai-analysis.read.
The request names the provider (OpenAI, Anthropic, Mistral, Gemini or any OpenAI-compatible endpoint), the model, the API key and the language of the result, which defaults to English.
The analysis runs in the background, retries temporary failures such as rate limits and server errors, and reports the tokens it used.
It fails if it doesn't start within 2 hours or makes no progress for 30 minutes, so a long recording is never cut off while the analysis keeps progressing.
Gateway keeps the API key and the base URL encrypted in memory only, never on disk or in its logs, so an analysis still running when Gateway restarts fails.
A new analysis replaces the earlier one only once it succeeds; a failed one removes what it made and leaves the earlier analysis in place.

Issue: DGW-462

@irvingoujAtDevolution irvingouj@Devolutions (irvingoujAtDevolution) changed the title feat/recording ai analysis feat(dgw): run AI analysis of a recording as a provisioner task Oct 6, 2026
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown

Let maintainers know that an action is required on their side

  • Add the label release-required Please cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module) when you request a maintainer to cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module)

  • Add the label release-blocker Follow-up is required before cutting a new release if a follow-up is required before cutting a new release

  • Add the label publish-required Please publish libraries (`Devolutions.Gateway.Utils`, OpenAPI clients, etc) when you request a maintainer to publish libraries (Devolutions.Gateway.Utils, OpenAPI clients, etc.)

  • Add the label publish-blocker Follow-up is required before publishing libraries if a follow-up is required before publishing libraries

Windows can't delete the database and log files a running Gateway holds open, so each test left its folder behind.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…er accepts

The server reads a `u32`, which an `int32` schema can't hold, so the clients now get a 64-bit integer from 0 to
4294967295.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The AI access of a Task now lives in the standalone `SecretStore`,
keyed by the Task ID, instead of the provisioning store, which is
about sessions only. Gateway holds one store in `DgwState`, hands it
to the provisioner tasks, and runs its expiry cleanup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Task secrets live in the standalone `SecretStore`, not in the
provisioning store, which is about sessions only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record the owner's rulings from the intent review of the AI analysis
Task: a new Task replaces the earlier analysis only once its success is
recorded, a failed Task leaves no file behind, attempt errors are
recoverable or permanent, token usage is always reported, a running
Task fails after 30 minutes without progress, and the base URL is a
secret like the API key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Task keeps its API key until its deadline, and that deadline now
moves while the Task makes progress, so the key must follow it.
An expired or removed secret stays gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Errors: an attempt fails with a closed kind and no text, recoverable
or permanent. A failed Task's payload is `{ error, usage }`, so the
caller words the error.

Usage: every payload but `queued` carries the tokens used so far, over
all attempts. Requests whose usage the provider did not report are
counted apart in `unreportedRequests`.

Time: a Task fails if it does not start within 2 hours. Once it runs,
each attempt start and each progress report moves its deadline 30
minutes ahead, and the runner cancels an attempt past it. Waiting for
a remux is not progress. The store fails an overdue Task with
`deadlinePassed` and keeps the usage of its last payload.

Success: the new log sits in the session unlisted until the success is
recorded. Ending the Task then lists it and drops the earlier analysis.
If Gateway stops in between, the job runs again at startup and redoes
that step. A failed Task removes everything it made and leaves the
earlier analysis in place.

The session manifest is written to a file of its own, then renamed
over the old one, so a crash never leaves half of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The end of a Task, such as listing its AI analysis log, ran from the
Task's own job, whose runs are shared with the attempts. Once those
runs were used up, a failed end was never done again, not even at the
next start.

A failed end now hands the Task to a new job that ends it again five
minutes later. The job is stored, so a restart runs it too.

Deadlines are also cut to whole seconds, as the store keeps them, so
the runner and the store agree on when one passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request was counted only once its answer came. One cut off by the
deadline or a restart was never counted, so the usage looked complete
when it was not. Each request now counts as unreported from the moment
it is sent, and its reported tokens replace that count on answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Task whose end failed could list its log later, over the analysis of
a newer Task of the session. A new Task now removes the logs earlier
Tasks left unlisted, so such a late end finds nothing to list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The manifest is written to a temporary file first. Its name now starts
with a dot, which jrec never serves, so a file a crash leaves behind is
never downloaded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OpenAPI document writes the error kinds out by hand, since utoipa
drops the descriptions of variants without fields. An exhaustive match
now stops the build until a new kind is documented too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new Task removed every log earlier Tasks left unlisted. When the end
of a succeeded Task was still due, the new Task dropped an analysis
whose success was recorded, and a new Task that then failed left the
session without it.

A new Task now does what the end of the earlier Task would: it lists
the log of a succeeded Task, and removes the log of a failed or unknown
one. The runner gives an attempt the state of another record for this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the success of a Task could not be recorded, any error that came
after the failure was recorded was only logged. An end that could not
be queued again was then lost. That error now reaches the job queue,
whose retry finds the Task finished and only ends it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When neither the success nor the failure of an attempt could be
recorded, the job ended and nothing ended the Task, so its log stayed
on disk. A job now runs just after the deadline, when the store has
failed the Task, and only ends it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ID of an unlisted log comes from a file name, so a Task of another
session now reads as unknown, and its log is removed instead of listed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No intent sentence decides what these seven tests expect, but each one alone catches a fault, so their markers now name that fault instead of an intent section that does not decide the value.

Issue: DGW-462

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The work of the progress test took 2.5 s against a first deadline 2 to 3 s away, and it ignored the cancellation, so the test passed even when progress never moved the deadline.
It now works for 4 s and stops once cancelled, so only a moved deadline lets it succeed.

Issue: DGW-462

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tokens of a failed Task are decided by the jrec intent, and the deadline an attempt start sets by the 30 minutes without progress of the provisioner tasks intent, not by the sections these tests cited.

Issue: DGW-462

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The metadata of a Task belongs to the task reading scope, so reading it with only `gateway.recordings.read` must be refused; no test sent that scope.

Issue: DGW-462

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants