[tanderson-ld]

Requirements

• I have followed the repository's pull request submission guidelines
• I have added test coverage for new or changed functionality — n/a, CI configuration only
• I have validated my changes against all supported platform versions — n/a, the nightly runs a single JDK, unchanged

Related issues

SDK-3007

Describe the solution you've provided

The nightly contract test job has been passing -enable-long-running-tests to the v3 harness step only. The harness's long-running RETRY-conformance tests live on the v2 line (released in harness v2.41.0), so every one of them has been skipping every night. From the 2026-09-03 nightly log:

SKIPPED: streaming/retry behavior/retry after unexpected HTTP error on initial connect (use -enable-long-running-tests to run)
SKIPPED: streaming/retry behavior/retry after unexpected HTTP error on reconnect (use -enable-long-running-tests to run)
SKIPPED: streaming/retry behavior/enters extended-regime backoff after unexpected HTTP error (use -enable-long-running-tests to run)
SKIPPED: streaming/retry behavior/does not permanently stop under sustained unexpected HTTP errors (use -enable-long-running-tests to run)
SKIPPED: streaming/retry behavior/returns to normal-regime backoff after healthy operation (use -enable-long-running-tests to run)
SKIPPED: polling/retry behavior (use -enable-long-running-tests to run)

The test service has declared retry-conformance-fdv1-streaming and retry-conformance-fdv1-polling since #200, so these tests were ready to run — they simply were not being asked to.

Changes:

• v2 step gets -enable-long-running-tests. This is the substantive change; everything else supports it.
• timeout-minutes: 120. The job goes from ~11 min to an estimated ~60–90 min, and the 6-hour default is too permissive for a suite whose failure mode is waiting.
• -junit on both harness steps, at distinct paths, plus if: always() artifact uploads of the results and the test service log. Triaging an hour-long run from raw step logs is impractical.
• New "Create JUnit output directory" step. The harness writes its JUnit file with os.WriteFile and does not create parent directories; a write error aborts the run, so -junit requires the directory to exist first.
• Notify on any non-clean pass rather than only failure, with the actual result included in the message, so a timeout or cancellation is not silently swallowed.

On the history here: the flag originally applied to both harness versions (#136) and was narrowed to v3-only the next day (#139). That was correct at the time — the v2 long-running tests did not exist yet. #164 faithfully preserved the narrowing when migrating to the shared action. This restores v2 coverage now that there is something there to run.

Describe alternatives you've considered

• Splitting the long-running tests into a separate job, so a timing flake cannot mask the fast suite. Not done: it duplicates build and service startup, and the fast suite costs about a minute inside the slow job. Worth revisiting if the first couple of weeks show flakiness.
• A concurrency group to stop a manual dispatch overlapping the scheduled run. Rejected: GitHub-hosted jobs each get their own ephemeral VM, so concurrent runs share no port and no state. A group would only queue a dispatch behind an in-flight ~90-minute run, or silently cancel a superseded pending run — which would obstruct the dispatch-based validation this change relies on.
• run_tests: 'false' on the shared CI action to skip duplicate unit tests. Rejected after measuring: gradlew build already runs them (~3.5 min), and the separate Run Tests step is :test UP-TO-DATE in one second. It would have saved a second, not the ten minutes I initially assumed.

Additional context

• Expect a much longer nightly. These tests wait on real RETRY backoff timing (a 5-minute extended-regime delay with up to 50% jitter) and the harness runs tests sequentially, so the runtime is almost entirely sleeping. Failures are slower than passes: a regression that stops retrying makes each scenario burn its full timeout instead of returning early.
• Validation before merge: dispatch this workflow against this branch and confirm zero use -enable-long-running-tests to run lines —
  gh workflow run nightly-contract-tests.yml --ref ta/SDK-3007/nightly-long-running-tests. That run takes about an hour.
• The Slack notification wiring is unchanged in shape. Note the webhook has not been functional, so failures are currently discoverable only from the Actions tab; fixing that is tracked separately and deliberately out of scope here.
• The v3 harness step stays pinned at v3.0.0-alpha.6; bumping it would pull in new FDv2 tests and require re-validating the FDv2 suppressions.
• actions/checkout@v3 in this workflow is flagged by actionlint as too old. Pre-existing on main and left alone here.

---

Note

Overview
The nightly contract test job now runs v2 harness long-running RETRY conformance tests by adding -enable-long-running-tests to the v2 step (they were previously skipped while only v3 had the flag).

To support ~hour-long runs, the main job gets a 120-minute timeout, JUnit output via -junit on both v2 and v3 steps (with a junit/ directory created first), and if: always() artifact uploads for JUnit XML and the contract test service log.

Slack notification now fires on any non-success outcome (timeout/cancel included) and reports the actual job result instead of only hard failures.

^{Reviewed by Cursor Bugbot for commit bd06463. Bugbot is set up for automated code reviews on this repo. Configure here.}