Skip to content

test(e2e): Run live delivery specs in parallel - #1331

Closed
utkarshcloudinary wants to merge 1 commit into
developfrom
ci/parallel-live-e2e-specs
Closed

utkarshcloudinary wants to merge 1 commit into
developfrom
ci/parallel-live-e2e-specs

Conversation

@utkarshcloudinary

Copy link
Copy Markdown
Collaborator

No issue. CI speed-up, follow-up to #1330.

Approach

The E2E job is the longest CI job, and its single-worker @serial pass (18 tests, about 1.8 min) is the biggest part. Five of the eight @serial files were tagged only because they need the real Cloudinary connection, which every analytics spec overwrites with fake credentials. They do not break each other: they write the same real URL, and each one changes only settings that the others do not check. This PR moves them to a new multi-worker pass.

  • New tag @live on the image, video, gallery, SVG and asset-edit delivery specs.
  • npm run test:e2e now runs three passes: analytics and other safe specs (3 workers), @live (3 workers), then @serial (1 worker).
  • The 3 files that really change the whole site stay @serial: deactivation (it uninstalls the plugin), wizard setup, and connection analytics.
  • New tests/e2e/utils/live-site.js. It makes the steps that act on site-wide state take turns through a file lock:
    • wp cloudinary sync, because it is not scoped to one attachment. Its analyze step clears and re-adds _cld_unsynced on every attachment, so two parallel syncs can make one of them find nothing.
    • The gallery's /cloudinary/v1/image_data sync.
    • Post cleanup.
    • Settings writes to the shared media_display option (SVG settings and video_player).
  • The lock is an atomic mkdir in artifacts/. It records the owner PID, so a worker that dies while holding it does not block the run.

I tested the lock alone with 4 processes and 320 lock sections (sync and async callbacks): no overlaps. A lock from a dead PID was taken over at once.

QA notes

  • In the E2E job log, npm run test:e2e should show three Playwright runs: about 20 tests on 3 workers, 9 @live tests on 3 workers, and 9 @serial tests on 1 worker.
  • Compare the "Run E2E tests" step time with recent develop runs (about 2.6 to 2.8 min).
  • To run only the new pass locally: npm run test:e2e:live.

Five specs were tagged @serial only because they need the real
Cloudinary connection, which the analytics specs overwrite with fake
credentials. They do not break each other, so run them in their own
multi-worker pass, tagged @LiVe, between the analytics pass and the
serial pass.

They share one site, so the steps that act on site-wide state take
turns through a file lock in tests/e2e/utils/live-site.js:
`wp cloudinary sync`, whose analyze step clears and re-adds
`_cld_unsynced` on every attachment, the gallery's image_data sync,
post cleanup, and settings writes to the shared media_display option.
@utkarshcloudinary

Copy link
Copy Markdown
Collaborator Author

Closing. The CI run showed no gain: the E2E step took 2:53 against 2:33 to 2:48 on develop. The site lock makes the slow sync steps run one at a time, so the @LiVe pass saved only about 6s. It also made asset-edit flaky, because the SVG spec changes image_format/image_quality while asset-edit compares full delivery URLs. A better next step is separate WordPress sites per pass (the second wp-env site, or 2 CI runners).

@utkarshcloudinary
utkarshcloudinary deleted the ci/parallel-live-e2e-specs branch October 20, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants