diff --git a/.changeset/capture-only-page-changes.md b/.changeset/capture-only-page-changes.md new file mode 100644 index 00000000..944a6320 --- /dev/null +++ b/.changeset/capture-only-page-changes.md @@ -0,0 +1,7 @@ +--- +'@wdio/devtools-service': patch +'@wdio/selenium-devtools': patch +'@wdio/nightwatch-devtools': patch +--- + +Trace mode no longer takes a page capture (screenshot plus page source, and DOM on web) for read and wait commands such as `getText`, `isDisplayed` or `waitForDisplayed`. Those rows still appear in the trace and now show the page as of the last command that changed it. Each capture costs a full page-source round trip, which on iOS is slow enough that a suite making a dozen interactions spent most of its time capturing reads: one reported run went from 3:10 to 7:10 in trace mode. The first traced command of each test is still captured even when it is a read, so a test that starts by reading has a frame to show (on Nightwatch's `describe`/`it` interface, the first of each module). diff --git a/CLAUDE.md b/CLAUDE.md index e06da11a..d06791e5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -367,7 +367,7 @@ Documented divergences from the conventions above. They exist today as debt to b - **Locator generation from a native page source must stay linear, and iOS is where it did not.** Every native action snapshot builds element locators from the page-source XML (`core/locators`), and each uniqueness check was a `//*[@name="…"]` XPath run by the `xpath` library over the whole document, several per element. Measured on the iOS Settings page source tiled up: 38 KB 82 ms, 188 KB 1.4 s, 752 KB 23.5 s, which is what a reporter saw as trace mode adding 1.5 min to a 2 min 20 s iOS test while Android, with smaller flatter trees, added 10 s. `locators/xpath-index.ts` indexes the document once per `generateAllElementLocators` walk (`withXPathIndex`) and answers `//*`, `//Tag` and `//Tag[@a="x" and …]` from it: 15 ms, 49 ms, 268 ms for the same three, identical elements. It is scoped to the walk so a caller of the exported `evaluateXPath` that edits its document is never answered stale, and it declines a namespaced document, where XPath name tests resolve namespaces and the index does not. The page-source request itself remains: 0.75-1.2 s per action on XCUITest even for Settings, against 0.09-0.5 s on UiAutomator2. - A filmstrip poll against a native session stacks requests: `setInterval` never waits for its async handler, a native screenshot takes ~1.2 s against the 200 ms default, and a serialised driver serves that queue ahead of the test's own commands — measured, a **15 ms command took 4.5–7.8 s**. `ScreencastRecorderBase`'s `#pollInFlight` latch bounds it to one outstanding shot, and `#pollGeneration` stops a shot orphaned by `stop()` from appending after the stop or clearing its successor's latch. - **`start()` and `stop()` are serialised against each other, because unserialised neither owns what it arms.** A `stop()` landing mid-handshake returned at its `#isRecording` guard — still false, the handshake not having finished — so it never reached its own CDP teardown, and the `start()` it interrupted then returned on the generation mismatch without stopping what it had armed: the session and its frame listener stayed live after teardown. The frames themselves were moot — every adapter builds a fresh recorder per session (`screencast-lifecycle.ts` does so even across `reloadSession`), so the leaked session and its browser-side stream were the damage, not buffer pollution. The obvious repair — stopping the session on that mismatch path — is wrong, and it is the shape to keep out: it guards a field both CDP overrides write at handshake **start** (`#cdpSession` / `#cdp`) with a flag they set only at handshake **end** (`#cdpActive`), so a start arriving in that window is indistinguishable from no start at all and the stale path tears down the *newer* recording's session instead. Overlapping starts are the adapters' normal mode rather than a corner: Selenium's `driverPatcher` documents the second test's body racing an in-flight `screencast.start()`, and Nightwatch's session rotation is fire-and-forget from a command hook. Queued, a stop always runs against a start that has finished arming and owns what it armed, which makes the mismatch branch unreachable and deleted — the tick-phase generation checks in `#startPolling` are **not**, since a shot orphaned by a stop still lands afterwards. Two consequences: `stop()` now waits on an in-flight handshake, and every awaited driver primitive in that handshake is ceilinged at `SCREENCAST_HANDSHAKE_TIMEOUT_MS` (the service's `getPuppeteer()`/`pages()`/`createCDPSession()`/`Page.startScreencast` and the stop-side `Page.stopScreencast`, Selenium's `createCDPConnection`, the polling path's first screenshot) — a driver that wedges gives up after the ceiling, falls back to polling, or reports the screencast unavailable when polling was what wedged. Nothing is claimed until the handshake has answered, so a timed-out start leaves nothing armed — and a session or connection that completes after the ceiling is detached (or its socket closed) when it lands, so no orphan outlives the recording. And a stop landing during a polling session's first screenshot now records that shot before tearing the loop down, where it previously cancelled the start outright; the interval is still never left armed underneath the stop. -- Residual: **a polling matcher fires one capture per poll** — each poll is a top-level mapped read — so "one capture per action" undercounts a real suite. Pre-existing in `d924a02`, unmeasured on a device, and the likeliest next lever. +- **Only a command that changes the page is captured** (`shared/trace-actions.ts` `changesPage`, read off the typed `METHOD_EFFECT` table that every `ACTION_MAP` method must appear in). Reads and waits still render rows and inherit the latest earlier capture through `claimAfter`'s `latestAtOrBefore` fallback. The gate used to be `mapCommandToAction` itself, so `5c54ae4` adding the reads to the table for the timeline also made each of them a capture, and a polling matcher one capture per poll: a reported iOS suite made 187 page-source reads for 12 interactions and went from 3:10 to 7:10. The first mapped command of each test is still captured even when it reads, or a test that opens with reads has no frame to inherit. - The next test's first pre-capture used to be stamped at the previous test's last-action timestamp — the log it scans is run-long, so a test boundary was invisible to it. That slot already holds the previous test's finalize capture, and the richer-screenshot merge could replace it: with a `reloadSession` between the tests the row replayed the post-reload page. The capture now stamps `Date.now()` when the scanned timestamp predates `#currentTestStartWallTime` (0 without per-test hooks, so the standalone path is unchanged) — the same rule that makes a session's first capture its initial frame. Per-test slices were never affected: `flushTest` runs inside `afterTest`, before the next test's commands. ### File-size (raw line counts; soft cap is 500 logic lines) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index ea5a488b..6c552c44 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -16,6 +16,7 @@ import { type CapturedPerformancePayload, type SessionCapturerOptions } from '@wdio/devtools-core' +import { changesPage } from '@wdio/devtools-shared' import { captureActionSnapshot } from './action-snapshot.js' import { nightwatchRunnerId } from './helpers/utils.js' import { NAVIGATION_COMMANDS } from './constants.js' @@ -91,6 +92,13 @@ export class SessionCapturer extends SessionCapturerBase { readonly actionSnapshots: ActionSnapshot[] = [] readonly snapshotCaptures: Promise[] = [] + /** True until the current test's first mapped command takes its capture. */ + awaitingInitialFrame = true + + resetTestCapture(): void { + this.awaitingInitialFrame = true + } + constructor( devtoolsOptions: SessionCapturerOptions = {}, browser?: NightwatchBrowser @@ -168,13 +176,18 @@ export class SessionCapturer extends SessionCapturerBase { * after the command completes, and export claims snapshots by exact equality * with the command timestamp (FrameSnapshotIndex.claimAfter / elementsAt). */ #queueActionSnapshot(command: string, timestamp: number): void { - if ( - this.traceMode !== 'trace' || - !this.#browser || - !mapCommandToAction(command) - ) { + if (this.traceMode !== 'trace' || !this.#browser) { return } + // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. + const initialFrame = + this.awaitingInitialFrame && !!mapCommandToAction(command) + if (!changesPage(command) && !initialFrame) { + return + } + if (initialFrame) { + this.awaitingInitialFrame = false + } this.snapshotCaptures.push( captureActionSnapshot( this.#browser, @@ -185,6 +198,8 @@ export class SessionCapturer extends SessionCapturerBase { ).then((snap) => { if (snap) { upsertRichestSnapshot(this.actionSnapshots, snap) + } else if (initialFrame && !changesPage(command)) { + this.awaitingInitialFrame = true } }) ) diff --git a/packages/nightwatch-devtools/src/test-lifecycle.ts b/packages/nightwatch-devtools/src/test-lifecycle.ts index 4c181704..508071e4 100644 --- a/packages/nightwatch-devtools/src/test-lifecycle.ts +++ b/packages/nightwatch-devtools/src/test-lifecycle.ts @@ -170,6 +170,7 @@ export function wrapBrowserOnce( ctx.isScriptInjected = true } ctx.browserProxy.resetTestTracking() + ctx.sessionCapturer.resetTestCapture() ctx.browserProxy.wrapBrowserCommands(browser) } diff --git a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts index bcf03368..60ae9582 100644 --- a/packages/nightwatch-devtools/tests/actionSnapshot.test.ts +++ b/packages/nightwatch-devtools/tests/actionSnapshot.test.ts @@ -1,7 +1,17 @@ +import type * as DevtoolsCore from '@wdio/devtools-core' import { describe, it, expect, vi } from 'vitest' +import { captureActionSnapshot as coreCapture } from '@wdio/devtools-core' import { SessionCapturer } from '../src/session.js' import type { CommandLog, NightwatchBrowser } from '../src/types.js' +vi.mock('@wdio/devtools-core', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + captureActionSnapshot: vi.fn(actual.captureActionSnapshot) + } +}) + function makeMockBrowser(): NightwatchBrowser { return { url: vi.fn(async () => ({})), @@ -59,6 +69,50 @@ describe('nightwatch action-snapshot timestamp binding', () => { cap.cleanup() }) + it('captures a click, but not a read or wait that follows it', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('click', 1) + await at('getText', 2) + await at('waitForElementVisible', 3) + await Promise.all(cap.snapshotCaptures) + + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + expect(cap.commandsLog).toHaveLength(3) + cap.cleanup() + }) + + it('captures a leading read as the initial frame, once per test', async () => { + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('getText', 1) + await at('getText', 2) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + cap.resetTestCapture() + await at('getText', 3) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 3]) + cap.cleanup() + }) + + it('retries the initial frame when the first read captures nothing', async () => { + vi.mocked(coreCapture).mockResolvedValueOnce(null) + const cap = makeCapturer() + const at = (command: string, timestamp: number) => + cap.captureCommand(command, ['#el'], undefined, undefined, { timestamp }) + await at('getText', 1) + await Promise.all(cap.snapshotCaptures) + await at('getText', 2) + await at('getText', 3) + await Promise.all(cap.snapshotCaptures) + expect(cap.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + cap.cleanup() + }) + it('does not capture outside trace mode', async () => { const cap = new SessionCapturer({}, makeMockBrowser()) await cap.captureCommand('click', [], undefined, undefined, { diff --git a/packages/selenium-devtools/src/helpers/commandPostActions.ts b/packages/selenium-devtools/src/helpers/commandPostActions.ts index 6acbd500..3879fc7a 100644 --- a/packages/selenium-devtools/src/helpers/commandPostActions.ts +++ b/packages/selenium-devtools/src/helpers/commandPostActions.ts @@ -11,6 +11,7 @@ import { type CapturedPerformancePayload, type RetryTracker } from '@wdio/devtools-core' +import { changesPage } from '@wdio/devtools-shared' import { ELEMENT_LOCATOR_METHODS } from '../constants.js' import { getDriverOriginals, getElementOriginals } from '../driverPatcher.js' import { captureOrReplaceCommand } from './captureOrReplaceCommand.js' @@ -357,14 +358,18 @@ function queueActionSnapshot( timestamp: number, error: unknown ): void { - if ( - ctx.options.mode !== 'trace' || - error || - !ctx.driver || - !mapCommandToAction(cmd.command) - ) { + if (ctx.options.mode !== 'trace' || error || !ctx.driver) { return } + // Reads and waits inherit the earlier capture; only the test's first mapped command adds an initial frame. + const initialFrame = + capturer.awaitingInitialFrame && !!mapCommandToAction(cmd.command) + if (!changesPage(cmd.command) && !initialFrame) { + return + } + if (initialFrame) { + capturer.awaitingInitialFrame = false + } ctx.snapshotCaptures.push( captureActionSnapshot( ctx.driver, @@ -374,6 +379,8 @@ function queueActionSnapshot( ).then((snap) => { if (snap) { upsertRichestSnapshot(ctx.actionSnapshots, snap) + } else if (initialFrame && !changesPage(cmd.command)) { + capturer.awaitingInitialFrame = true } }) ) diff --git a/packages/selenium-devtools/src/session.ts b/packages/selenium-devtools/src/session.ts index 6450cce8..349e4144 100644 --- a/packages/selenium-devtools/src/session.ts +++ b/packages/selenium-devtools/src/session.ts @@ -39,6 +39,13 @@ export class SessionCapturer extends SessionCapturerBase { * post-navigation re-injection) unreachable rather than merely redundant. */ preloadRegistered = false + /** True until the current test's first mapped command takes its capture. */ + awaitingInitialFrame = true + + resetTestCapture(): void { + this.awaitingInitialFrame = true + } + #clientConnected = false #clientConnectedWaiters: Array<() => void> = [] #onClientDisconnected?: () => void diff --git a/packages/selenium-devtools/src/test-management.ts b/packages/selenium-devtools/src/test-management.ts index b8b6cd96..4a952863 100644 --- a/packages/selenium-devtools/src/test-management.ts +++ b/packages/selenium-devtools/src/test-management.ts @@ -149,6 +149,7 @@ export function startTest( } ctx.testManager!.startMarkedTest(name, resolvedMeta) ctx.retryTracker.reset() + ctx.sessionCapturer?.resetTestCapture() if (file) { ctx.sessionCapturer?.captureSource(file).catch(() => {}) } @@ -200,6 +201,7 @@ export function startScenario( const file = meta.file ?? root?.file ?? process.cwd() ctx.suiteManager.startScenarioSuite(name, file, meta.callSource, meta.file) ctx.retryTracker.reset() + ctx.sessionCapturer?.resetTestCapture() if (meta.file) { ctx.sessionCapturer?.captureSource(meta.file).catch(() => {}) } diff --git a/packages/selenium-devtools/tests/action-snapshot.test.ts b/packages/selenium-devtools/tests/action-snapshot.test.ts index 7e75bcd8..0b328769 100644 --- a/packages/selenium-devtools/tests/action-snapshot.test.ts +++ b/packages/selenium-devtools/tests/action-snapshot.test.ts @@ -1,3 +1,4 @@ +import type * as DevtoolsCore from '@wdio/devtools-core' import { afterEach, describe, expect, it, vi } from 'vitest' import { resetSignatureCounters } from '../src/helpers/utils.js' @@ -11,10 +12,21 @@ import { } from '../src/helpers/commandPostActions.js' import { captureActionSnapshot } from '../src/action-snapshot.js' import { getDriverOriginals } from '../src/driverPatcher.js' -import { RetryTracker } from '@wdio/devtools-core' +import { + RetryTracker, + captureActionSnapshot as coreCapture +} from '@wdio/devtools-core' import type { ActionSnapshot } from '@wdio/devtools-shared' import type { CapturedCommand, SeleniumDriverLike } from '../src/types.js' +vi.mock('@wdio/devtools-core', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + captureActionSnapshot: vi.fn(actual.captureActionSnapshot) + } +}) + const capturers: SessionCapturer[] = [] afterEach(() => { @@ -115,6 +127,43 @@ describe('selenium action-snapshot timestamp binding', () => { }) }) +describe('selenium action-snapshot capture gate', () => { + async function run(commands: string[], ctxBag = makeCtx(fakeDriver())) { + let t = 1 + for (const command of commands) { + await handleOnCommand(ctxBag.ctx, cmd({ command, timestamp: t++ })) + } + await Promise.all(ctxBag.snapshotCaptures) + return ctxBag + } + + it('captures a click, but not a read or wait that follows it', async () => { + const { actionSnapshots } = await run([ + 'click', + 'getText', + 'waitForDisplayed' + ]) + expect(actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + }) + + it('captures a leading read as the initial frame, once per test', async () => { + const bag = makeCtx(fakeDriver()) + await run(['getText', 'isDisplayed'], bag) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1]) + + bag.capturer.resetTestCapture() + await handleOnCommand(bag.ctx, cmd({ command: 'getText', timestamp: 9 })) + await Promise.all(bag.snapshotCaptures) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([1, 9]) + }) + + it('retries the initial frame when the first read captures nothing', async () => { + vi.mocked(coreCapture).mockResolvedValueOnce(null) + const bag = await run(['getText', 'isDisplayed', 'getText']) + expect(bag.actionSnapshots.map((s) => s.timestamp)).toEqual([2]) + }) +}) + describe('selenium action-snapshot locator dialect', () => { it('injects the XPath text form, which By.xpath resolves', async () => { // `a*=Logout` is WebdriverIO-only syntax; a Selenium user pastes the diff --git a/packages/service/src/index.ts b/packages/service/src/index.ts index 07ee2321..a41bdd25 100644 --- a/packages/service/src/index.ts +++ b/packages/service/src/index.ts @@ -35,6 +35,7 @@ import { captureActionSnapshot, settleAfterLastAction } from './action-snapshot.js' +import { changesPage } from '@wdio/devtools-shared' import type { ActionSnapshot, TestMetadataMap } from '@wdio/devtools-shared' import { SevereServiceError } from 'webdriverio' import type { Services, Capabilities, Options, Reporters } from '@wdio/types' @@ -155,6 +156,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { /** Wall-clock ms at the current test's start, set in beforeTest/beforeScenario; * the lower bound of that test's video frame window (per-test slicing). */ #currentTestStartWallTime = 0 + #awaitingInitialCapture = true /** Map of testUid → metadata for trace group events and per-spec partitioning. */ #testMetadata: TestMetadataMap = new Map() @@ -374,6 +376,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { }) { this.resetStack() this.#currentTestStartWallTime = Date.now() + this.#awaitingInitialCapture = true this.#currentStepIndex = 0 this.#currentStepUid = undefined @@ -407,6 +410,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { beforeTest(test?: { file?: string; title?: string; fullTitle?: string }) { this.resetStack() this.#currentTestStartWallTime = Date.now() + this.#awaitingInitialCapture = true // Track test identity for command tagging. Generate a stable UID // from file + title so commands can be partitioned across reruns. @@ -575,7 +579,11 @@ export default class DevToolsHookService implements Services.ServiceInstance { // action's has no successor and this is the only capture of it. A session // that ran no action names it FINAL_SNAPSHOT_COMMAND, which the per-test // screenshot reads as "post-teardown frame, do not use". - const lastAction = this.#lastAction() + const lastChange = this.#lastCommandWhere(changesPage) + const lastAction = + lastChange && lastChange.timestamp >= this.#currentTestStartWallTime + ? lastChange + : this.#lastAction() // A session with no action has no timestamp of its own to key on, so its // frame is recognised by the marker instead — otherwise `Date.now()` differs // between the per-test finalize and `after()` and both capture. @@ -632,20 +640,22 @@ export default class DevToolsHookService implements Services.ServiceInstance { * self-scopes, and a slot already filled at its own boundary converges on * `alreadyCaptured`. */ #lastAction(): { command: string; timestamp: number } | undefined { + return this.#lastCommandWhere(() => true) + } + + #lastCommandWhere( + predicate: (command: string) => boolean + ): { command: string; timestamp: number } | undefined { const commands = this.#sessionCapturer.commandsLog for (let i = commands.length - 1; i >= 0; i--) { const cmd = commands[i]! - if (this.#isActionCommand(cmd.command)) { + if (this.#isActionCommand(cmd.command) && predicate(cmd.command)) { return cmd } } return undefined } - #lastActionTimestamp(): number { - return this.#lastAction()?.timestamp ?? Date.now() - } - private resetStack() { this.#commandStack = [] // No command is in flight after a reset; without this a window stranded by a @@ -720,8 +730,11 @@ export default class DevToolsHookService implements Services.ServiceInstance { // the previous action left behind. Taken HERE, before the command is issued, // because that is the one moment the driver is guaranteed idle and the app // at rest — a capture taken the instant a command returns catches whatever - // transition it started. Stamped at the previous action's end (or now, for - // the first, which makes it the initial frame). Trace mode only. + // transition it started. Stamped at the previous page-changing command's + // end (or now, for the first, which makes it the initial frame). Trace mode + // only. + // + // Reads/waits inherit the last page-changing capture; capturing them costs a page-source round trip. // // Not while a mobile-web Appium session has a document to probe — the // re-entrant probes can deadlock a serialising driver (#374). A native @@ -734,16 +747,18 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#browser, this.#sessionCapturer.currentContext ) && - this.#isActionCommand(command) + this.#isActionCommand(command) && + (changesPage(command) || this.#awaitingInitialCapture) ) { - // Stamped at the previous action's end so this capture IS that action's + // Stamped at the previous page-changing command's end so this capture IS that command's // result — except across a test boundary, where that slot already holds // its own finalize capture and a second frame at the same timestamp lets // the richer-screenshot merge replace it (with a reloadSession between // the tests, the row then replays the post-reload page). The first // capture of a test stamps now instead, the same rule the session's first // capture uses to become the initial frame. - const previousEnd = this.#lastActionTimestamp() + const previousEnd = + this.#lastCommandWhere(changesPage)?.timestamp ?? Date.now() const snap = await captureActionSnapshot( this.#browser, command, @@ -753,6 +768,7 @@ export default class DevToolsHookService implements Services.ServiceInstance { this.#sessionCapturer.currentContext ) if (snap) { + this.#awaitingInitialCapture = false upsertRichestSnapshot(this.#actionSnapshots, snap) } } diff --git a/packages/service/tests/trace-action-capture.test.ts b/packages/service/tests/trace-action-capture.test.ts index 16dcbfed..ae26c2f5 100644 --- a/packages/service/tests/trace-action-capture.test.ts +++ b/packages/service/tests/trace-action-capture.test.ts @@ -16,7 +16,7 @@ vi.mock('stack-trace', () => ({ /** * The command has to be in the log by the time `afterCommand` returns: - * `#lastActionTimestamp()` reads it to stamp the NEXT action's capture, and that + * `#lastPageChange()` reads it to stamp the NEXT action's capture, and that * stamp is what makes an action's result the following action's "before". */ const commandsLog: { command: string; timestamp: number }[] = [] @@ -199,6 +199,65 @@ describe('trace mode: one capture per action, taken before it', () => { expect(stampedAt(1)).toBe(commandsLog[0]!.timestamp) }) + describe('reads and waits do not capture', () => { + const run = async ( + service: DevToolsHookService, + commands: string[] + ): Promise => { + for (const command of commands) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + } + const started = async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + vi.mocked(captureActionSnapshot).mockClear() + return service + } + + it('captures only the page-changing commands of click, getText x3, click', async () => { + const service = await started() + await run(service, ['click', 'getText', 'getText', 'getText', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(namedAt(0)).toBe('click') + expect(namedAt(1)).toBe('click') + }) + + it('stamps the capture after a wait at the preceding click end', async () => { + const service = await started() + await run(service, ['click', 'waitForDisplayed', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(stampedAt(1)).toBe(commandsLog[0]!.timestamp) + }) + + it('takes an initial capture on a leading read, then one on the click', async () => { + const service = await started() + await run(service, ['getText', 'click']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + expect(namedAt(0)).toBe('getText') + expect(namedAt(1)).toBe('click') + }) + + it('retries the initial capture when the first read captures nothing', async () => { + const service = await started() + vi.mocked(captureActionSnapshot).mockResolvedValueOnce(null) + await run(service, ['getText', 'getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + + it('takes the initial capture again for a new test', async () => { + const service = await started() + service.beforeTest({ file: '/spec/a.ts', title: 'one' } as never) + await run(service, ['getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(1) + service.beforeTest({ file: '/spec/a.ts', title: 'two' } as never) + await run(service, ['getText', 'getText']) + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(2) + }) + }) + it('captures the final state when the session ends', async () => { const browser = nativeBrowser() const service = new DevToolsHookService({ mode: 'trace' }) @@ -313,6 +372,51 @@ describe('trace mode: one capture per action, taken before it', () => { expect(stampedAt(0)).toBe(commandsLog[0]!.timestamp) }) + it('stamps the final capture at the test last page change, past later reads', async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + const test = { file: '/spec/a.ts', title: 'submit' } + service.beforeTest(test as never) + clock = Date.now() + 1000 + for (const command of ['click', 'getText', 'getText']) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + + vi.mocked(captureActionSnapshot).mockClear() + await service.afterTest(test as never, {} as never, {} as never) + + expect(vi.mocked(captureActionSnapshot)).toHaveBeenCalledTimes(1) + expect(namedAt(0)).toBe('click') + expect(stampedAt(0)).toBe(commandsLog[0]!.timestamp) + }) + + it('stamps the final capture at the last read when the test changed no page', async () => { + const browser = nativeBrowser() + const service = new DevToolsHookService({ mode: 'trace' }) + await service.before({} as never, [], browser) + const first = { file: '/spec/a.ts', title: 'first' } + service.beforeTest(first as never) + await service.beforeCommand('click' as never, []) + await service.afterCommand('click' as never, [], undefined) + await service.afterTest(first as never, {} as never, {} as never) + + const second = { file: '/spec/a.ts', title: 'second' } + service.beforeTest(second as never) + clock = Date.now() + 1000 + for (const command of ['getText', 'getText']) { + await service.beforeCommand(command as never, []) + await service.afterCommand(command as never, [], undefined) + } + + vi.mocked(captureActionSnapshot).mockClear() + await service.afterTest(second as never, {} as never, {} as never) + + expect(namedAt(0)).toBe('getText') + expect(stampedAt(0)).toBe(commandsLog.at(-1)!.timestamp) + }) + it('does not merge the next test first capture into the previous test slot', async () => { const browser = nativeBrowser() const service = new DevToolsHookService({ mode: 'trace' }) diff --git a/packages/shared/src/trace-actions.ts b/packages/shared/src/trace-actions.ts index a79fcb48..4bf09158 100644 --- a/packages/shared/src/trace-actions.ts +++ b/packages/shared/src/trace-actions.ts @@ -42,7 +42,58 @@ export function mapAssertCommand(command: string): TraceAction | null { return match ? { class: ASSERT_ACTION_CLASS, method: match[1] } : null } -export const ACTION_MAP: Record = { +export type ActionEffect = 'changes' | 'reads' + +export const METHOD_EFFECT = { + navigate: 'changes', + goBack: 'changes', + goForward: 'changes', + reload: 'changes', + goto: 'changes', + click: 'changes', + dblclick: 'changes', + fill: 'changes', + selectOption: 'changes', + hover: 'changes', + scrollIntoViewIfNeeded: 'changes', + dragTo: 'changes', + clear: 'changes', + submit: 'changes', + press: 'changes', + evaluate: 'changes', + tap: 'changes', + switchToWindow: 'changes', + getText: 'reads', + getValue: 'reads', + getAttribute: 'reads', + getProperty: 'reads', + getCSSProperty: 'reads', + getTagName: 'reads', + getLocation: 'reads', + getSize: 'reads', + getRect: 'reads', + isDisplayed: 'reads', + isExisting: 'reads', + isEnabled: 'reads', + isSelected: 'reads', + isClickable: 'reads', + isFocused: 'reads', + waitForDisplayed: 'reads', + waitForHidden: 'reads', + waitForExist: 'reads', + waitForNotExist: 'reads', + waitForEnabled: 'reads', + waitForClickable: 'reads', + waitForFunction: 'reads', + getTitle: 'reads', + getUrl: 'reads', + getPageSource: 'reads' +} as const satisfies Record + +export const ACTION_MAP: Record< + string, + TraceAction & { method: keyof typeof METHOD_EFFECT } +> = { // WDIO browser-level url: { class: 'Page', method: 'navigate' }, navigateTo: { class: 'Page', method: 'navigate' }, @@ -208,3 +259,12 @@ export function isInputDispatchingCommand(command: string): boolean { const action = ACTION_MAP[command] return action !== undefined && POINTABLE_METHODS.has(action.method) } + +/** Whether a command can alter what the page shows; reads and waits don't + * warrant their own capture (cost) and inherit the latest earlier one via `claimAfter`. */ +export function changesPage(command: string): boolean { + if (!Object.prototype.hasOwnProperty.call(ACTION_MAP, command)) { + return false + } + return METHOD_EFFECT[ACTION_MAP[command].method] === 'changes' +} diff --git a/packages/shared/tests/change-page.test.ts b/packages/shared/tests/change-page.test.ts new file mode 100644 index 00000000..611b7dde --- /dev/null +++ b/packages/shared/tests/change-page.test.ts @@ -0,0 +1,37 @@ +import { describe, it, expect } from 'vitest' +import { ACTION_MAP, METHOD_EFFECT, changesPage } from '@wdio/devtools-shared' + +describe('changesPage', () => { + it.each([ + 'click', + 'setValue', + 'url', + 'execute', + 'keys', + 'sendKeys', + 'switchToWindow' + ])('is true for %s', (command) => { + expect(changesPage(command)).toBe(true) + }) + + it.each([ + 'getText', + 'isDisplayed', + 'waitForDisplayed', + 'waitUntil', + 'getTitle', + 'waitForElementVisible', + 'expect.toHaveText', + 'assert.equal', + 'findElement', + 'constructor' + ])('is false for %s', (command) => { + expect(changesPage(command)).toBe(false) + }) + + it('classifies the method of every ACTION_MAP entry', () => { + for (const { method } of Object.values(ACTION_MAP)) { + expect(Object.keys(METHOD_EFFECT)).toContain(method) + } + }) +}) diff --git a/packages/trace/tests/trace-frame-snapshots.test.ts b/packages/trace/tests/trace-frame-snapshots.test.ts index 26a87eac..5171bfab 100644 --- a/packages/trace/tests/trace-frame-snapshots.test.ts +++ b/packages/trace/tests/trace-frame-snapshots.test.ts @@ -54,6 +54,25 @@ describe('FrameSnapshotIndex', () => { expect(index.refs()).toEqual([]) }) + it('resolves every read row to the capture of the latest earlier page change', () => { + const index = new FrameSnapshotIndex([ + snap({ timestamp: 100, command: 'click', screenshot: 'A1' }), + snap({ timestamp: 200, command: 'click', screenshot: 'B2' }) + ]) + expect(index.claimAfter(50, 'read@0')).toBeUndefined() + const reads: [number, string][] = [ + [150, 'A1'], + [160, 'A1'], + [250, 'B2'] + ] + for (const [ts, shot] of reads) { + const id = `read@${ts}` + expect(index.claimAfter(ts, id)).toBe(`after@${id}`) + const ref = index.refs().find((r) => r.callId === id)! + expect(ref.snapshot.screenshot).toBe(shot) + } + }) + it('ignores snapshots without a screenshot', () => { const index = new FrameSnapshotIndex([snap({ screenshot: undefined })]) expect(index.claimAfter(2000, 'call@2')).toBeUndefined()