Skip to content

fix(ios): verify deep-tree back on the sole screen - #3894

Merged
thymikee merged 3 commits into
mainfrom
perf/ios-back-deep-tree
Oct 22, 2026
Merged

thymikee merged 3 commits into
mainfrom
perf/ios-back-deep-tree

Conversation

@thymikee

@thymikee thymikee commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Summary

  • In-app back now recovers after the XCTest query rejection that deep trees cause on single-screen iPhones. It reuses the sole-screen synthesized dispatch and the before/after visual check. Healthy control selection and the fix(ios): use the resolved display for screenshots and visual verification #2728 outcomes are unchanged.
  • The top-leading fallback taps one point, 0.1·h (56–96), on both the window route and the sole-screen route. The health of the accessibility tree picks the route and the capture, not the target. A rejected query skips the windows query only where the sole screen answers; an iPad still asks for its windows.
  • 9 files: runner, display capture, regression tests, help, and the panel ADR.

Validation

Head 2864217fbe (main e3ed027a1d merged).

  • On an iPhone 17 Pro (402x874), both the heavy-feed header and the automation-lab header put the back control at y 62–106. A tap at y 132 missed both; a tap at y 87 hit. Live heavy-feed back succeeded 3/3 (1127–1748 ms). With the heavy feed as the root screen, back reported changed=no, so back was unavailable. On an iPad deep tree the outcome is the same as base (unavailable). Details: fix(ios): verify deep-tree back on the sole screen #3894 (comment)
  • Earlier A/B on the Release heavy-feed fixture, base 9935ef51c2 vs db9c6a836a: 0/3 → 3/3, median 903 ms.
  • Runner Navigation tests: 18/18 on iPhone and on iPad. Mutations: an always-changed sole-screen capture fails only the new miss test; skipping windows on an iPad fails the new window-route test. Combined with fix(ios): synthesize long presses and the gesture viewport on deep trees #3893: 54/54 on both.
  • check:xctest-selection passed. check:affected --run --head 2864217fbe passed (exactCommit: true).

Known gap: on an iPad the bar sits at y 32–86, and the fallback point (y 96) misses it on both routes. This PR does not change that.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-22 11:10 UTC

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.53 MB 5.54 MB +2.2 kB
Package (unpacked) 5.53 MB 5.54 MB +2.2 kB
Package (download) 1.68 MB 1.68 MB +388 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 39.4 ms 40.8 ms +1.4 ms
CLI --help 80.9 ms 82.2 ms +1.3 ms

@thymikee

Copy link
Copy Markdown
Member Author

I reviewed db9c6a8. I found no blocking problems in the code, but I did not rerun the heavy-feed live A/B or the native runner suite, so the live numbers come from the PR body alone. Cubic and Copilot have not reviewed this head yet.

Lint, Typecheck & Package fails at packages/platform-apple/src/snapshot-route.test.ts:1003 (max-lines). This PR does not touch that file, the failure reproduces on base, and #3890 owns the fix. The check stays red until #3890 lands on main and this branch rebases. There are no conflicts.

Not blocking, and you can take or leave these: (1) topLeadingNavigationFallbackPoint in RunnerTests+Navigation.swift:182 picks the y offset from windowResolved (0.155h clamped 56..132 versus 0.1h clamped 56..96). On a one-screen iPhone the resolved window frame and the sole-screen frame are the same rect, so the tap point for the same header depends on whether XCTest could read the tree. A deep-tree RN app with the tall search header would now be tapped at about 87 pt. (2) windowQueryRejected at RunnerTests+Navigation.swift:76 skips resolveRunnerWindowElement, but on an iPad soleScreenReferenceFrame returns nil, so back returns .unavailable. The base tried app.windows there, and I could not tell whether it still resolves on an iPad deep tree when navigationBars.buttons is rejected. (3) RunnerTests+NavigationTests.swift:37 drives the counter with a hand-written rejection issue, and no test covers a tap that lands on nothing on the sole-screen route, so an always-changed capture would still pass.

Simplicity question: could you keep the capture choice keyed on context.resolvedWindow as it is, but use one geometry-derived fallback point, so AX health picks the route but never the target? Could windowQueryRejected also apply only where soleScreenReferenceFrame can answer?

The next step is your answer on the y-offset in (1).

thymikee and others added 2 commits October 22, 2026 10:55
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AX health picks the route but no longer the target: the fallback taps
0.1h (56-96) on a resolved window and on the sole screen alike, the
leading control of a standard bar (y 62-106 on a 402x874 iPhone).
A rejected query skips the windows query only where the sole screen
answers, so an iPad still asks for its windows as before.

Tests cover a sole-screen tap that changes nothing and the iPad
windows route after a rejected query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Thanks. Answers at 2864217fbe: main merged (e3ed027a1d), plus one commit for these points.

(1) The y offset: use one point, 0.1·h clamped to 56–96

I measured on my own iPhone 17 Pro simulator (402x874, iOS 27.0) with the Release test app:

header bar back control tap at (32, 132) tap at (32, 87)
Heavy feed (native-stack header over an unreadable tree) y 62–116 Automation y 62–106 lands on the RN title Heavy feed (y 124–148); screen unchanged goes back to Automation
Automation lab (normal RN native-stack header, healthy tree) y 62–116 back y 62–106 below the bar, in the content inside the back control

The 0.155·h value (132 pt here) came from #1158 ("common custom RN search headers") with no recorded measurement. It misses both headers. The test app has no custom RN top-leading back: the product screen's back is a body button at y 418. So I found no header where 132 was right. I applied your simplification. The health of the accessibility tree now picks the route and the capture (context.resolvedWindow), but never the target. Both routes tap 0.1·h (87.4 pt on 402x874).

Known gap, unchanged by this PR: on an iPad (834x1210) the bar is y 32–86 and the back control y 32–76, so both 96 and 132 miss.

(2) windowQueryRejected applies only where the sole screen answers

Applied. After a rejected query, synthesizedCoordinateContext skips the windows query only when soleScreenReferenceFrame resolves. An iPad asks for its windows as base did. Live on my iPad Pro 11 (M5) heavy feed, pushed over the automation lab:

  • Base back path: in-app back control is not available after about 6 s, with 6 rejected queries including app.windows.
  • This head: the same outcome in 1.3 s. The windows query is still asked and rejected (AX_SNAPSHOT_ISSUE_SUPPRESSED count=2).

So on an iPad deep tree, app.windows does not resolve either. The new test testInAppBackStillAsksForWindowsWhereNoSoleScreenAnswers rejects one query and then lets XCTest answer. On iPad, the window route then carries the tap. With the previous head's rule planted (skip windows on any rejection), it fails on iPad.

(3) A tap that lands on nothing

Added testInAppBackOnTheSoleScreenReportsATapThatChangedNothingAsUnavailable. It launches the runner app without a back control, rejects the tree, and expects .unavailable from the sole-screen tap at (32.2, 87.4). I ran 2 mutations in a disposable worktree on iPhone:

  • The sole-screen capture returns a new frame on every call (an always-changed capture): only this test fails (1 failed, 17 passed).
  • The observation always returns .changed: this test fails, along with the existing pure tests.

The existing sole-screen test now states the outcome off a single-screen iPhone (.unavailable).

Live at this head (iPhone 17 Pro, heavy feed)

  • back: 3/3 returned to Automation, in 1748, 1276 and 1127 ms:
    AGENT_DEVICE_RUNNER_IN_APP_BACK_SKIPPED_XCTEST_ENUMERATION bundle=com.callstack.agentdevicelab
    AGENT_DEVICE_RUNNER_SYNTHESIZED_SOLE_SCREEN_ROUTE resolved={{0, 0}, {402, 874}}
    AGENT_DEVICE_RUNNER_SYNTHESIZED_DISPATCH kind=tap point=(32.2,87.4) reference=(0.0,0.0,402.0,874.0) orientation=1
    AGENT_DEVICE_RUNNER_IN_APP_BACK_VISUAL_VERIFICATION beforeBytes=210375 afterBytes=313292 changed=yes
    
  • Live miss, with the heavy feed as the root screen: changed=no (191809/191809 bytes), so in-app back control is not available.

Validation

@thymikee

Copy link
Copy Markdown
Member Author

This PR is ready at 2864217. The notes from the earlier review (#3894 (comment)) are addressed, and I found no new problems. All 19 checks pass at this commit, and the earlier max-lines failure came from the base and cleared with the main merge. I know of no conflicts, and nothing else needs to happen before merge.

Not blocking, take or leave: testInAppBackStillAsksForWindowsWhereNoSoleScreenAnswers (apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+NavigationTests.swift:56) rejects one query, but on a single-screen iPhone soleScreenReferenceFrame answers, so the tap takes the sole-screen route either way. The test tells the fix apart only on iPad, so a doc comment naming that, or an XCTSkipUnless when soleScreenReferenceFrame answers, would stop an iPhone pass from reading as coverage.

I did not rerun the runner Navigation suite, the two mutations, or the heavy-feed live back. Those results come from the author's comment 6337876717. The change also lowers the resolved-window fallback height from 0.155h (132 pt cap, as on main) to 0.1h. The measurements cover a healthy native-stack header, not a back run through the window-route fallback, and no app with a tall custom RN search header was tried. The visual verification still guards a miss on that route. On an iPad the fallback point (y 96) still misses the bar (y 32-86), which the author notes is an older gap.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 22, 2026
@thymikee
thymikee merged commit 4254083 into main Oct 22, 2026
19 checks passed
@thymikee
thymikee deleted the perf/ios-back-deep-tree branch October 22, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant