Native E2E (iOS / Android)
The native lane runs in EAS Workflows (PLAN.md decision 1): fingerprint → get-build / build → repack → maestro → update → approval → submit. Base builds come from EAS Build keyed by @expo/fingerprint (decision 2), and every PR reuses one by injecting a fresh JS bundle with @expo/repack-app instead of rebuilding natively. This page covers the local twins of those jobs; the workflow itself, the Maestro flow layout and the merge gate are filled in by E4 / E8.
Local reproduce
When the e2e workflow goes red, run the same steps on a laptop. Each script is plain Node in scripts/, takes --help, and fails fast with an install hint when a tool is missing. Nothing here runs Metro against a dev server: the app under test is the release build EAS produced, with the JS bundle from your working tree.
| EAS Workflows job | Local script | What it does |
|---|---|---|
fingerprint | bun run fingerprint [--platform <p>] | Native fingerprint hash of the tree (scripts/fingerprint.js). The other scripts compute it themselves; run this to compare with the workflow's value. |
get-build / build | bun run e2e:build [--platform <p>] | Finds a finished EAS build of the E2E profile (e2e-ios-sim / e2e-android-apk in eas.json) whose fingerprint matches, downloads it to e2e/builds/<p>/base.(app|apk) and records it in base.json. On a miss it prints the eas build command and exits 2; pass --build to run that (paid) build and wait for it. --build-id <id> downloads a specific build. |
repack | bun run e2e:repack [--platform <p>] | @expo/repack-app: runs expo export:embed for the current tree (Hermes bytecode, APP_VARIANT=development like the base build) and injects it into base.(app|apk), writing e2e/builds/<p>/repacked.(app|apk). No native rebuild. |
maestro | bun run e2e:ios / bun run e2e:android | Boots a simulator / emulator, installs repacked.* (or base.* if you skipped repack) and runs maestro test .maestro --include-tags <p> -e MAESTRO_APP_ID=<bundle id | package> with JUnit output in maestro-<p>/ (same layout as bun run e2e:web), Maestro's debug output in maestro-<p>/debug/ and, after a failure, the simulator log / logcat in maestro-<p>/device/ (Failure artifacts). |
--platform defaults to ios. e2e/builds/, maestro-ios/ and maestro-android/ are git-ignored.
Typical loop after a red maestro job:
bun run e2e:build --platform ios # once per fingerprint; exits 2 with the eas build command on a miss
bun run e2e:repack --platform ios # every time the JS changes
bun run e2e:ios # add --keep to leave the simulator running for a lookPrerequisites
bun run doctor checks all of these (versions + install hints, docs/doctor.md).
| Tool | Needed by | Notes |
|---|---|---|
EAS login (bun run eas login) | e2e:build | Repo-pinned eas-cli; EXPO_TOKEN works too. eas build:download caches the artifact in its own directory; the script copies it under e2e/builds/. |
| Xcode + command line tools | e2e:repack (ios), e2e:ios | xcrun simctl picks an already-booted iPhone, else the newest available one (--device <udid|name> to override). The simulator .app is unsigned: repack strips the old signature and nothing needs re-signing. |
JDK (java) + Android build-tools | e2e:repack (android) | @expo/repack-app ships apktool.jar and a debug keystore; it needs java on PATH and aapt2 / zipalign / apksigner under $ANDROID_SDK_ROOT/build-tools (falls back to $ANDROID_HOME). The APK is debug-signed (android), which emulators accept. |
adb (+ emulator or a running AVD) | e2e:android | Uses the first online adb device; otherwise starts the first AVD from emulator -list-avds and waits for sys.boot_completed. --device <serial> to pin one. |
| Maestro CLI | e2e:ios, e2e:android | curl -Ls "https://get.maestro.mobile.dev" | bash; resolved from PATH or ~/.maestro/bin. CI pins 2.10.0 — use the same locally, 2.3.0 has known selector bugs (see .maestro/subflows/select-tab.yaml). |
Differences from the workflow, on purpose: the workflow's repack job signs with the project's EAS credentials so the artifact can also ship as an internal build; locally the simulator .app stays unsigned and the APK is debug-signed. The workflow's maestro job also runs on a fresh EAS device; locally the app is installed over whatever is already on the simulator (simctl install / adb install -r replace it in place).
Workflow (.eas/workflows/e2e.yml)
E2E (native) runs on every pull request into main (on: pull_request), when a PR is labelled e2e:ios (on: pull_request_labeled), and by hand (workflow_dispatch). A push to main trigger exists but is switched off (if: false) unless the project runs iOS in main-only tier — see Tiered mode; the staging OTA workflow (E5) covers main otherwise. A new push to the same branch cancels the run in flight (concurrency.cancel_in_progress).
fingerprint ─┬─ get_build_ios ─────┬─ repack_ios (hit: reuse base build, inject this JS)
│ └─ build_ios (miss: full EAS build, paid)
│ └──────────────── maestro_ios
└─ get_build_android ─┬─ repack_android
└─ build_android
└──────────────── maestro_android
└── comment (PR only, runs after everything)| Job | Type | Inputs | Outputs used downstream |
|---|---|---|---|
fingerprint | fingerprint | environment: development, env.APP_VARIANT=development (must equal the E2E build profiles, or the hash never matches) | ios_fingerprint_hash, android_fingerprint_hash |
get_build_<p> | get-build | platform, profile: e2e-ios-sim | e2e-android-apk, simulator: true (ios), fingerprint_hash, wait_for_in_progress; iOS only: the IOS_MODE if: (Tiered mode) — skipping it skips the whole iOS chain | build_id (empty on a miss) |
build_<p> | build | if: !get_build.build_id; platform, profile (same E2E profile) | build_id |
repack_<p> | repack | if: get_build.build_id; build_id of the cached base build, profile (same E2E profile) | build_id (the repacked build) |
maestro_<p> | maestro | after: [repack, build]; build_id: repack || build, flow_path: .maestro, include_tags: [<p>], exclude_tags: [quarantine], maestro_version: 2.10.0, shards: 2, retries: 2, retry_failed_only: true, record_screen: true, output_format: junit, env.MAESTRO_APP_ID; hooks.after_maestro_tests collects + uploads device logs, then runs the informational a11y audit (scripts/a11y-audit.js --no-fail, ADR-0005); Android only: then Flashlight (scripts/flashlight.js --no-fail, off behind the FLASHLIGHT constant, ADR-0007) | Run artifacts: Maestro Test Results (recordings, JUnit, Maestro debug output), Device logs (<p>), A11y audit (<p>) and Flashlight (android) (Failure artifacts) |
comment | github-comment | after: every job above; if: github.event_name == 'pull_request'; params.payload (custom markdown built from after.<job>.status / .outputs) | comment_url (unused) |
How the cache works: get-build asks EAS for a finished build of the E2E profile whose fingerprint equals this commit's. JS-only PRs hit (fingerprint unchanged since the last native build), so the run is repack + maestro only — a few minutes and no build minutes. A PR that touches the native surface (a config plugin, a native dependency, app.config.ts identifiers) misses; build_<p> then produces a fresh base for that fingerprint, which the same PR's later pushes and every later PR with that fingerprint reuse. A fresh build already embeds the commit's JS, so it is not repacked; maestro takes whichever build_id exists (Expo's documented fingerprint + repack idiom). Two PRs racing on the same new fingerprint do not both build: wait_for_in_progress makes the second wait for the first's build.
maestro_<p> depends on repack_<p> / build_<p> through after:, not needs: — exactly one of the two runs, and needs would skip the job along with the skipped one. A dependency declared with after: is readable only through the after.<job> context, so both the if: and params.build_id read after.repack_<p>.outputs.build_id || after.build_<p>.outputs.build_id (workflow syntax); the same rule governs deploy-staging.yml and promote.yml. Reading needs.<job> for an after: dependency silently resolves to nothing, which makes the if: falsy and skips Maestro entirely — that bug shipped in this file (T10.1) and was only caught by reading, not by a run.
Unverified. The native lane has not yet had a real end-to-end run on EAS; it has passed eas workflow:validate only. It is blocked on Owner checklist → Link the Expo GitHub App; the first PR that triggers e2e.yml is still owed — confirm that both maestro_<p> jobs actually start, that the build id resolves, and that the PR comment shows a flow count rather than ⏭️ skipped, then update First real runs still owed.
Maestro on EAS: flow_path: .maestro is the workspace directory, so config.yaml is read and include_tags selects the native entries exactly like bun run e2e:<p>. The app id reaches the flows as MAESTRO_APP_ID — Maestro exposes MAESTRO_* shell variables to flows, which is the only way a pre-packaged job can pass a value; the local scripts set the same name with -e. The value is spelled out in the workflow because the job cannot run expo config; bun run init (T7.1) rewrites it with the identifiers in app.config.ts. Sharding (shards: 2, 4 flows) is experimental on EAS — set it to 1 first if a run misbehaves. retries: 2 re-runs only the failed flows and exclude_tags: [quarantine] keeps quarantined flows out of the gate — the policy is the Flake budget. Recordings, screenshots and the JUnit report are in the Maestro Test Results artifact on the run page; an after_maestro_tests hook adds the simulator log / logcat as Device logs (<p>) — see Failure artifacts for what each contains and how to pull it.
PR comment
The last job, comment (type: github-comment), posts the run's outcome on the pull request (PLAN.md decision 12: install links are shared by the slack and github-comment jobs). It is wired with after: on every other job — needs would skip it as soon as anything failed, and half of the jobs are skipped by design (build_<p> xor repack_<p>) — so it posts whatever happened. after.<job>.status is success | failure | skipped, and the payload turns that into the ✅ / ❌ / ⏭️ per platform; an iOS row reads ⏭️ skipped (ios_mode; …) when the tier, not a failure, skipped the lane. The job is skipped on workflow_dispatch and push (there is no PR to post to; without the if it would fail the run).
What the comment contains, and where each value comes from:
| Line | Source |
|---|---|
| Maestro result | after.maestro_<p>.status, plus successful_flows_count / total_flows_count and failed_flow_names_json from the maestro job outputs. |
| Build | "cached base + repack" when get_build_<p> returned a build_id, else "fresh build". The link goes to the build page of the build Maestro actually ran (repack_<p>.build_id || build_<p>.build_id), https://expo.dev/accounts/<account>/projects/<slug>/builds/<id>. No context exposes the Expo account or slug, so the prefix is spelled out in the YAML. |
| Fingerprint | after.fingerprint.outputs.<p>_fingerprint_hash, shortened to 12 characters. |
| Install / QR | The build page. The Android APK is an internal-distribution build, so its page hosts the install link and QR code; neither build nor get-build exposes an install URL output. The iOS build is a simulator .app — no QR, download it from the page or with bun run e2e:build --platform ios --build-id <id>. |
| Recordings / JUnit / device logs | ${{ workflow.url }} → the run page → Maestro Test Results and Device logs (<p>) artifacts; the triage walkthrough is Failure artifacts. |
| Reproduce locally | The three-script loop from Local reproduce. |
The job's payload mode is fully custom markdown (it cannot be combined with message / build_ids; the default mode renders EAS's own builds table instead). The job has no update-in-place option, so every run adds a new comment rather than editing the previous one. Posting needs the Expo GitHub App linked to the repository (prerequisite 1 below) — the comment is written by the app, no GitHub token is involved. Expression gotcha for anyone editing the payload: the evaluator resolves both branches of a a ? b : c, so string functions must be guarded (substring(x || '', 0, 8)), otherwise a skipped job's missing output throws and the comment job fails.
Example (JS-only PR on iOS, native change on Android, one Android flow failed):
## E2E (native) · `0123456`
| Platform | Maestro | Build | Fingerprint |
| --------------- | --------------------- | ------------------------------------ | -------------- |
| iOS (simulator) | ✅ passed (4/4 flows) | cached base + repack — [bbbbbbbb](…) | `abcdef012345` |
| Android (APK) | ❌ failed (3/4 flows) | fresh build — [cccccccc](…) | `fedcba987654` |
- Android failed flows: `["native/tabs"]`
- **Install**: the Android build page hosts the install link + QR code (internal distribution). The iOS build is a simulator `.app` with no QR code: download it from its page or `bun run e2e:build --platform ios --build-id <id>`.
- **Recordings / JUnit / device logs**: [workflow run](…) → artifacts → **Maestro Test Results** + **Device logs (ios | android)**. Triage: docs/native-e2e.md → Failure artifacts.
- **Reproduce locally**: `bun run e2e:build --platform <p> && bun run e2e:repack --platform <p> && bun run e2e:<p>` (docs/native-e2e.md).Run it by hand: bun run eas workflow:run .eas/workflows/e2e.yml (or expo.dev → project → Workflows → E2E (native) → Run); the only input is ios_mode, described next. Validate after editing: bun run eas workflow:validate .eas/workflows/e2e.yml — EAS caps a workflow file at 16 KiB, so long explanations belong on this page, not in the YAML.
Tiered mode
Both platforms run on every PR by default (PLAN.md decision 14). iOS is the expensive lane — a macOS worker for every repack and Maestro run, ~15 min of macOS build time on a fingerprint miss — so a project can dial it down to one of three tiers. Android always runs: it is the cheap signal and catches most JS regressions on its own.
| Event | always (default) | main-only | label |
|---|---|---|---|
| PR opened / synchronize | iOS ✅ Android ✅ | iOS ⏭️ Android ✅ | iOS ⏭️ Android ✅ |
PR carries the e2e:ios label | iOS ✅ Android ✅ | iOS ✅ Android ✅ | iOS ✅ Android ✅ |
e2e:ios label added (labeled) | iOS ✅ Android ✅ | iOS ✅ Android ✅ | iOS ✅ Android ✅ |
Push to main | no run | iOS ✅ Android ✅ | no run |
workflow_dispatch (input as set) | iOS ✅ Android ✅ | iOS ✅ Android ✅ | iOS ⏭️ Android ✅ |
"PR carries the label" means every later push to a labelled PR keeps running iOS; the labeled row is the extra run EAS starts the moment the label lands (it cancels the run in flight for that branch and re-runs both platforms). The label is an escape hatch in every tier: in main-only it is how a reviewer asks for iOS on one risky PR without touching the file. In always it is redundant.
How it is wired. EAS workflows have no top-level env, and inputs.* are empty on any run that is not a workflow_dispatch, so the tier is a literal in the YAML — a repo-level constant a PR author cannot override from the outside — with the dispatch input as the per-run override. The three places, all marked IOS_MODE n/3 in .eas/workflows/e2e.yml:
get_build_ios.if—(inputs.ios_mode || 'always'), twice: the'always'fallback is the constant. The expression ismode == always || (mode == main-only && event != pull_request) || PR has the e2e:ios label. Skippingget_build_iosskipsbuild_ios/repack_iosthroughneeds, andmaestro_iosthrough its empty-build_idguard;commentstill posts.on.push.if—falseforalwaysandlabel,trueformain-only(the post-merge run is the only place iOS runs in that tier; in the other tiers it would repeat what the PR run just proved). Cost oftrue: one run per merge — repack + Maestro per platform on a JS-only merge, a full build per platform on a fingerprint change.on.workflow_dispatch.inputs.ios_mode.default— the value the dispatch form pre-fills. Keep it equal to the constant so a plain "Run" behaves like a PR would.
To change the tier, edit all three (grep IOS_MODE) and re-run workflow:validate.
Forcing a full run. Add the e2e:ios label to the PR (create the label once in the repo: gh label create e2e:ios -c 5319e7 -d "Run the iOS Maestro lane on this PR"), or run bun run eas workflow:run .eas/workflows/e2e.yml -F ios_mode=always from the branch (a dispatch run has no PR, so no comment is posted — read the run page instead). Dispatching with -F ios_mode=label is the inverse: an Android-only run.
Limits. The label check is contains(toJSON(github.event.pull_request.labels), '"e2e:ios"') — a substring match on the PR's label list, good enough unless a project adds another label whose name contains "e2e:ios" verbatim. The github context lists event_name as pull_request | push | schedule | workflow_dispatch; the labeled run is assumed to arrive as pull_request (the expression and the comment guard both tolerate either), which is unverified until the Expo GitHub App is linked and a labelled PR has run (Owner checklist → Link the Expo GitHub App). No GitHub Actions helper or EXPO_TOKEN is involved: EAS starts the labelled run from its own webhook.
Human prerequisites (once)
- Link the Expo GitHub App so PRs trigger the workflow and the run reports back as a check: expo.dev → account → project expo-boilerplate → GitHub settings → install the app and link
seandillon1224/expo-boilerplate. The Expo user needs a linked GitHub account. NoEXPO_TOKENis involved — EAS triggers itself from the webhook; GitHub Actions never calls EAS. - Budget for the first run: with no stored E2E build for the current fingerprint, both
build_iosandbuild_androidrun (two paid builds, ~15 min each). Every JS-only PR after that repacks instead. - Make the check required: after the first PR run, read the exact check context from
gh pr checks <n>, add that string verbatim toREQUIRED_CHECKSinscripts/repo-settings.jsand runbun run repo:settings:apply(seedocs/js-gate.md→ How EAS checks appear on the PR). Until then the EAS check is informational.
Failure artifacts
What a red maestro_<p> job leaves behind, where it lives, and the local twin of each file. On the dashboard everything hangs off the run page: expo.dev → account → project expo-boilerplate → Workflows → the run (the PR comment links it) → Artifacts (one list per run) and, per job, the step logs. The maestro job's only artifact-related inputs are record_screen and output_format (checked against the live schema, https://api.expo.dev/v2/workflows/schema): there is no failure-only recording switch and no debug-output or device-log input, which is why device logs come from an after_maestro_tests hook that runs the same collector the local scripts use.
| Artifact | Produced by | Dashboard | CLI | Local equivalent |
|---|---|---|---|---|
| Screen recording per flow | record_screen: true (EAS records the simulator / emulator screen; large runner on Android for that reason) | run → Artifacts → Maestro Test Results | — (no artifact download command in eas-cli 23; eas/download_artifact only works inside a later job) | none: rerun with bun run e2e:<p> --keep and watch, or add startRecording to a flow |
JUnit report (report.xml, one <testcase> per flow with the failing command in <failure>) | output_format: junit | run → Artifacts → Maestro Test Results (Expo's docs also show it as a Maestro Test Report (junit) artifact — check both names on the first run) | — | maestro-<p>/report.xml |
Maestro debug output: failure screenshots, maestro.log, per-flow command/hierarchy JSON | Maestro CLI ($MAESTRO_TESTS_DIR on the worker, ~/.maestro/tests/<timestamp>/ by default) | run → Artifacts → Maestro Test Results | — | maestro-<p>/debug/ (--debug-output … --flatten-debug-output, same flags as bun run e2e:web) |
Device logs: simulator unified log of the app's processes + crash reports (ios), logcat -d (android) | hooks.after_maestro_tests → node scripts/e2e-device-logs.js → eas/upload_artifact (if: always(), so it runs after a red Maestro step) | run → Artifacts → Device logs (ios | android) | — | maestro-<p>/device/ (device.log + crashes/*.ips, or logcat.txt; written only when Maestro exits non-zero) |
A11y audit: report.json + one hierarchy dump per screen (screen-reader labels of every interactive element, ADR-0005) | hooks.after_maestro_tests → node scripts/a11y-audit.js --no-fail → eas/upload_artifact (if: always(); informational — skips with a notice when maestro / the device is unreachable from the hook) | run → Artifacts → A11y audit (ios | android) | — | maestro-<p>/a11y/ (bun run e2e:<p> --keep && bun run e2e:a11y --platform <p>; exits 1 on a finding) |
Flashlight: results.json, report/ (scored HTML) and README.txt (release-build CPU / RAM / FPS of one flow, android only, ADR-0007) | hooks.after_maestro_tests → node scripts/flashlight.js --platform android --no-fail --out flashlight → eas/upload_artifact (if: always(); skipped with a notice unless FLASHLIGHT is enabled or the run was dispatched with -F flashlight=enabled; installs Flashlight on the worker) | run → Artifacts → Flashlight (android) | — | flashlight/ (bun run e2e:android --keep && bun run perf:flashlight --platform android; exits 1 on a problem) |
| Step logs (device boot, install, Maestro's per-flow output, the hook) | EAS | run → job → step | bun run eas workflow:logs <run or job id> --all-steps (--json for machines) | the terminal output of bun run e2e:<p> |
| Run / job summary (statuses, ids, the build that was tested) | EAS | run page | bun run eas workflow:runs, workflow:view <run id> --json, workflow:status <run id>, workflow:cancel | — |
Sharding and retries: every shard and every retry attempt of a job writes into the same Maestro Test Results artifact, so a flow can appear more than once. The flow name: (native/<flow>, set in .maestro/flows/<flow>.yaml) is the stable key — search the artifact by it, and read the last attempt's entry as the verdict (retry_failed_only: true re-runs only the failed flows).
Retention: Expo does not document a retention window for workflow artifacts; treat them as living as long as the run does and download anything worth keeping (attach it to the issue). The web lane's maestro-web GitHub Actions artifact (same layout: report.xml, debug/) is kept for 14 days (retention-days in .github/workflows/ci.yml).
A flow failed on the PR — now what
- PR comment → the
<p> failed flowsline names the flow(s); the workflow run link opens the run. No comment (GitHub App not linked yet, or aworkflow_dispatchrun)?bun run eas workflow:runslists recent runs with ids. - Job logs: run → Maestro (<p>) → the Maestro step. Maestro prints each flow as it runs and, for the failing one, the command and selector that failed, so this alone usually answers "which step".
bun run eas workflow:logs <run id> --all-steps > run.logpulls the whole run. - Recording: Artifacts → Maestro Test Results → the failing flow's video. Scrub to the end: is the screen you expected there at all (wrong tab, permission dialog, blank screen after a JS crash), or is it the right screen with the wrong
testID? - JUnit:
report.xml→ the<failure>message of that<testcase>is the exact assertion (command, selector, timeout). Useful when the recording is ambiguous. - Maestro debug output (same artifact): the failure screenshot, then
maestro.logaround the failing command, then the flow's command JSON for the view hierarchy Maestro saw — the place to confirm atestIDis really missing rather than off-screen. Reading guide below. - Device logs: Artifacts → Device logs (<p>) →
device.log/logcat.txt. Look forReactNativeJS/RCTLoglines (JS exceptions, red-box text),Fatal,SIGABRT, and network failures; a crash on iOS also lands ascrashes/*.ips. - Reproduce:
bun run e2e:build --platform <p> && bun run e2e:repack --platform <p> && bun run e2e:<p> --keepwrites the same set undermaestro-<p>/and leaves the device up formaestro studio/ a look around. Fix, rerun, push. - Green on retry? The job already re-runs failures (
retries: 2); a flow that only passes on the second attempt is a flake, not a fix — twice in a week and it goes through the Flake budget (issue,quarantinetag, fix or delete).
Reading Maestro's debug output
--flatten-debug-output (both bun run e2e:* and the workflow's Maestro run) puts every file of the run flat in one directory, no per-run timestamp folder (names as of Maestro 2.10):
| File | What it is |
|---|---|
maestro.log | The CLI's full log: driver setup, every command with its result, the failing command's stack. Grep for the flow name, then Failed. |
screenshot-❌-<timestamp>-(<flow>).png | Taken automatically at the failing command (✅-prefixed ones come from explicit takeScreenshot steps). One glance usually explains the failure. |
commands-(<flow>).json | Every command of the flow with its status and timing; failed commands carry the error and the view hierarchy Maestro matched against — search it for the testID you expected to find. |
<flow>.mp4 / <name>.png | Only if a flow uses startRecording / takeScreenshot with a relative path; EAS's record_screen videos are separate and not in this directory locally. |
Web parity: the JS gate's Maestro web job uploads maestro-web/ (report.xml + debug/, the same layout) as the maestro-web artifact on the GitHub Actions run (docs/js-gate.md); there is no recording and no device log on web — the headless Chromium's console output is in maestro.log.
Maestro flows and tags
.maestro/ is a Maestro workspace: always point the CLI at the directory (maestro test .maestro …) so config.yaml is read — Maestro only loads it from the directory it is given, and its flow discovery is otherwise non-recursive. The header comment in config.yaml is the canonical description; in short:
| Path | Role |
|---|---|
.maestro/config.yaml | Workspace config: flows: ['flows/*', 'flows/web/*'], env contract (MAESTRO_APP_ID, APP_URL), tag and selector rules. |
.maestro/flows/<name>.yaml | Native entry: appId: ${MAESTRO_APP_ID}, tags: [ios, android], then runFlow launch + steps. Discovered by --include-tags ios|android. |
.maestro/flows/web/<name>.yaml | Web entry: url: ${APP_URL}, tags: [web], same steps. Discovered by --include-tags web (CI maestro-web, bun run e2e:web). |
.maestro/subflows/launch.yaml | Native launchApp (clearState, all permissions allowed), then sign-in.yaml. |
.maestro/subflows/launch-web.yaml | Web launchApp (opens APP_URL, clearState), then sign-in.yaml. |
.maestro/subflows/sign-in.yaml | Taps through the demo sign-in gate so every flow still starts on Home; delete it with the session demo. |
.maestro/subflows/select-tab.yaml | Tab-bar tap with a when: platform branch per OS — the only non-testID selector, see below. |
.maestro/subflows/steps/<name>.yaml | The shared steps (smoke, tabs, fetch, updates, session), written once and run by both entries. |
.maestro/fixtures/*.json | Offline API responses for the web lane, served at /fixtures/<name> by scripts/serve-web.js. Not flows; see Fetch flow data. |
Two entry files per flow are unavoidable: Maestro picks the Chromium driver from a url: header alone (url beats appId; --platform does not override it), so one file cannot serve both lanes. Subflows are never discovered as flows (the globs only match flows/* and flows/web/*) but Maestro 2.x still requires a config section in every file, hence the inert appId: ${MAESTRO_APP_ID} header on each subflow.
Tags: web, ios, android select entries; quarantine marks a flaky entry and every gate excludes it (--exclude-tags quarantine) — see Flake budget for the rules. Maestro's tag logic: several tags in one flag are OR (--include-tags ios,android = either); --include-tags and --exclude-tags together are AND (included, then minus excluded). There is no way to require two tags at once, which is why the quarantine run selects quarantine and excludes web rather than asking for quarantine + ios.
Selectors are testIDs only (id: = accessibilityIdentifier on iOS, resource-id on Android, DOM id on web). The tab bar is the one exception, isolated in select-tab.yaml: NativeTabs.Trigger gets testID="tab-<route>" in src/app/(tabs)/_layout.tsx, which is the item's accessibility identifier on iOS (id: tab-settings), but only a view tag on Android (Maestro cannot read it, so the visible label is matched via TAB_LABEL) and absent on web (Radix generates radix-<uid>-trigger-(settings)-<nanoid>, matched by regex).
There is no way to give the Android tab bar a real id, and it is worth knowing why before anyone tries again. NativeTabs.Trigger exposes exactly two selectable strings on Android: testID, which expo-router documents as "the item's view tag, which Espresso-based drivers like Detox read but Maestro and Appium do not", and accessibilityLabel, which becomes the item's contentDescription. The second one is matchable — but it is also the string TalkBack reads aloud, so buying a stable selector with it means every blind user hears "tab-settings" instead of "Settings", and bun run e2e:a11y exists to catch exactly that. The label selector stays, and the variable it depends on gets pinned instead.
Device locale
Matching a visible label is only deterministic if the device's language is. Today the app bundles one catalog (src/i18n/locales/en) and every other locale falls back to it, so a French emulator still renders "Settings" — but the template invites a second catalog, and the day one lands, a worker image with a different locale turns a healthy app into a failing tab tap. So the Android lane pins the locale rather than assuming it:
| Where | How |
|---|---|
bun run e2e:android | An AVD this script boots gets -prop persist.sys.locale=en-US at launch; an already-running device is pinned by pinAndroidLocale() before the APK is installed. |
maestro_android (.eas/workflows/e2e.yml) | A before_maestro_tests hook runs the same file: node scripts/e2e-pin-locale.js. |
scripts/e2e-pin-locale.js reads persist.sys.locale (falling back to the image's ro.product.locale) and does nothing when the device is already on an English locale — the common path on a stock emulator image is one getprop. When it does differ, the property alone is not enough: the framework reads it at start, so the script restarts the framework (adb shell stop && start, ~20 s) and waits for sys.boot_completed. It never exits non-zero — a locale that could not be pinned (a production-signed device rejects setprop) is a warning, so the run still fails on the flows rather than on the preflight.
iOS needs none of this: the iOS branch of select-tab.yaml selects by accessibility identifier, which does not move with the language. Change the pinned locale in one place, DEFAULT_LOCALE in scripts/e2e-pin-locale.js, and keep the TAB_LABEL values in subflows/steps/*.yaml spelled the way that locale's catalog spells them.
Adding a flow
Write the steps once in
.maestro/subflows/steps/<name>.yamlwith anappId: ${MAESTRO_APP_ID}header. Select byid:only; for the tab bar userunFlow: { file: ../select-tab.yaml, env: { TAB: settings, TAB_LABEL: Settings } }.Add the native entry
.maestro/flows/<name>.yaml:yamlname: native/<name> appId: ${MAESTRO_APP_ID} tags: [ios, android] --- - runFlow: ../subflows/launch.yaml - runFlow: ../subflows/steps/<name>.yamlAdd the web entry
.maestro/flows/web/<name>.yamlwithname: web/<name>,url: ${APP_URL},tags: [web]and../../subflows/launch-web.yaml+../../subflows/steps/<name>.yaml.Verify web locally (
bun run export:web:e2e && bun run serve:web &thenbun run e2e:web) and native with the loop above. A step that must differ per platform goes in its own subflow withwhen: { platform: iOS | Android | Web }blocks, likeselect-tab.yaml.
Fetch flow data
subflows/steps/fetch.yaml is shared, but the data behind it is not:
- Web reads
.maestro/fixtures/posts.json, served at/fixtures/postsbyscripts/serve-web.js, becausebun run export:web:e2ebakesEXPO_PUBLIC_API_URL=http://localhost:8081/fixturesinto the export.Maestro webis a required check, so it owns its data and never reaches the public internet. - Native still hits
https://jsonplaceholder.typicode.com— theEXPO_PUBLIC_API_URLdefault. The native lane is not a GitHub-required check and a real HTTPS round trip on a real device is part of what it is testing.
That asymmetry is deliberate but not load-bearing. If jsonplaceholder starts costing the native lane green runs, give it the same treatment: set EXPO_PUBLIC_API_URL on the e2e-* profiles in eas.json to a host the worker can reach, or add the quarantine tag while you do (Flake budget) — do not delete the assertions.
The fixture must satisfy postSchema in src/features/posts/api.ts: since T13.2 the response is Zod-parsed, so a drifted fixture shows up as the flow's error state, not as a silently empty list. Full wiring: Testing → Test data.
Flake budget
A flow that fails and then passes on retry proves nothing about the app and everything about the test. The budget keeps such flows from silently turning the gate into "retry until green" without letting them block unrelated PRs either.
Policy
| Rule | Native (EAS e2e.yml) | Web (CI maestro-web) and local (bun run e2e:*) |
|---|---|---|
| Retries | retries: 2, retry_failed_only: true. Two because they cover two different things: one for an infra hiccup (emulator boot, first-launch install), one for a genuine flake. A flow that fails three times in a row is a real failure, never a flake — read the artifacts, do not re-run. | None. Maestro CLI 2.10 has no flow-level retry flag or config.yaml option, and a wrapper re-running the whole suite would hide exactly the signal this section is about. Flakes therefore show up faster on web and locally — that is a feature. |
| What counts as flaky | Green on retry (attempt 1 red, attempt 2/3 green) at least twice in one week for the same flow, or any green-on-retry that a human cannot explain from the artifacts. | Any run that is red then green on an unchanged tree, twice in a week. |
| What is never allowed | Deleting or weakening assertions, adding sleeps, or widening a timeout past the point where the assertion still means something, to make a flow pass. Fix the cause (extendedWaitUntil, a deterministic fixture, a testID) or quarantine it. | Same. |
| Quarantine max age | 2 weeks from the tag landing on main. Then: fixed and un-quarantined, or the flow is deleted (with the issue explaining why the coverage was not worth keeping). | Same. |
Quarantining a flow
File the issue with the Flaky Maestro flow template (
.github/ISSUE_TEMPLATE/flaky-flow.yml, labelsflaky-flow+e2e): flow file, platforms, run URL, failure rate, recording and JUnit<failure>text, suspected cause. Download the artifacts you cite — runs are not kept forever.Tag the entry file — the native entry (
.maestro/flows/<name>.yaml) or the web entry (.maestro/flows/web/<name>.yaml), never a subflow — keeping the platform tags:yamlname: native/<name> appId: ${MAESTRO_APP_ID} # quarantine: #123 — passes on retry ~30% of Android runs; assertion races the fetch tags: [ios, android, quarantine]One native entry serves both platforms, so quarantine applies to both; an iOS-only flake still takes the flow out of the Android gate for those two weeks. The web entry is a separate file and stays in the gate unless it is tagged too.
Open a PR for the tag alone (Conventional Commit
test(e2e): quarantine <flow> (#123)). If this is the first quarantined flow, uncommentschedulein.eas/workflows/e2e-quarantine.ymlin the same PR so it starts running weekly. The gate goes green because every lane excludes the tag:exclude_tags: [quarantine]ine2e.yml,--exclude-tags quarantineinbun run e2e:webandscripts/e2e-run.js. Tick the Quarantine box on the issue with the PR number.Fix it within 2 weeks on a branch that removes the tag, so the PR's own E2E run proves the fix (see the checklist below). No fix in 2 weeks → delete the entry, its web twin and the steps subflow if nothing else uses it, and close the issue saying so.
Un-quarantine checklist
- [ ] Root cause named in the issue (not "made it more robust").
- [ ] The fix is in the flow or the app, not in the assertion: no removed
assertVisible, no addedsleep, no timeout stretched past what the feature promises. - [ ]
bun run e2e:<p> --quarantine-only(native) or a temporary tag swap for web passes 5× in a row locally. - [ ] The tag and the
# quarantine:comment are removed in the fixing PR; that PR'sE2E (native)/Maestro webchecks are green on the first attempt (look at the JUnit in Maestro Test Results for a retry entry). - [ ] Last quarantined flow gone? Comment
scheduleback out ine2e-quarantine.yml(maestro testexits 1 when no flow matches, so a weekly cron with nothing to run is a weekly red run). - [ ] Issue closed by the PR (
Closes #123).
Running quarantined flows
| Where | How |
|---|---|
| EAS | .eas/workflows/e2e-quarantine.yml: bun run eas workflow:run .eas/workflows/e2e-quarantine.yml (or weekly via schedule, once enabled). Same fingerprint → get-build / build → repack → maestro shape as e2e.yml, include_tags: [quarantine] + exclude_tags: [web], retries: 0, no PR comment, no hooks. |
| Local | bun run e2e:ios --quarantine-only / bun run e2e:android --quarantine-only — same selection as the workflow (quarantine minus web). |
| Web | No script flag: swap the tags by hand — maestro test .maestro -e APP_URL=http://localhost:8081 --include-tags quarantine --exclude-tags ios,android --headless after bun run export:web:e2e && bun run serve:web &. |
maestro test exits 1 when no flow matches the tags, on every lane.
Metrics
- Retries that saved a run: the JUnit
report.xmlin Maestro Test Results (native) or themaestro-webartifact (web) lists one<testcase>per attempt of a flow, keyed by itsname:(native/<flow>/web/<flow>); a flow with a<failure>entry followed by a passing entry is a retry that passed. Count those per flow per week — two is the quarantine threshold. - Run history:
bun run eas workflow:runs(--jsonfor scripting) lists theE2E (native)andE2E (quarantine)runs with status and commit;workflow:view <run id> --jsongives the per-job outcome. Web:gh run list --workflow ci.yml --json conclusion,headSha,url. - Quarantine size and age:
grep -rn '# quarantine:' .maestro/flows— the issue number in each comment dates it. Anything older than 2 weeks is overdue.
Maestro Cloud (optional)
.eas/workflows/e2e-cloud.yml (PLAN.md D7, ADR-0006) runs the same native flows on Maestro Cloud, mobile.dev's hosted device farm, through EAS's pre-packaged maestro-cloud job. It is not the maestro job of e2e.yml: that one runs Maestro on the EAS worker's own simulator / emulator and costs EAS minutes only; Maestro Cloud runs on real devices, in parallel, and bills its own plan. Use it when a project already pays for Maestro Cloud, needs real-device coverage (permissions, keyboards, OEM quirks) or has more flows than one worker shards comfortably. Everything else stays on e2e.yml.
Shape. fingerprint → get_build_<p> → repack_<p> (cache hit only) → cloud_<p> per platform, both platforms as independent jobs, plus a refuse job. The maestro-cloud job fetches the repack (this commit's JS on the cached base), installs Maestro 2.10.0 and uploads with --include-tags <p> --exclude-tags quarantine, MAESTRO_APP_ID in the job env and name: <p> <sha>. Device model / OS are Maestro Cloud's defaults; set device_model / device_os on a cloud_<p> job (maestro list-cloud-devices lists the values) to pin them.
What it refuses. There is no build_<p> job. On a fingerprint miss the platform is skipped and refuse prints the fix: run E2E (native) on the PR (or bun run e2e:build --platform <p> --build) so a base build exists, then re-add the label. A label click must never cost a native build.
Enable (owner, once). Off by default so a labelled PR on a fresh project is skipped green.
- Maestro Cloud → Settings: create an API key and copy the project id (
proj_…). bun run eas env:create --scope project --environment development --name MAESTRO_CLOUD_API_KEY --value <key> --visibility secret --type string --non-interactive—developmentis the e2e lane's environment; the job reads it as the default ofmaestro_api_key. It is never a literal in the file.- In one PR: replace
proj_REPLACE_MEin bothcloud_<p>jobs and flipMAESTRO_CLOUDtoenabled(theif:literal oncloud_ios,cloud_android,refuseand themaestro_cloudinput default — Repo constants). bun run repo:settings:apply --only labelscreates thee2e:cloudlabel.
Run. Add the e2e:cloud label to a PR (each add is a run), or by hand: bun run eas workflow:run .eas/workflows/e2e-cloud.yml -F maestro_cloud=enabled (the input beats the file constant, so a run is possible before the flip).
Results. The Maestro Cloud console has the recordings and per-flow logs; the run page's cloud_<p> job outputs carry maestro_cloud_url and the flow counts (total_flows_count, failed_flows_count, failed_flow_names_json). Maestro Cloud posts its own PR check when the upload is tied to a PR — there is no github-comment job here. The job fails when a flow fails (unless async: true is added, which only reports the upload).
Unverified. The template has no Maestro Cloud account (Owner checklist → Maestro Cloud), so this workflow has passed eas workflow:validate only. The first real run confirms that job env reaches the flows as ${MAESTRO_APP_ID} (documented for MAESTRO_* names), that flows: .maestro (the workspace with config.yaml) is accepted rather than needing flows: .maestro/flows + maestro_config, and whether the PR check appears. Record what you find in ADR-0006's follow-ups.
Update → approval → submit
The rungs past the E2E check — OTA to staging on merge, approval-gated promotion to UAT and production, and the store release — are the release ladder.