You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci(e2e-test-stack): mobile runs by platform, with TSIO upload, mobile's own shard steps and email on - #49
Four changes to the by-hand mobile E2E run on e2e-test-stack servers (e2e-test-stack-mobile.yml), each needed to compare it with mobile's own runs.
Platform input and run name. A required platform input (detox-ios, detox-android; maestro later), checked against the build artifact's name, so iOS and Android are started and debugged separately. A run-name shows the platform, the specs and the shard count.
Upload to Test System IO. With upload_to_tsio, the tsio job merges the shards' results and uploads them as mobile's report job does, with mobile's scripts and TSIO's action, under this repository as mobile-e2e-test-stack-<platform>. That puts the run beside mobile's own reports for the same build.
Mobile's own shard steps. The shard workflows had drifted from mobile's e2e-ios and e2e-android jobs. They are regenerated from mobile's templates (13407ac), and every line that differs is marked # test-public-project::
checkout of mattermost-mobile at the build's commit; matrix.* read from inputs
steps before mobile's: the create response, Node, the servers, site assignment (in place of "Assign test sites by shard role"), Bash 5 on iOS (macos-15 ships 3.2), and provisioning with this repository's adapted copy of mobile's action
empty tunnel secrets for the webhook sidecar; id: detox on the test step; overwrite: true on the first two result uploads, so a job re-run uploads again
not continue-on-error, and a final ci/fail-on-failed-tests, so a failed shard fails its job and keeps its batch for "Re-run failed jobs"
Android runs on the runner the caller picks (android_runner)
In the run workflow, each job is marked test-public-project: or names the mobile job it mirrors. A shard that times out (reported as cancelled) keeps its batch and logs, as a failed one does; only cancelling the run destroys it.
Email on.SendEmailNotifications, FeedbackEmail and ReplyToAddress are added to the servers' environment, as on Matterwick's servers; the stack leaves email off. Without them, Android run 44464370672 failed MM-T5114_3, which taps the "Immediately" email option the app shows only when email is on. Mobile's own run on the same build (13407ac) passed it.
Test plan
actionlint, yamllint and the pre-commit hook pass
Every shard step compared with mobile's templates: only marked lines differ
detox-android smoke run (account, autocomplete) on ubuntu-latest-8-cores
detox-ios smoke run (account, autocomplete)
Run with upload_to_tsio
Change Impact: 🟡 Medium
Changes Breakdown:
+327 -153 Production Changes (100%)
+0 -0 Test Changes (0%)
Regression Risk: The shard workflows remove the device_name and device_os_version inputs and change artifact download, emulator, and failure handling, so existing callers can break. The detox-ios, detox-android, and upload_to_tsio smoke runs are not complete, so these changed paths lack validation.
…t differs here
The iOS and Android shard workflows are rebuilt from mattermost-mobile's e2e-ios and
e2e-android jobs, changed only where marked "test-public-project:". The run workflow
takes a required platform input (detox-ios, detox-android) and a run-name, uploads to
Test System IO by hand, and marks each job as either specific to this repository or
mirroring mobile. A shard that times out keeps its batch and logs, as a failed one does.
…ick's
The account smoke spec's notification settings test taps the Immediately email option,
which the app shows only when the server sends email notifications. The stack leaves it off.
saturninoabril
changed the title
ci(e2e-test-stack): copy mobile's Detox jobs as they are, marking what differs here
ci(e2e-test-stack): mobile runs by platform, with TSIO upload, mobile's own shard steps and email on
Nov 19, 2026
The mobile E2E workflow now requires a platform selection and checks that the build artifact matches it. It updates Android and iOS shard setup and outcome handling, and adds an optional job to merge shard results and upload them to TSIO.
Changes: Mobile E2E Stack
Layer / File(s)
Summary
Platform selection and build artifacts .github/workflows/e2e-test-stack-mobile.yml, e2e-test-stack/README.md
The workflow accepts detox-ios or detox-android, checks the resolved artifact platform, and stages Android APKs in the shard download layout. The README documents platform selection, optional TSIO uploads, and AWS-role requirements.
Server planning and shard lifecycle .github/workflows/e2e-test-stack-mobile.yml
The workflow configures server email settings and updates planning, shard input wiring, log collection, and destruction conditions for shard and workflow outcomes.
Android shard setup and execution .github/workflows/e2e-test-stack-mobile-android-shard.yml
The Android shard uses AVD, API-level, and attempt inputs. It installs dependencies after provisioning, validates the artifact run ID, downloads the build, and updates SDK and emulator cache settings.
iOS shard setup and execution .github/workflows/e2e-test-stack-mobile-ios-shard.yml
The iOS shard adds low-bandwidth setup and proxy handling, adjusts job and Detox timeouts, and updates readiness checks and retry processing.
Shard evidence, reports, and failure handling .github/workflows/e2e-test-stack-mobile-android-shard.yml, .github/workflows/e2e-test-stack-mobile-ios-shard.yml
The shard workflows add conditional triage-evidence uploads. The iOS workflow also resets network settings and can upload mitmdump output. Both workflows retain report retries and remove report-upload failure checks.
Optional TSIO result upload .github/workflows/e2e-test-stack-mobile.yml
When enabled and planning succeeds, the TSIO job downloads shard results, merges them against planned specs, creates a failure stub if merging fails, and uploads the report with its identity configuration.
The iOS shard workflow has regressions in test infrastructure. A failed retry-result merge can let a shard report success. Metro startup checks can hang without a time limit. In normal mode the job timeout can expire before the Detox step timeout, which can cost evidence and reports. These are CI-only, easily reversible issues; fix them before merging.
Pre-merge checks: 4
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title names the ci(e2e-test-stack) scope and the main change: the mobile E2E workflow now runs by platform. It also names the TSIO upload, the mobile shard steps, and the email setting. The change…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the ci(e2e-test-stack) scope and the main change: the mobile E2E workflow now runs by platform. It also names the TSIO upload, the mobile shard steps, and the email setting. The changeset matches each part. The title is long (100 characters, 15 words) and lists several changes, but it stays on topic and is specific.
Finishing Touches
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Included review availability: This review used your included allowance. 1 included review remains after this review. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four changes to the by-hand mobile E2E run on e2e-test-stack servers (
e2e-test-stack-mobile.yml), each needed to compare it with mobile's own runs.Platform input and run name. A required
platforminput (detox-ios,detox-android; maestro later), checked against the build artifact's name, so iOS and Android are started and debugged separately. Arun-nameshows the platform, the specs and the shard count.Upload to Test System IO. With
upload_to_tsio, thetsiojob merges the shards' results and uploads them as mobile's report job does, with mobile's scripts and TSIO's action, under this repository asmobile-e2e-test-stack-<platform>. That puts the run beside mobile's own reports for the same build.Mobile's own shard steps. The shard workflows had drifted from mobile's
e2e-iosande2e-androidjobs. They are regenerated from mobile's templates (13407ac), and every line that differs is marked# test-public-project::matrix.*read from inputsid: detoxon the test step;overwrite: trueon the first two result uploads, so a job re-run uploads againcontinue-on-error, and a finalci/fail-on-failed-tests, so a failed shard fails its job and keeps its batch for "Re-run failed jobs"android_runner)In the run workflow, each job is marked
test-public-project:or names the mobile job it mirrors. A shard that times out (reported as cancelled) keeps its batch and logs, as a failed one does; only cancelling the run destroys it.Email on.
SendEmailNotifications,FeedbackEmailandReplyToAddressare added to the servers' environment, as on Matterwick's servers; the stack leaves email off. Without them, Android run 44464370672 failed MM-T5114_3, which taps the "Immediately" email option the app shows only when email is on. Mobile's own run on the same build (13407ac) passed it.Test plan
detox-androidsmoke run (account, autocomplete) onubuntu-latest-8-coresdetox-iossmoke run (account, autocomplete)upload_to_tsioChange Impact: 🟡 Medium
Changes Breakdown:
+327 -153 Production Changes (100%)
+0 -0 Test Changes (0%)
Regression Risk: The shard workflows remove the
device_nameanddevice_os_versioninputs and change artifact download, emulator, and failure handling, so existing callers can break. Thedetox-ios,detox-android, andupload_to_tsiosmoke runs are not complete, so these changed paths lack validation.Generated by CodeRabbitAI