Skip to content

ci(e2e-test-stack): mobile runs by platform, with TSIO upload, mobile's own shard steps and email on - #49

Merged
saturninoabril merged 2 commits into
masterfrom
e2e-test-stack-mobile-platform
Nov 19, 2026
Merged

saturninoabril merged 2 commits into
masterfrom
e2e-test-stack-mobile-platform

Conversation

@saturninoabril

@saturninoabril saturninoabril commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Four changes to the by-hand mobile E2E run on e2e-test-stack servers (e2e-test-stack-mobile.yml), each needed to compare it with mobile's own runs.

  1. Platform input and run name. A required platform input (detox-ios, detox-android; maestro later), checked against the build artifact's name, so iOS and Android are started and debugged separately. A run-name shows the platform, the specs and the shard count.

  2. Upload to Test System IO. With upload_to_tsio, the tsio job merges the shards' results and uploads them as mobile's report job does, with mobile's scripts and TSIO's action, under this repository as mobile-e2e-test-stack-<platform>. That puts the run beside mobile's own reports for the same build.

  3. Mobile's own shard steps. The shard workflows had drifted from mobile's e2e-ios and e2e-android jobs. They are regenerated from mobile's templates (13407ac), and every line that differs is marked # test-public-project::

    • checkout of mattermost-mobile at the build's commit; matrix.* read from inputs
    • steps before mobile's: the create response, Node, the servers, site assignment (in place of "Assign test sites by shard role"), Bash 5 on iOS (macos-15 ships 3.2), and provisioning with this repository's adapted copy of mobile's action
    • empty tunnel secrets for the webhook sidecar; id: detox on the test step; overwrite: true on the first two result uploads, so a job re-run uploads again
    • not continue-on-error, and a final ci/fail-on-failed-tests, so a failed shard fails its job and keeps its batch for "Re-run failed jobs"
    • Android runs on the runner the caller picks (android_runner)

    In the run workflow, each job is marked test-public-project: or names the mobile job it mirrors. A shard that times out (reported as cancelled) keeps its batch and logs, as a failed one does; only cancelling the run destroys it.

  4. Email on. SendEmailNotifications, FeedbackEmail and ReplyToAddress are added to the servers' environment, as on Matterwick's servers; the stack leaves email off. Without them, Android run 44464370672 failed MM-T5114_3, which taps the "Immediately" email option the app shows only when email is on. Mobile's own run on the same build (13407ac) passed it.

Test plan

  • actionlint, yamllint and the pre-commit hook pass
  • Every shard step compared with mobile's templates: only marked lines differ
  • detox-android smoke run (account, autocomplete) on ubuntu-latest-8-cores
  • detox-ios smoke run (account, autocomplete)
  • Run with upload_to_tsio

Change Impact: 🟡 Medium

Changes Breakdown:

+327 -153 Production Changes (100%)
+0 -0 Test Changes (0%)

Regression Risk: The shard workflows remove the device_name and device_os_version inputs and change artifact download, emulator, and failure handling, so existing callers can break. The detox-ios, detox-android, and upload_to_tsio smoke runs are not complete, so these changed paths lack validation.

Generated by CodeRabbitAI

…t differs here

The iOS and Android shard workflows are rebuilt from mattermost-mobile's e2e-ios and
e2e-android jobs, changed only where marked "test-public-project:". The run workflow
takes a required platform input (detox-ios, detox-android) and a run-name, uploads to
Test System IO by hand, and marks each job as either specific to this repository or
mirroring mobile. A shard that times out keeps its batch and logs, as a failed one does.
…ick's

The account smoke spec's notification settings test taps the Immediately email option,
which the app shows only when the server sends email notifications. The stack leaves it off.
@saturninoabril saturninoabril changed the title ci(e2e-test-stack): copy mobile's Detox jobs as they are, marking what differs here ci(e2e-test-stack): mobile runs by platform, with TSIO upload, mobile's own shard steps and email on Nov 19, 2026
@saturninoabril
saturninoabril changed the base branch from main to master November 19, 2026 05:47
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Explain Change →

Walkthrough

The mobile E2E workflow now requires a platform selection and checks that the build artifact matches it. It updates Android and iOS shard setup and outcome handling, and adds an optional job to merge shard results and upload them to TSIO.

Changes: Mobile E2E Stack

Layer / File(s) Summary
Platform selection and build artifacts
.github/workflows/e2e-test-stack-mobile.yml, e2e-test-stack/README.md
The workflow accepts detox-ios or detox-android, checks the resolved artifact platform, and stages Android APKs in the shard download layout. The README documents platform selection, optional TSIO uploads, and AWS-role requirements.
Server planning and shard lifecycle
.github/workflows/e2e-test-stack-mobile.yml
The workflow configures server email settings and updates planning, shard input wiring, log collection, and destruction conditions for shard and workflow outcomes.
Android shard setup and execution
.github/workflows/e2e-test-stack-mobile-android-shard.yml
The Android shard uses AVD, API-level, and attempt inputs. It installs dependencies after provisioning, validates the artifact run ID, downloads the build, and updates SDK and emulator cache settings.
iOS shard setup and execution
.github/workflows/e2e-test-stack-mobile-ios-shard.yml
The iOS shard adds low-bandwidth setup and proxy handling, adjusts job and Detox timeouts, and updates readiness checks and retry processing.
Shard evidence, reports, and failure handling
.github/workflows/e2e-test-stack-mobile-android-shard.yml, .github/workflows/e2e-test-stack-mobile-ios-shard.yml
The shard workflows add conditional triage-evidence uploads. The iOS workflow also resets network settings and can upload mitmdump output. Both workflows retain report retries and remove report-upload failure checks.
Optional TSIO result upload
.github/workflows/e2e-test-stack-mobile.yml
When enabled and planning succeeds, the TSIO job downloads shard results, merges them against planned specs, creates a failure stub if merging fails, and uploads the report with its identity configuration.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TSIOJob
  participant ShardArtifacts
  participant MergeUtility
  participant TSIOUploader
  TSIOJob->>ShardArtifacts: Download platform-specific shard results
  TSIOJob->>MergeUtility: Merge results against planned specs
  MergeUtility-->>TSIOJob: Return merged results or merge failure
  TSIOJob->>TSIOUploader: Upload merged report with identity configuration
Loading

Possibly related PRs

Merge Risk: Moderate · up to 71a2b

The iOS shard workflow has regressions in test infrastructure. A failed retry-result merge can let a shard report success. Metro startup checks can hang without a time limit. In normal mode the job timeout can expire before the Detox step timeout, which can cost evidence and reports. These are CI-only, easily reversible issues; fix them before merging.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the ci(e2e-test-stack) scope and the main change: the mobile E2E workflow now runs by platform. It also names the TSIO upload, the mobile shard steps, and the email setting. The change…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the ci(e2e-test-stack) scope and the main change: the mobile E2E workflow now runs by platform. It also names the TSIO upload, the mobile shard steps, and the email setting. The changeset matches each part. The title is long (100 characters, 15 words) and lists several changes, but it stays on topic and is specific.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 37c37c59-e7a5-4bfa-9092-3dae0bc525fa
Commits

Reviewing files that changed from the base of the PR and between 7cb0cab and 71a2bf6.

Files selected for processing: 4

  • .github/workflows/e2e-test-stack-mobile-android-shard.yml
  • .github/workflows/e2e-test-stack-mobile-ios-shard.yml
  • .github/workflows/e2e-test-stack-mobile.yml
  • e2e-test-stack/README.md

Included review availability: This review used your included allowance. 1 included review remains after this review. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.

@saturninoabril
saturninoabril merged commit 1a5b199 into master Nov 19, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant