Running

CI and GitHub

Runs in CI, suites as checks on commits, and comments on pull requests.

TestedOK runs wherever a simulator or emulator runs: a Mac in GitHub Actions, Xcode Cloud, a Mac mini under a desk. The result comes back as a check on the commit and a comment on the pull request.

Running in CI

.github/workflows/testedok.yml

jobs:
  flows:
    runs-on: macos-15
    steps:
      - uses: actions/checkout@v4
      - run: xcodebuild -scheme Rote -destination 'platform=iOS Simulator,name=iPhone 17 Pro' -derivedDataPath build build
      - run: curl -fsSL https://testedok.dev/install | sh
      - run: xcrun simctl boot "iPhone 17 Pro" && xcrun simctl install booted build/Build/Products/Debug-iphonesimulator/Rote.app
      - run: testedok run --suite smoke --env ci --junit report.xml
        env:
          TESTEDOK_API_URL: https://api.testedok.dev
          TESTEDOK_TOKEN: ${{ secrets.TESTEDOK_TOKEN }}
          TESTEDOK_SECRET_STAGING_PASSWORD: ${{ secrets.STAGING_PASSWORD }}

TESTEDOK_TOKEN is an API token from Plan and tokens in the dashboard. Secrets arrive as TESTEDOK_SECRET_<NAME>. The job fails when a flow does, and --junit gives the runner something to show. An environment in testedok.yaml whose app is the built .app path installs it itself.

Where a run came from

The CLI records the commit, repository, branch and pull request with each run: from GITHUB_SHA, GITHUB_REPOSITORY and GITHUB_REF in GitHub Actions, from the checkout elsewhere, or from TESTEDOK_COMMIT, TESTEDOK_REPO, TESTEDOK_BRANCH and TESTEDOK_PR for pipelines that know better. The run page shows them.

The GitHub App

Connect it once per project (Settings → GitHub → Connect GitHub), choosing the repositories. From then on, every run that knows its commit posts a check run on that commit: one check named after the suite, listing every flow that ran on the commit with its result, time and a link to the run, red if any failed. On a pull request the same summary is kept as one comment, edited as runs finish, never one per run.

Checks are what branch protection reads. Make the smoke suite required and a broken flow stops the merge; leave a nightly suite unrequired and it only reports.

Other pipelines

Xcode Cloud, Expo EAS and Fastlane can all run the same two lines: install the CLI, run a suite with --junit. A Fastlane action and copyable Xcode Cloud and EAS scripts are coming; until then, ci_post_xcodebuild.sh with the commands above is enough.