CI and GitHub
Runs in CI, suites as checks on commits, and comments on pull requests.
TestedOK runs wherever a simulator or emulator runs: a Mac in GitHub Actions, Xcode Cloud, a Mac mini under a desk. The result comes back as a check on the commit and a comment on the pull request.
Running in CI
jobs:
flows:
runs-on: macos-15
steps:
- uses: actions/checkout@v4
- run: xcodebuild -scheme Rote -destination 'platform=iOS Simulator,name=iPhone 17 Pro' -derivedDataPath build build
- run: curl -fsSL https://testedok.dev/install | sh
- run: xcrun simctl boot "iPhone 17 Pro" && xcrun simctl install booted build/Build/Products/Debug-iphonesimulator/Rote.app
- run: testedok run --suite smoke --env ci --junit report.xml
env:
TESTEDOK_API_URL: https://api.testedok.dev
TESTEDOK_TOKEN: ${{ secrets.TESTEDOK_TOKEN }}
TESTEDOK_SECRET_STAGING_PASSWORD: ${{ secrets.STAGING_PASSWORD }}TESTEDOK_TOKEN is an API token from Account settings in the dashboard. Secrets arrive as TESTEDOK_SECRET_<NAME>. The job fails when a flow does, and --junit gives the runner something to show. An environment in testedok.yaml whose app is the built .app path installs it itself.
Where a run came from
The CLI records the commit, repository, branch and pull request with each run: from GitHub Actions, Xcode Cloud, EAS Build, Bitrise, CircleCI or GitLab CI when it runs there, from the checkout elsewhere, or from TESTEDOK_COMMIT, TESTEDOK_REPO, TESTEDOK_BRANCH and TESTEDOK_PR for pipelines that know better. The run page shows them.
The GitHub App
Connect it once per project (App settings → GitHub → Connect GitHub), choosing the repositories. From then on, every run that knows its commit posts a check run on that commit: one check named after the suite, listing every flow that ran on the commit with its result, time and a link to the run, red if any failed. On a pull request the same summary is kept as one comment, edited as runs finish, never one per run.
Checks are what branch protection reads. Make the smoke suite required and a broken flow stops the merge; leave a nightly suite unrequired and it only reports.
A suite marked blocking: false in testedok.yaml is informational: its failures appear in the check's table, marked as such, but the check's conclusion is neutral rather than failure. See Schedules, health and notifications.
Fastlane
The fastlane-plugin-testedok plugin runs a suite against the build a lane just made, installing the CLI if the machine lacks it, and fails the lane when a blocking flow does.
lane :smoke do
build_app(scheme: "Rote", destination: "generic/platform=iOS Simulator", derived_data_path: "build", skip_archive: true, skip_codesigning: true)
sh("xcrun simctl boot 'iPhone 17 Pro' || true")
testedok(suite: "smoke", env: "ci", app: "build/Build/Products/Debug-iphonesimulator/Rote.app", junit: "fastlane/test_output/testedok.xml")
endOptions: suite, tag, env, app (the .app or .apk to install; default the environment's), paths, sim, junit, video, project, token (or TESTEDOK_TOKEN), api_url, install_dir, fail_build. Add gem "fastlane-plugin-testedok" to the Gemfile.
Xcode Cloud
Xcode Cloud runs ci_scripts/ci_post_xcodebuild.sh after every build. Ours finds the simulator .app in DerivedData, installs the CLI, boots a simulator and runs the suite; the commit, branch and pull request come from Xcode Cloud's own CI_* variables, so the check and the comment land on GitHub.
mkdir -p ci_scripts && curl -fsSL https://testedok.dev/integrations/xcode-cloud/ci_post_xcodebuild.sh -o ci_scripts/ci_post_xcodebuild.sh && chmod +x ci_scripts/ci_post_xcodebuild.shIn the workflow's environment set TESTEDOK_TOKEN (secret), TESTEDOK_SUITE (default smoke), TESTEDOK_ENV (optional), TESTEDOK_SIMULATOR (default "iPhone 17 Pro") and any TESTEDOK_SECRET_*. The workflow's build action needs a simulator destination; an archive has no .app to run and the script says so and exits clean.
Expo EAS
On a simulator build profile (eas.json: "ios": { "simulator": true }), an eas-build-on-success hook runs the suite on the build worker:
mkdir -p scripts && curl -fsSL https://testedok.dev/integrations/eas/testedok-eas.sh -o scripts/testedok-eas.sh && chmod +x scripts/testedok-eas.sh{ "scripts": { "eas-build-on-success": "sh scripts/testedok-eas.sh" } }Set TESTEDOK_TOKEN and friends as EAS secrets or in the profile's env. The commit and branch come from EAS_BUILD_GIT_*; add TESTEDOK_REPO (owner/name) and TESTEDOK_PR for the check and the comment.
Bitrise, CircleCI, GitLab
Any macOS runner works with the two lines from the top of this page: install the CLI, testedok run --suite smoke --env ci --app <built .app> --junit report.xml. The CLI reads the commit, branch and pull request from Bitrise (BITRISE_GIT_*), CircleCI (CIRCLE_*) and GitLab CI (CI_COMMIT_SHA, CI_MERGE_REQUEST_IID) on its own; --app installs the build every flow then runs against.