Test/real provider e2e agent04 - #122
Merged
ClaudiaFang merged 6 commits intoAug 13, 2026
Merged
Conversation
Adds e2e/suites/sync-manager.e2e.test.ts, parametrized by E2E_PROVIDER so it reuses the existing github/gitlab/gitea adapters and verifiers rather than duplicating provider-service contract coverage. Real SyncManager + real production provider service; only the Obsidian filesystem boundary is faked (e2e/shim/fake-vault.ts, an in-memory Map, not vi.fn() mocks). Covers: local new file -> push, unchanged file -> no remote mutation, remote update -> pull, conflict protection (push must not silently overwrite, and must not falsely mark synced), rename/move in exactly one commit, delete via the real service (SyncStatusView's actual call path, not a SyncManager method), and batch push-all in exactly one commit. Extends e2e/shim/obsidian-request-url.ts with the minimal set of real `obsidian` values SyncManager's dependency graph needs at runtime (TFile, Notice, Platform, FileSystemAdapter, Modal, plus PluginSettingTab/ TextComponent/AbstractInputSuggest/TFolder/Setting, pulled in transitively via `../settings`'s pure functions sharing a module with the settings-tab UI class) -- see that file's header comment for the full trace. Wires the new suite into scripts/run-e2e.mjs so `npm run test:e2e -- provider <name>` covers both the contract suite and SyncManager scenarios in one command/container lifecycle. Verified for real: `npm run test:e2e -- --provider gitea` against local Docker, 14/14 passing across multiple consecutive runs. GitHub/GitLab share the same harness but have only been lint/build/typecheck-verified here (no sandbox credentials in this environment) -- see docs/testing/real-provider-e2e.md.
GitHubVerifier already had a GitHub-specific listCommitShas (used by the symlink/GraphQL regression suite); promotes it to the shared RemoteVerifier contract and implements it for Gitea and GitLab too, so the new cross-provider SyncManager suite can assert "rename/batch landed as exactly one commit" identically for all three providers instead of casting to a provider-specific verifier type. Also adds E2E_KEEP_BRANCH support to all three provisioners' teardown (skip branch deletion / container removal for debugging a failing run) -- called out in the issue's sandbox-lifecycle acceptance criteria but not yet implemented.
Adds a provider-e2e matrix job (github/gitlab/gitea) to .github/workflows/ci.yml, per the issue's runner-fleet revision -- runs-on: [self-hosted, linux, x64, 32gb-ram], one matrix instead of three hand-written jobs. - `changes` job (dorny/paths-filter) decides whether provider-e2e should run for a given push/PR without gating the whole workflow by path -- on.push.paths/on.pull_request.paths would have also blocked the release-critical CI job for unrelated changes, which this avoids. - Fork PRs only get the Gitea cell (no repo secrets needed/exposed); internal PRs, pushes to main, workflow_dispatch, and the weekly schedule (Monday 06:00 UTC, API-drift detection) get all three. - scripts/run-e2e-ci.mjs wraps scripts/run-e2e.mjs for CI: sweeps stale gfs-e2e-<provider>-* branches first (scripts/e2e-sweep-branches.mjs), and turns a missing required credential into a hard failure rather than a silent skip, for any cell the job-level `if:` already decided should run. - e2e-gate aggregates the matrix (if: always(), success/skipped pass through, anything else fails) and CI now needs it, so a real provider regression blocks the shared CI/semantic-release workflow instead of shipping. E2E_GITLAB_PROJECT_ID is read from secrets, not vars, in this workflow -- confirmed via `gh secret list` that it's configured as a secret (unlike E2E_GITHUB_OWNER/REPO, which are plain vars) on this repo. Verified: scripts/run-e2e-ci.mjs and scripts/e2e-sweep-branches.mjs run correctly against Gitea locally (14/14 passing) and fail explicitly (exit 1, no silent skip) when GitHub credentials are absent; workflow YAML validated with js-yaml. Not verified: actual execution on the self-hosted runner fleet or the full e2e-gate -> CI dependency chain in a real workflow run (no self-hosted runner access from this checkout) -- see docs/testing/real-provider-e2e.md's "Known gaps".
Consolidates local setup, required secrets/vars (cross-checked against what's actually configured on firstsun-dev/git-files-sync via `gh secret list`/`gh variable list`, not just what the issue originally proposed), CI wiring, fork/secrets behavior, release gating, and cleanup/ troubleshooting into one operational doc -- docs/test/github-e2e-plan.md (agent 02) stays as the GitHub-specific implementation notes; this is the cross-provider operational reference. Explicitly lists what's unverified from this environment (GitHub/GitLab SyncManager E2E execution, self-hosted runner behavior, branch-protection required-check setup) rather than leaving it implicit.
GitHub Actions rejects the workflow file with 'Unrecognized named-value: matrix' -- job-level `if:` has no access to the `matrix` context, only step-level `if:` does. The provider-e2e job's `if:` referenced matrix.provider to skip GitHub/GitLab legs on fork PRs and to filter by workflow_dispatch input, which is invalid. Fix: keep only the non-matrix-dependent condition (path-relevance/dispatch/ schedule/main) on the job's own if:, and move the matrix-dependent part into a new 'Determine whether this provider leg should run' step that gates every subsequent step via its output. A gated-off leg's steps are all skipped without failing, so the job (and therefore the matrix as a whole, for the e2e-gate aggregation) still reports success -- same external behavior as originally intended, just relocated to where GitHub Actions actually allows matrix to be read. Also corrected docs/testing/real-provider-e2e.md and the comment in scripts/run-e2e-ci.mjs that described the now-nonexistent job-level if mechanism. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…failures
Investigated the three provider-e2e CI failures from the run right after the
matrix if: fix:
- github: sync-manager.e2e.test.ts's rename/batch tests asserted
listCommitShas(branch).length grew by exactly 1, but the sandbox repo's
main already has 47 commits -- past the API's default page size (30) --
so both the 'before' and 'after' calls silently cap at 30 and the
assertion ("expected 30 to be 31") fails deterministically, not flakily.
Confirmed by querying the real sandbox repo directly. Fixed by comparing
HEAD-before against the two newest commits after (listCommitShas(ref, 2))
instead of full-list length -- exact regardless of total history depth,
matching the pattern github.e2e.test.ts already uses correctly. Verified
against the real sandbox: both previously-failing assertions now pass.
- gitea: Docker container never answered its healthcheck within 60s on the
self-hosted runner. Reproduced locally with the same code (Docker
available here) and it passed cleanly in ~17s -- not a code bug, looks
like a runner-side Docker/network blip. Can't fix infra flakiness from
here, so instead made the next occurrence self-diagnosing: capture the
container's own stdout/stderr on a readiness timeout and attach it to the
thrown error (docker.ts: containerLogsAllowFailure; wired into
gitea-provision.ts's catch), so a future CI failure shows *why* Gitea
never came up instead of just "fetch failed".
- Same teardown TypeError as already fixed in gitlab.e2e.test.ts
(afterAll running adapter.teardown(ctx) with ctx still undefined when
beforeAll fails) also existed in gitea.e2e.test.ts and
sync-manager.e2e.test.ts -- applied the same 'if (ctx)' guard to both.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Type of Change
Engineering Standards
Related Issues