diff --git a/CHANGELOG.md b/CHANGELOG.md index d21675d..708ea77 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,23 @@ +## [1.5.6](https://github.com/firstsun-dev/git-files-sync/compare/1.5.5...1.5.6) (2026-08-07) + +### Bug Fixes + +* **e2e:** correct gitlab suite against real GitLab.com behavior ([77a3355](https://github.com/firstsun-dev/git-files-sync/commit/77a33552dc74f6a82c46862a37986db5ae89c93a)), closes [#101](https://github.com/firstsun-dev/git-files-sync/issues/101) [#101](https://github.com/firstsun-dev/git-files-sync/issues/101) + +### Documentation + +* **e2e:** write GitHub E2E test plan ([3cef03e](https://github.com/firstsun-dev/git-files-sync/commit/3cef03e962fb5d073a9a24aafee1cdefcd620b07)) + +## [1.5.5](https://github.com/firstsun-dev/git-files-sync/compare/1.5.4...1.5.5) (2026-08-07) + +### Bug Fixes + +* **provider:** preserve sync correctness ([ce61588](https://github.com/firstsun-dev/git-files-sync/commit/ce61588e7db9085dc6c8fdbda6b964734e33fa5a)) + +### Documentation + +* tighten token permission guidance to least-privilege scopes ([#116](https://github.com/firstsun-dev/git-files-sync/issues/116)) ([45dc39d](https://github.com/firstsun-dev/git-files-sync/commit/45dc39d28de5fbb80a1b89b43a35fbff2b98f198)) + ## [1.5.4](https://github.com/firstsun-dev/git-files-sync/compare/1.5.3...1.5.4) (2026-08-07) ### Bug Fixes diff --git a/README.md b/README.md index 3520ba5..7467ca8 100644 --- a/README.md +++ b/README.md @@ -63,13 +63,17 @@ A single dashboard shows the state of every tracked file: | | Provider | Required info | Token scope | |:---:|---|---|---| -| GitHub | **GitHub** | Personal access token, owner, repo name | `repo` | -| GitLab | **GitLab** | Personal access token, project ID, base URL | `api` | -| Gitea | **Gitea** | Personal access token, base URL, owner, repo name | (all) | +| GitHub | **GitHub** | Personal access token, owner, repo name | `Contents: Read and write` (fine-grained); `repo` (classic) | +| GitLab | **GitLab** | Personal access token, project ID, base URL | `read_repository`, `write_repository` | +| Gitea | **Gitea** | Personal access token, base URL, owner, repo name | `write:repository` (1.19+); account-wide on older versions | -- **GitHub token:** Settings → Developer settings → Personal access tokens → `repo` scope. -- **GitLab token:** User settings → Access tokens → `api` scope. Base URL defaults to `https://gitlab.com`; change it for self-hosted instances. -- **Gitea token:** User settings → Applications → Access tokens. Point the base URL at your instance (e.g. `https://gitea.example.com`). +> **Security tip:** scope every token as narrowly as possible — one repo, the minimum permissions, and a short expiration — and store it only in this plugin's settings. Never paste it into a note that gets synced. Rotate it immediately if it's ever exposed, and revoke tokens you're no longer using. + +- **GitHub token:** create a [fine-grained personal access token](https://github.com/settings/personal-access-tokens/new) (Settings → Developer settings → Personal access tokens → Fine-grained tokens) rather than a classic one. Set **Repository access** to *Only select repositories* and pick just the repo you're syncing, set an **Expiration** (90 days or less), and grant only **Contents: Read and write**. If you must use a classic token, limit the `repo` scope to that one use and set an expiration. +- **GitLab token:** prefer a [project access token](https://docs.gitlab.com/user/project/settings/project_access_tokens/) (Project → Settings → Access tokens) over a personal one — it's scoped to a single project and can be revoked without affecting your account. The plugin only calls the repository tree/blobs/commits/branches endpoints, so grant just `read_repository` and `write_repository` (**not** `api`, which also grants issues, merge requests, CI, and account-wide access). Role **Developer** is the minimum that can push to a non-protected branch. Set an expiration date, and base URL defaults to `https://gitlab.com`; change it for self-hosted instances. +- **Gitea token:** User settings → Applications → Access tokens. The plugin only touches repository content, branches, and git data, so on Gitea 1.19+ (which supports per-scope tokens) select just **`write:repository`** — that implies read access too — instead of "Select all". Older Gitea versions (down to the 1.12 minimum) don't support scoped tokens, so the token is account-wide by default; in that case, use a dedicated bot/service account with access to only the target repo rather than your personal account's token. Set an expiration if your instance offers one, and point the base URL at your instance (e.g. `https://gitea.example.com`). + +All three providers let you revoke a token instantly from its settings page — do that first if a token may have leaked, then issue a new one. Other settings: **language** (system default, English, Traditional Chinese, or Simplified Chinese); **auto-refresh Sync Status on startup**; **branch** to sync against (default `main`); **root path** prefix inside the repo; **vault folder** to scope which notes are tracked; and **symbolic link handling** (*real* — recreate the link, GitHub only; *follow* — sync the target's content; *skip*). See [Symbolic link handling](docs/symlink-handling.md) for details. diff --git a/USAGE_zh-cn.md b/USAGE_zh-cn.md index f9de707..7ba6971 100644 --- a/USAGE_zh-cn.md +++ b/USAGE_zh-cn.md @@ -34,9 +34,14 @@ Git File Sync 不会同步整个 vault；您可以只选择要分享、发布或 1. **选择服务**：在 `设置` > `Git File Sync` 中选择 GitLab、GitHub 或 Gitea。 2. **填写凭据**： - - **GitHub**：需要个人访问令牌 (PAT)、用户名或组织名、仓库名。令牌需要 `repo` 权限。 - - **GitLab**：需要个人访问令牌 (PAT)、项目 ID、服务器网址（默认为 `https://gitlab.com`；自建实例请改为对应网址）。令牌需要 `api` 权限。 - - **Gitea**：需要个人访问令牌 (PAT)、服务器网址（例如 `https://gitea.example.com`）、用户名或组织名、仓库名。可在 `用户设置` > `应用程序` > `访问令牌` 中创建令牌。 + + > **安全提示：** 请把每个令牌的权限范围缩到最小：只允许需要同步的仓库、只授予必要权限，并设置较短的有效期。令牌只应保存在本插件的设置中，不要粘贴到会被同步的笔记里。如果令牌可能泄露，请立即撤销并重新创建；不再使用的令牌也应直接撤销。 + + - **GitHub**：建议创建 [fine-grained personal access token](https://github.com/settings/personal-access-tokens/new)，而不是 classic token。在 **Repository access** 中选择 *Only select repositories*，只指定要同步的仓库；设置 **Expiration**（建议不超过 90 天），并只授予 **Contents: Read and write**。如果必须使用 classic token，则使用 `repo` scope，并为该用途设置到期时间。 + - **GitLab**：建议优先使用 [project access token](https://docs.gitlab.com/user/project/settings/project_access_tokens/)（`Project` > `Settings` > `Access tokens`），而不是个人访问令牌，使权限限制在单个项目内，也可以独立撤销。插件只会调用 repository tree、blob、commit 和 branch 相关 API，因此只需要 `read_repository` 和 `write_repository`，**不需要** `api`。如需推送到非 protected branch，最低角色为 **Developer**。请设置到期时间；服务器网址默认为 `https://gitlab.com`，自建环境请改为您的实例网址。 + - **Gitea**：在 `用户设置` > `应用程序` > `访问令牌` 中创建令牌。插件只会操作仓库内容、分支和 Git data；Gitea 1.19+ 支持 scoped token，请只选择 **`write:repository`**（已包含读取权限），不要选择全部权限。较旧版本（最低支持到 1.12）没有 scoped token，令牌默认是账号级别；这种情况下建议使用只拥有目标仓库权限的专用 bot / service account，而不是个人账号。若实例支持到期时间也请设置，并将服务器网址指向您的 Gitea 实例（例如 `https://gitea.example.com`）。 + + 三种服务都可以从各自的设置页面立即撤销令牌。如果令牌可能泄露，请先撤销，再签发新的令牌。 3. **仓库路径**：如需把笔记存放在仓库中的特定目录（例如 `notes/`），请设置 `Root Path`。 4. **语言和自动刷新**：可以选择跟随系统、English、繁體中文或简体中文。“Obsidian 启动时自动刷新同步状态”默认开启，也可在设置中关闭。 @@ -104,5 +109,5 @@ Git File Sync 不会同步整个 vault；您可以只选择要分享、发布或 ## 🔒 隐私与安全 -- 个人访问令牌只保存在本地 vault 的插件数据目录中，只会发送给您配置的 Git 服务。 +- 个人访问令牌只保存在本地 vault 的插件数据目录中，只会发送到您配置的 Git 服务。 - 插件不收集使用数据或分析信息。 diff --git a/USAGE_zh.md b/USAGE_zh.md index d5619ae..83eae03 100644 --- a/USAGE_zh.md +++ b/USAGE_zh.md @@ -36,9 +36,14 @@ 1. **選擇服務**：在 `設定` > `Git File Sync` 中選擇 GitLab、GitHub 或 Gitea。 2. **填寫憑證**： - - **GitHub**：需要 個人存取權杖 (PAT)、帳號名稱、儲存庫名稱。權杖需具備 `repo` 權限。 - - **GitLab**：需要 個人存取權杖 (PAT)、專案 ID、伺服器網址（預設為 `https://gitlab.com`，自架請改為您的網址）。權杖需具備 `api` 權限。 - - **Gitea**：需要 個人存取權杖 (PAT)、伺服器網址（例如 `https://gitea.example.com`）、帳號名稱、儲存庫名稱。權杖在 `使用者設定` > `應用程式` > `存取權杖` 中建立。 + + > **安全性提示：** 請將每個權杖的範圍縮到最小：只允許需要同步的儲存庫、只授予必要權限，並設定較短的有效期限。權杖只應儲存在此外掛的設定中，不要貼到會被同步的筆記。如果權杖可能外洩，請立即撤銷並重新建立；不再使用的權杖也應直接撤銷。 + + - **GitHub**：建議建立 [fine-grained personal access token](https://github.com/settings/personal-access-tokens/new)，而不是 classic token。在 **Repository access** 選擇 *Only select repositories*，只指定要同步的儲存庫；設定 **Expiration**（建議 90 天以內），並只授予 **Contents: Read and write**。若必須使用 classic token，則使用 `repo` scope，並為此用途設定到期日。 + - **GitLab**：建議優先使用 [project access token](https://docs.gitlab.com/user/project/settings/project_access_tokens/)（`Project` > `Settings` > `Access tokens`），而不是個人存取權杖，讓權限限制在單一專案，也能獨立撤銷。外掛只會呼叫 repository tree、blob、commit 與 branch 相關 API，因此只需要 `read_repository` 與 `write_repository`，**不需要** `api`。若要推送到非 protected branch，最低角色需為 **Developer**。請設定到期日；伺服器網址預設為 `https://gitlab.com`，自架環境請改成您的實例網址。 + - **Gitea**：在 `使用者設定` > `應用程式` > `存取權杖` 建立權杖。外掛只會操作儲存庫內容、分支與 Git data；Gitea 1.19+ 支援 scoped token，請只選 **`write:repository`**（已包含讀取權限），不要選擇全部權限。較舊版本（最低支援至 1.12）沒有 scoped token，權杖預設為帳號層級；這種情況建議使用只具備目標儲存庫權限的專用 bot / service account，而不是個人帳號。若實例支援到期日也請設定，並將伺服器網址指向您的 Gitea 實例（例如 `https://gitea.example.com`）。 + + 三種服務都可以從各自的設定頁立即撤銷權杖。如果權杖可能外洩，請先撤銷，再簽發新的權杖。 3. **儲存庫路徑**：如果您想將筆記存放在儲存庫的特定資料夾（例如 `notes/`），請在 `Root Path` 中設定。 4. **語言與自動重新整理**：可選擇跟隨系統、English、繁體中文或简体中文；「Obsidian 啟動時自動重新整理同步狀態」預設開啟，亦可在設定中關閉。 @@ -108,5 +113,5 @@ ## 🔒 隱私與安全 -- 您的存取權杖 (Token) 僅會儲存在您本機的 Obsidian 資料夾內，不會傳送至任何第三方伺服器。 -- 本插件不會收集任何個人數據或使用紀錄。 +- 您的存取權杖 (Token) 僅會儲存在本機 vault 的外掛資料目錄中，只會傳送到您設定的 Git 服務。 +- 本外掛不會收集任何個人資料或使用紀錄。 diff --git a/manifest.json b/manifest.json index 0d1dec2..98708d9 100644 --- a/manifest.json +++ b/manifest.json @@ -1,7 +1,7 @@ { "id": "git-file-sync", "name": "Git File Sync", - "version": "1.5.4", + "version": "1.5.6", "minAppVersion": "1.11.0", "description": "Selectively sync individual notes with GitLab or GitHub. Push, pull, diff, and resolve conflicts — file by file, on mobile and desktop.", "author": "ClaudiaFang", diff --git a/package.json b/package.json index dda3210..77e1bb8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "git-file-sync", - "version": "1.5.4", + "version": "1.5.6", "description": "Selectively sync individual notes with GitLab or GitHub. Push, pull, diff, and resolve conflicts — file by file, on mobile and desktop.", "main": "main.js", "type": "module", diff --git a/src/logic/sync-manager.ts b/src/logic/sync-manager.ts index 5708064..7799a03 100644 --- a/src/logic/sync-manager.ts +++ b/src/logic/sync-manager.ts @@ -22,7 +22,7 @@ type PlanClassification = { kind: 'addition' | 'modification' | 'move' | 'unchan type ToPushEntry = { path: string; name: string; repoPath: string; content: string | ArrayBuffer; existingSha?: string; existingRevision?: string }; /** A renamed file classified as a safe move, queued for the grouped batch-commit call. */ -type ToMoveEntry = { path: string; name: string; repoPath: string; oldPath: string; oldRepoPath: string; content: string | ArrayBuffer }; +type ToMoveEntry = { path: string; name: string; repoPath: string; oldPath: string; oldRepoPath: string; content: string | ArrayBuffer; oldRevision?: string }; /** * Result of a batch push. `syncedPaths` lists every path that's now confirmed @@ -682,6 +682,7 @@ export class SyncManager { } const treeEntry = treeByFullPath.get(this.getFullPathForTree(repoPath)); + await this.migrateGitLabLegacyBaseline(path, repoPath, treeEntry); const outcome = await this.classifyAgainstTreeEntry(path, content, treeEntry, true); if (outcome === 'queued') return { kind: treeEntry ? 'modification' : 'addition' }; // classifyAgainstTreeEntry's dry-run path only ever resolves to @@ -714,7 +715,7 @@ export class SyncManager { if (!renamedFrom) return undefined; const scratch: ToMoveEntry[] = []; - const outcome = this.queueMove(path, name, renamedFrom, content, treeByFullPath, scratch); + const outcome = await this.queueMove(path, name, renamedFrom, content, treeByFullPath, scratch); return outcome === 'queued' ? { kind: 'move', movedFrom: renamedFrom } : { kind: 'conflict' }; } @@ -742,6 +743,7 @@ export class SyncManager { const localSha = await gitBlobSha(await this.getFileContent(fileOrPath)); if (localSha === entry.sha) return 'unchanged'; + await this.migrateGitLabLegacyBaseline(path, repoPath, entry); const lastSynced = this.settings.syncMetadata[path]; if (lastSynced && entry.sha !== lastSynced.lastSyncedSha) return 'conflict'; return 'modification'; @@ -891,15 +893,18 @@ export class SyncManager { const trackedOldPath = this.settings.syncMetadata[path]?.renamedFrom; const renamedFrom = trackedOldPath ?? (hasOrphans ? await this.detectRename(fileOrPath, content, treeByFullPath) : null); if (renamedFrom) { - return this.queueMove(path, name, renamedFrom, content, treeByFullPath, toMove); + return await this.queueMove(path, name, renamedFrom, content, treeByFullPath, toMove); } } - const treeEntry = treeByFullPath.get(this.getFullPathForTree(repoPath)); + let treeEntry = treeByFullPath.get(this.getFullPathForTree(repoPath)); + await this.migrateGitLabLegacyBaseline(path, repoPath, treeEntry); + const revision = await this.refreshGitLabBatchRevision(repoPath, treeEntry); + if (revision) treeEntry = { ...treeEntry!, sha: revision.sha }; const outcome = await this.classifyAgainstTreeEntry(path, content, treeEntry); if (outcome !== 'queued') return outcome; - toPush.push({ path, name, repoPath, content, existingSha: treeEntry?.sha }); + toPush.push({ path, name, repoPath, content, existingSha: treeEntry?.sha, existingRevision: revision?.revision }); return 'queued'; } @@ -912,28 +917,50 @@ export class SyncManager { * silently deleted. Both surface as 'conflict' so the batch can't quietly * clobber either side the way a plain content push already refuses to. */ - private queueMove( + private async queueMove( path: string, name: string, oldPath: string, content: string | ArrayBuffer, treeByFullPath: Map, toMove: ToMoveEntry[] - ): BatchOutcome | 'queued' { + ): Promise { const repoPath = this.getNormalizedPath(path); const oldRepoPath = this.getNormalizedPath(oldPath); if (treeByFullPath.get(this.getFullPathForTree(repoPath))) return 'conflict'; - const oldEntry = treeByFullPath.get(this.getFullPathForTree(oldRepoPath)); + let oldEntry = treeByFullPath.get(this.getFullPathForTree(oldRepoPath)); + await this.migrateGitLabLegacyBaseline(oldPath, oldRepoPath, oldEntry); + const oldRevision = await this.refreshGitLabBatchRevision(oldRepoPath, oldEntry); + if (oldRevision) oldEntry = { ...oldEntry!, sha: oldRevision.sha }; const metadata = this.settings.syncMetadata[path] ?? this.settings.syncMetadata[oldPath]; const safeToDeleteOld = !oldEntry?.sha || !metadata?.lastSyncedSha || oldEntry.sha === metadata.lastSyncedSha; if (oldEntry?.sha && !safeToDeleteOld) return 'conflict'; - toMove.push({ path, name, repoPath, oldPath, oldRepoPath, content }); + toMove.push({ path, name, repoPath, oldPath, oldRepoPath, content, oldRevision: oldRevision?.revision }); return 'queued'; } + /** GitLab tree rows expose blob identity but not the commit revision needed + * for optimistic locking. Read it during planning and compare the fresh blob + * again before accepting the action; the stored revision then protects the + * interval between planning and the atomic commit. */ + private async refreshGitLabBatchRevision(repoPath: string, entry: GitTreeEntry | undefined): Promise<{ sha: string; revision?: string } | undefined> { + if (this.settings.serviceType !== 'gitlab' || !entry?.sha) return undefined; + const remote = await this.gitService.getFile(repoPath, this.settings.branch); + return remote.sha ? { sha: remote.sha, revision: remote.revision } : undefined; + } + + /** Migrates a legacy GitLab last_commit_id baseline only when the current + * file endpoint proves it still describes this tree blob. */ + private async migrateGitLabLegacyBaseline(path: string, repoPath: string, entry: GitTreeEntry | undefined): Promise { + const metadata = this.settings.syncMetadata[path]; + if (this.settings.serviceType !== 'gitlab' || !metadata?.lastSyncedSha || !entry?.sha || entry.sha === metadata.lastSyncedSha) return; + const remote = await this.gitService.getFile(repoPath, this.settings.branch); + if (remote.sha === entry.sha && remote.revision === metadata.lastSyncedSha) await this.updateMetadata(path, remote.sha); + } + /** * Decides a non-symlink, non-renamed file's outcome purely from a * pre-fetched tree entry and a locally-computed git blob sha — no network @@ -1060,7 +1087,7 @@ export class SyncManager { try { const commitMessage = `Push ${chunk.length} file(s) from Obsidian`; const batchResults = await this.gitService.pushBatch!( - chunk.map(f => ({ path: f.repoPath, content: f.content, existedRemotely: !!f.existingSha })), + chunk.map(f => ({ path: f.repoPath, content: f.content, existedRemotely: !!f.existingSha, revision: f.existingRevision })), this.settings.branch, commitMessage ); @@ -1106,8 +1133,8 @@ export class SyncManager { const commitMessage = this.combinedChunkCommitMessage(pushEntries.length, moveEntries.length); const batchResults = await this.gitService.commitBatch!( - pushEntries.map(f => ({ path: f.repoPath, content: f.content, existedRemotely: !!f.existingSha })), - moveEntries.map(f => ({ oldPath: f.oldRepoPath, newPath: f.repoPath, content: f.content })), + pushEntries.map(f => ({ path: f.repoPath, content: f.content, existedRemotely: !!f.existingSha, revision: f.existingRevision })), + moveEntries.map(f => ({ oldPath: f.oldRepoPath, newPath: f.repoPath, content: f.content, oldRevision: f.oldRevision })), this.settings.branch, commitMessage ); @@ -1249,6 +1276,8 @@ export class SyncManager { return 'unchanged'; } + await this.migrateGitLabLegacyBaseline(path, this.getNormalizedPath(path), entry); + // Same conflict check as the content path below: local differs and the // remote has moved since we last synced, so pulling would discard one of // the two changes. diff --git a/src/services/git-service-base.ts b/src/services/git-service-base.ts index f922529..1442351 100644 --- a/src/services/git-service-base.ts +++ b/src/services/git-service-base.ts @@ -1,6 +1,7 @@ import { requestUrl, RequestUrlResponse } from 'obsidian'; import { logger } from '../utils/logger'; import { GitTreeEntry } from './git-service-interface'; +import { isBinaryPath } from '../utils/path'; export interface GitFile { content: string | ArrayBuffer; @@ -195,18 +196,6 @@ export abstract class BaseGitService { return cleanRoot + path; } - protected isBinary(path: string): boolean { - const ext = path.split('.').pop()?.toLowerCase(); - if (!ext) return false; - const BINARY_EXTENSIONS = new Set([ - 'png', 'jpg', 'jpeg', 'gif', 'bmp', 'ico', 'pdf', 'zip', 'gz', '7z', 'rar', - 'mp3', 'mp4', 'wav', 'ogg', 'webm', 'mov', 'avi', 'wmv', 'webp', - 'doc', 'docx', 'xls', 'xlsx', 'ppt', 'pptx', 'epub', 'exe', 'dll', 'so', - 'ttf', 'woff', 'woff2', 'eot', 'wasm', 'dmg', 'iso' - ]); - return BINARY_EXTENSIONS.has(ext); - } - protected encodeContent(content: string | ArrayBuffer): string { if (typeof content === 'string') { const bytes = new TextEncoder().encode(content); @@ -235,7 +224,7 @@ export abstract class BaseGitService { bytes[i] = cp !== undefined ? cp : 0; } - return this.isBinary(path) ? bytes.buffer : new TextDecoder().decode(bytes); + return isBinaryPath(path) ? bytes.buffer : new TextDecoder().decode(bytes); } /** diff --git a/src/services/git-service-interface.ts b/src/services/git-service-interface.ts index 619cbc4..e3d1015 100644 --- a/src/services/git-service-interface.ts +++ b/src/services/git-service-interface.ts @@ -37,6 +37,8 @@ export interface BatchPushItem { * commit ignores it (a tree entry upserts regardless). */ existedRemotely?: boolean; + /** Revision read during batch planning, used by GitLab's optimistic lock. */ + revision?: string; } /** Result for one file after a batch push completes. */ @@ -53,6 +55,8 @@ export interface BatchMoveItem { /** Path relative to rootPath, where the file now lives. */ newPath: string; content: string | ArrayBuffer; + /** Revision of oldPath read during batch planning, used by GitLab's optimistic lock. */ + oldRevision?: string; } export interface GitServiceInterface { diff --git a/src/services/gitea-service.ts b/src/services/gitea-service.ts index f41a79f..43a44b7 100644 --- a/src/services/gitea-service.ts +++ b/src/services/gitea-service.ts @@ -1,6 +1,5 @@ import { GitServiceInterface, GitTreeEntry, BatchPushItem, BatchPushResult, BatchMoveItem } from './git-service-interface'; import { BaseGitService, ConnectionTestResult, GitFile, GitHubContentResponse, GitHubTreeResponse, GIT_SYMLINK_MODE } from './git-service-base'; -import { logger } from '../utils/logger'; /** One entry in a Gitea "change multiple files" request. */ interface GiteaChangeFileOperation { @@ -145,9 +144,7 @@ export class GiteaService extends BaseGitService implements GitServiceInterface const treeResponse = await this.safeRequest(treeUrl, 'GET'); const treeData = this.parseJson(treeResponse); - if (treeData.truncated) { - logger.warn('Gitea tree result is truncated. Some files might not be shown.'); - } + if (treeData.truncated) throw new Error(`Gitea tree for branch "${branch}" is truncated; sync stopped to avoid treating an incomplete remote tree as a snapshot.`); const entries = treeData.tree .filter(item => item.type === 'blob') diff --git a/src/services/github-service.ts b/src/services/github-service.ts index 8da3cb5..3476d0a 100644 --- a/src/services/github-service.ts +++ b/src/services/github-service.ts @@ -1,6 +1,5 @@ import { GitServiceInterface, GitTreeEntry, BatchPushItem, BatchPushResult, BatchMoveItem } from './git-service-interface'; import { BaseGitService, ConnectionTestResult, GitFile, GitHubContentResponse, GitHubTreeResponse, GIT_SYMLINK_MODE, BLOB_CREATE_CONCURRENCY } from './git-service-base'; -import { logger } from '../utils/logger'; import { PushTimingCollector, PushTimingHandler, PushTimingRecord } from './push-timing'; /** @@ -109,6 +108,10 @@ export class GitHubService extends BaseGitService implements GitServiceInterface } async pushFile(path: string, content: string | ArrayBuffer, branch: string, message: string, _existingSha?: string, _revision?: string): Promise<{ path: string, sha?: string }> { + const entry = (await this.listFilesDetailed(branch, false)).find(item => item.path === this.getFullPath(path)); + if (entry?.symlink) { + throw new Error(`Cannot overwrite symlink "${path}" with a regular file.`); + } const [result] = await this.pushBatch([{ path, content }], branch, message); return result ?? { path }; } @@ -314,9 +317,7 @@ export class GitHubService extends BaseGitService implements GitServiceInterface throw this.branchNotFoundError(e, branch); } - if (data.truncated) { - logger.warn('GitHub tree result is truncated. Some files might not be shown.'); - } + if (data.truncated) throw new Error(`GitHub tree for branch "${branch}" is truncated; sync stopped to avoid treating an incomplete remote tree as a snapshot.`); const entries = data.tree .filter(item => item.type === 'blob') diff --git a/src/services/gitlab-service.ts b/src/services/gitlab-service.ts index 430d013..514370d 100644 --- a/src/services/gitlab-service.ts +++ b/src/services/gitlab-service.ts @@ -1,5 +1,6 @@ import { GitServiceInterface, GitTreeEntry, BatchPushItem, BatchPushResult, BatchMoveItem } from './git-service-interface'; import { BaseGitService, ConnectionTestResult, GitFile, GitLabFileResponse, GitLabTreeItem, GIT_SYMLINK_MODE } from './git-service-base'; +import { isBinaryPath } from '../utils/path'; export class GitLabService extends BaseGitService implements GitServiceInterface { private baseUrl: string = 'https://gitlab.com'; @@ -61,12 +62,13 @@ export class GitLabService extends BaseGitService implements GitServiceInterface const encodedProjectId = encodeURIComponent(this.projectId); const url = `${this.baseUrl}/api/v4/projects/${encodedProjectId}/repository/commits`; - const actions = items.map(item => ({ + const actions = await Promise.all(items.map(async item => ({ action: item.existedRemotely ? 'update' : 'create', file_path: this.getFullPath(item.path), content: this.encodeContent(item.content), encoding: 'base64', - })); + ...(item.existedRemotely && item.revision ? { last_commit_id: item.revision } : {}), + }))); await this.safeRequest(url, 'POST', { branch, commit_message: message, actions }); @@ -89,19 +91,21 @@ export class GitLabService extends BaseGitService implements GitServiceInterface const url = `${this.baseUrl}/api/v4/projects/${encodedProjectId}/repository/commits`; const actions = [ - ...additions.map(item => ({ + ...await Promise.all(additions.map(async item => ({ action: item.existedRemotely ? 'update' : 'create', file_path: this.getFullPath(item.path), content: this.encodeContent(item.content), encoding: 'base64', - })), - ...moves.map(item => ({ + ...(item.existedRemotely && item.revision ? { last_commit_id: item.revision } : {}), + }))), + ...await Promise.all(moves.map(async item => ({ action: 'move', file_path: this.getFullPath(item.newPath), previous_path: this.getFullPath(item.oldPath), content: this.encodeContent(item.content), encoding: 'base64', - })), + ...(item.oldRevision ? { last_commit_id: item.oldRevision } : {}), + }))), ]; await this.safeRequest(url, 'POST', { branch, commit_message: message, actions }); @@ -160,7 +164,7 @@ export class GitLabService extends BaseGitService implements GitServiceInterface const encodedProjectId = encodeURIComponent(this.projectId); const url = `${this.baseUrl}/api/v4/projects/${encodedProjectId}/repository/blobs/${sha}/raw`; const response = await this.safeRequest(url, 'GET'); - const content = this.isBinary(path) ? response.arrayBuffer : response.text; + const content = isBinaryPath(path) ? response.arrayBuffer : response.text; return { content, sha }; } @@ -179,7 +183,11 @@ export class GitLabService extends BaseGitService implements GitServiceInterface const encodedProjectId = encodeURIComponent(this.projectId); const url = `${this.baseUrl}/api/v4/projects/${encodedProjectId}/repository/commits`; - const actions = paths.map(path => ({ action: 'delete', file_path: this.getFullPath(path) })); + const actions = await Promise.all(paths.map(async path => ({ + action: 'delete', + file_path: this.getFullPath(path), + last_commit_id: (await this.getFile(path, branch)).revision, + }))); await this.safeRequest(url, 'POST', { branch, commit_message: message, actions }); } diff --git a/tests/logic/sync-manager-batch.test.ts b/tests/logic/sync-manager-batch.test.ts index 5eafa5d..b34edc5 100644 --- a/tests/logic/sync-manager-batch.test.ts +++ b/tests/logic/sync-manager-batch.test.ts @@ -148,6 +148,27 @@ describe('SyncManager Batch Operations', () => { ]); }); + it('reads and forwards GitLab revision for an existing batch update', async () => { + const path = 'locked.md'; + const adapter = mockApp.vault.adapter as Mocked; + mockSettings.serviceType = 'gitlab'; + mockSettings.syncMetadata = { + [path]: { lastSyncedSha: 'remote-blob', lastSyncedAt: 0, lastKnownPath: path } + }; + vi.mocked(adapter.exists).mockResolvedValue(true); + vi.mocked(adapter.read).mockResolvedValue('local edit'); + vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([{ path, symlink: false, sha: 'remote-blob' }]); + vi.mocked(mockGitService.getFile).mockResolvedValue({ content: 'remote original', sha: 'remote-blob', revision: 'remote-commit' }); + mockGitService.pushBatch = vi.fn().mockResolvedValue([{ path, sha: 'new-blob' }]); + + const results = await manager.pushAllFiles([path]); + + expect(results.success).toBe(1); + expect(mockGitService.pushBatch).toHaveBeenCalledWith([ + { path, content: 'local edit', existedRemotely: true, revision: 'remote-commit' }, + ], 'main', expect.any(String)); + }); + it('reports syncedPaths via the sequential fallback when the provider has no pushBatch', async () => { const files = ['a.md', 'b.md']; const adapter = mockApp.vault.adapter as Mocked; @@ -319,6 +340,25 @@ describe('SyncManager Batch Operations', () => { expect(adapter.write).not.toHaveBeenCalled(); }); + it('migrates a legacy GitLab last_commit_id baseline instead of creating a false pull conflict', async () => { + const path = 'legacy.md'; + const adapter = mockApp.vault.adapter as Mocked; + mockSettings.serviceType = 'gitlab'; + mockSettings.syncMetadata = { + [path]: { lastSyncedSha: 'legacy-last-commit', lastSyncedAt: 0, lastKnownPath: path } + }; + vi.mocked(adapter.exists).mockResolvedValue(true); + vi.mocked(adapter.read).mockResolvedValue('local old copy'); + vi.mocked(mockGitService.listFilesDetailed).mockResolvedValue([{ path, symlink: false, sha: 'remote-blob' }]); + vi.mocked(mockGitService.getFile).mockResolvedValue({ content: 'remote current copy', sha: 'remote-blob', revision: 'legacy-last-commit' }); + + const results = await manager.pullAllFiles([path]); + + expect(results.conflicts).toBe(0); + expect(results.success).toBe(1); + expect(mockSettings.syncMetadata[path]?.lastSyncedSha).toBe('remote-blob'); + }); + it('still downloads when the local file differs and the remote has not moved', async () => { const path = 'stale.md'; mockSettings.syncMetadata = { diff --git a/tests/services/git-service-base.test.ts b/tests/services/git-service-base.test.ts index 58c0b96..597a55c 100644 --- a/tests/services/git-service-base.test.ts +++ b/tests/services/git-service-base.test.ts @@ -177,6 +177,8 @@ describe('BaseGitService', () => { it('should correctly encode and decode UTF-8 content', async () => { const original = 'Hello, 世界! 🌍'; vi.mocked(requestUrl) + // Tree mode check prevents a regular-file push from replacing a symlink. + .mockResolvedValueOnce({ status: 200, json: { tree: [{ path: 'test.md', type: 'blob', mode: '100644' }] } } as unknown as RequestUrlResponse) // Branch head for expectedHeadOid, read over GraphQL. .mockResolvedValueOnce({ status: 200, json: { data: { repository: { ref: { target: { oid: 'commit1' } } } } } } as unknown as RequestUrlResponse) .mockResolvedValueOnce({ status: 200, json: { data: { createCommitOnBranch: { commit: { oid: 'commit2' } } } } } as unknown as RequestUrlResponse); @@ -185,7 +187,7 @@ describe('BaseGitService', () => { await service.pushFile('test.md', original, 'main', 'test'); const calls = vi.mocked(requestUrl).mock.calls; - const body = JSON.parse((calls[1]?.[0] as { body: string }).body) as { variables: { input: { fileChanges: { additions: Array<{ contents: string }> } } } }; + const body = JSON.parse((calls[2]?.[0] as { body: string }).body) as { variables: { input: { fileChanges: { additions: Array<{ contents: string }> } } } }; const decoded = atob(body.variables.input.fileChanges.additions[0]?.contents.replace(/\s/g, '') ?? ''); const bytes = new Uint8Array(decoded.length); for (let i = 0; i < decoded.length; i++) { diff --git a/tests/services/gitea-service.test.ts b/tests/services/gitea-service.test.ts index e97f007..a55ce07 100644 --- a/tests/services/gitea-service.test.ts +++ b/tests/services/gitea-service.test.ts @@ -205,12 +205,9 @@ describe('GiteaService', () => { expect(await service.listFiles('main', false)).toEqual(['vault/file1.md', 'other/file2.md']); }); - it('should log warning and return files when result is truncated', async () => { + it('fails closed when the tree result is truncated', async () => { mockListFiles([{ path: 'file1.md', type: 'blob' }], true); - const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); - const result = await service.listFiles('main'); - expect(result).toEqual(['file1.md']); - warnSpy.mockRestore(); + await expect(service.listFiles('main')).rejects.toThrow(/truncated; sync stopped/); }); it('should throw a message naming the branch when the branch is not found', async () => { diff --git a/tests/services/github-service.test.ts b/tests/services/github-service.test.ts index b115bed..7dd1af5 100644 --- a/tests/services/github-service.test.ts +++ b/tests/services/github-service.test.ts @@ -369,17 +369,25 @@ describe('GitHubService', () => { describe('pushFile', () => { it('commits one regular file through GraphQL instead of the Contents API', async () => { vi.mocked(requestUrl) + .mockResolvedValueOnce({ status: 200, json: { tree: [{ path: 'note.md', type: 'blob', mode: '100644' }] } } as unknown as RequestUrlResponse) .mockResolvedValueOnce(headOidResponse('commit1')) .mockResolvedValueOnce({ status: 200, json: { data: { createCommitOnBranch: { commit: { oid: 'commit2' } } } } } as unknown as RequestUrlResponse); await expect(service.pushFile('note.md', 'new content', 'main', 'update', 'old-sha')).resolves.toEqual({ path: 'note.md' }); const calls = vi.mocked(requestUrl).mock.calls.map(call => call[0] as RequestUrlParam); - expect(calls).toHaveLength(2); - expect(calls[1]?.url).toBe('https://api.github.com/graphql'); - const body = JSON.parse(calls[1]?.body as string) as { variables: { input: { fileChanges: { additions: Array<{ path: string; contents: string }> } } } }; + expect(calls).toHaveLength(3); + expect(calls[2]?.url).toBe('https://api.github.com/graphql'); + const body = JSON.parse(calls[2]?.body as string) as { variables: { input: { fileChanges: { additions: Array<{ path: string; contents: string }> } } } }; expect(body.variables.input.fileChanges.additions).toEqual([{ path: 'note.md', contents: btoa('new content') }]); }); + + it('refuses to turn an in-repository symlink into a regular file', async () => { + mockRequest({ status: 200, json: { tree: [{ path: 'link.md', type: 'blob', mode: '120000' }] } }); + + await expect(service.pushFile('link.md', 'replacement', 'main', 'update')).rejects.toThrow(/Cannot overwrite symlink/); + expect(requestUrl).toHaveBeenCalledTimes(1); + }); }); describe('listFiles', () => { @@ -423,15 +431,12 @@ describe('GitHubService', () => { expect(await service.listFiles('main')).toEqual(['src/content/index.md']); }); - it('should return files and log warning when result is truncated', async () => { + it('fails closed when the tree result is truncated', async () => { mockRequest({ status: 200, json: { truncated: true, tree: [ { path: 'file1.md', type: 'blob' }, { path: 'file2.md', type: 'blob' }, ] } }); - const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); - const result = await service.listFiles('main'); - expect(result).toEqual(['file1.md', 'file2.md']); - warnSpy.mockRestore(); + await expect(service.listFiles('main')).rejects.toThrow(/truncated; sync stopped/); }); it('should throw a message naming the branch when the branch is not found', async () => { diff --git a/tests/services/gitlab-service.test.ts b/tests/services/gitlab-service.test.ts index 045d41e..0923d9e 100644 --- a/tests/services/gitlab-service.test.ts +++ b/tests/services/gitlab-service.test.ts @@ -251,7 +251,10 @@ describe('GitLabService', () => { }); it('posts a Commits API actions array with action: delete, no content/encoding', async () => { - mockRequest({ status: 201, json: { id: 'commit-sha' } }); + vi.mocked(requestUrl) + .mockResolvedValueOnce({ status: 200, json: { content: btoa('a'), blob_id: 'a-blob', last_commit_id: 'revision-a' } } as unknown as RequestUrlResponse) + .mockResolvedValueOnce({ status: 200, json: { content: btoa('b'), blob_id: 'b-blob', last_commit_id: 'revision-b' } } as unknown as RequestUrlResponse) + .mockResolvedValueOnce({ status: 201, json: { id: 'commit-sha' } } as unknown as RequestUrlResponse); await service.deleteBatch(['a.md', 'b.md'], 'main', 'Delete 2 file(s) from Obsidian'); @@ -262,8 +265,8 @@ describe('GitLabService', () => { expect(body.branch).toBe('main'); expect(body.commit_message).toBe('Delete 2 file(s) from Obsidian'); expect(body.actions).toEqual([ - { action: 'delete', file_path: 'a.md' }, - { action: 'delete', file_path: 'b.md' }, + { action: 'delete', file_path: 'a.md', last_commit_id: 'revision-a' }, + { action: 'delete', file_path: 'b.md', last_commit_id: 'revision-b' }, ]); }); }); @@ -275,6 +278,27 @@ describe('GitLabService', () => { expect(requestUrl).not.toHaveBeenCalled(); }); + it('sends last_commit_id for existing updates and moves', async () => { + vi.mocked(requestUrl) + .mockResolvedValueOnce({ status: 201, json: { id: 'commit-sha' } } as unknown as RequestUrlResponse) + .mockResolvedValueOnce({ status: 200, json: [ + { path: 'existing.md', type: 'blob', id: 'updated-blob' }, + { path: 'new.md', type: 'blob', id: 'moved-blob' }, + ] } as unknown as RequestUrlResponse); + + await service.commitBatch( + [{ path: 'existing.md', content: 'new', existedRemotely: true, revision: 'update-revision' }], + [{ oldPath: 'old.md', newPath: 'new.md', content: 'moved', oldRevision: 'move-revision' }], + 'main', 'locked batch' + ); + + const body = JSON.parse((vi.mocked(requestUrl).mock.calls[0]?.[0] as { body: string }).body) as { actions: Array<{ action: string; last_commit_id?: string }> }; + expect(body.actions).toEqual([ + expect.objectContaining({ action: 'update', last_commit_id: 'update-revision' }), + expect.objectContaining({ action: 'move', last_commit_id: 'move-revision' }), + ]); + }); + it('uses the Commits API native action: move for renames, alongside create/update for plain additions', async () => { vi.mocked(requestUrl) .mockResolvedValueOnce({ status: 201, json: { id: 'commit-sha' } } as unknown as RequestUrlResponse) // POST commits diff --git a/versions.json b/versions.json index 7b264a8..0f153e0 100644 --- a/versions.json +++ b/versions.json @@ -14,5 +14,7 @@ "1.5.0": "1.11.0", "1.5.1": "1.11.0", "1.5.3": "1.11.0", - "1.5.4": "1.11.0" + "1.5.4": "1.11.0", + "1.5.5": "1.11.0", + "1.5.6": "1.11.0" } \ No newline at end of file