Skip to content

fix(mount): add low-memory sync diagnostics#155

Merged
khaliqgant merged 2 commits into
mainfrom
codex/mount-low-memory-diagnostics
May 14, 2026
Merged

fix(mount): add low-memory sync diagnostics#155
khaliqgant merged 2 commits into
mainfrom
codex/mount-low-memory-diagnostics

Conversation

@khaliqgant
Copy link
Copy Markdown
Member

Summary

  • stream full-pull application instead of retaining a full remote file map
  • stream local scan hashing and only hydrate file contents when a writeback is needed
  • add low-memory mount mode plus pprof and runtime memory logging flags

Notes

  • --low-memory / RELAYFILE_MOUNT_LOW_MEMORY=true omits detailed per-file public state and reduces bulk write batch size.
  • Diagnostics are available with --pprof-addr / RELAYFILE_MOUNT_PPROF_ADDR and --memlog-interval / RELAYFILE_MOUNT_MEMLOG_INTERVAL.
  • The shared 429 blast radius still needs a separate cloud/CLI isolation fix: rate limits should be per user/workspace/token and invalid polling should back off.

Tests

  • go test ./...

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 88b7db56-c90c-414a-b2b3-18711eaa0c9e

📥 Commits

Reviewing files that changed from the base of the PR and between c93ea56 and ba9c28c.

📒 Files selected for processing (2)
  • internal/mountsync/syncer.go
  • internal/mountsync/syncer_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/mountsync/syncer_test.go
  • internal/mountsync/syncer.go

📝 Walkthrough

Walkthrough

This PR adds low-memory mode and runtime diagnostics to the relayfile mount. New flags --low-memory, --pprof-addr, and --memlog-interval are wired into the syncer; local file content reads can be deferred and per-file public-state omitted in low-memory mode, and remote pulls apply files immediately and record visited paths to reduce memory.

Changes

Low-Memory Mount Mode and Runtime Diagnostics

Layer / File(s) Summary
CLI flag definition and wiring
cmd/relayfile-cli/main.go, cmd/relayfile-mount/main.go
Three new flags (--low-memory, --pprof-addr, --memlog-interval) are defined on mount CLIs, added to flag allowlists and help text, and wired into mount config and syncer initialization.
Runtime diagnostics support
internal/mountsync/syncer.go
Imports pprof and implements exported StartDiagnostics(ctx, addr, memlogInterval, logger) to serve /debug/pprof/* and optionally log Go runtime.MemStats on a ticker.
Low-memory mode option plumbing
internal/mountsync/syncer.go
Adds SyncerOptions.LowMemory *bool, resolves it in NewSyncer (flag/env), stores Syncer.lowMemory, and exposes lowMemory in the public JSON state; adjusts thresholds when enabled.
Local snapshot content deferral
internal/mountsync/syncer.go
Adds readLocalSnapshot(path, includeContent) and updates newLocalSnapshot to handle RawContent; callers use hashing-only streaming mode when possible and full reads for writeback paths.
Low-memory public state generation
internal/mountsync/syncer.go
savePublicState skips scanLocalFiles() in low-memory mode, omits per-file publicFileState entries, but preserves aggregate counters (pending/denied/conflict) and includes lowMemory in serialization.
Remote pull memory optimization
internal/mountsync/syncer.go
Refactors full export/full-tree pulls to apply remote files immediately, track visited remotePaths, and perform deletions with applyRemoteSnapshotDeletes instead of buffering full remote snapshot maps; provider materialization derives membership from path sets.
Push/local write callers
internal/mountsync/syncer.go
handleLocalWriteOrCreate and pushLocal now use readLocalSnapshot(..., true) for full-content evaluation when needed.
Low-memory mode tests
internal/mountsync/syncer_test.go
Updates TestScanLocalFilesSkipsSymlinkedDirectories to assert deferred content reads and adds TestLowMemoryPublicStateOmitsPerFileDetails to verify public-state omissions and aggregate counters.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • AgentWorkforce/relayfile#84: Modifies publicState/savePublicState paths (adds failedWritebacks) and overlaps with this PR's state-publication changes.

Poem

🐰 I nudge the syncer, light on paws and feet,
Deferring heavy reads to keep the memory neat.
Remote paths are counted, files applied on the fly,
Pprof hums softly while memstats ripple by.
A rabbit’s little hop makes mounts a bit more sweet.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 5.88% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix(mount): add low-memory sync diagnostics' accurately captures the main changes: adding low-memory mode and diagnostic flags to the mount functionality.
Description check ✅ Passed The description clearly explains the core changes: streaming optimizations, low-memory mode, and diagnostics flags, with details on how to use them.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/mount-low-memory-diagnostics

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 14, 2026

Relayfile Eval Review

Run: .relayfile/evals/runs/2026-05-14T13-14-35-618Z-HEAD-provider
Mode: provider
Git SHA: 1a2e14a

Passed: 4 | Needs human: 0 | Reviewable: 0 | Missing output: 0 | Failed: 0 | Skipped: 0

Human Review Cases

No reviewable human-review cases captured Relayfile output.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 5 additional findings in Devin Review.

Open in Devin Review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Low-memory mode inflates pendingWriteback count for all synced files

In savePublicState, when s.lowMemory is true, currentFiles is an empty map (line 2655). The else if branch at line 2691 fires for every tracked file because currentFiles[remotePath] always returns ok=false. Any tracked file that is not denied, not write-denied, has a non-empty hash, and has fileStatus == "ready" (i.e., a normally synced file) is incorrectly reclassified as "writeback-pending". This inflates the pendingWriteback counter, incorrectly sets HasPendingWriteback: true in the public state flags, and can cause the overall status to show "writeback-pending" instead of "ready". The test at internal/mountsync/syncer_test.go:3121-3126 only seeds a Dirty: true file, so the case tracked.Dirty branch matches before the else if and the bug is not caught.

(Refers to lines 2691-2693)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ba9c28c. In low-memory mode, savePublicState now skips local-drift inference entirely instead of treating every unscanned tracked file as missing/pending. I also extended TestLowMemoryPublicStateOmitsPerFileDetails with a clean tracked file so only the dirty tracked file contributes to pendingWriteback. Verified with go test ./... .

@khaliqgant khaliqgant merged commit ddc2b54 into main May 14, 2026
8 checks passed
@khaliqgant khaliqgant deleted the codex/mount-low-memory-diagnostics branch May 14, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant