Skip to content

feat: add offline secrets and refactor reconciliation - #68

Merged
veerendra2 merged 16 commits into
mainfrom
36-offlinegit-secrets-management
Aug 31, 2026
Merged

feat: add offline secrets and refactor reconciliation#68
veerendra2 merged 16 commits into
mainfrom
36-offlinegit-secrets-management

Conversation

@veerendra2

@veerendra2 veerendra2 commented Aug 22, 2026

Copy link
Copy Markdown
Owner

What

  • Add Age-encrypted local secrets with layered root, stack, and included-directory resolution.
  • Separate local and remote secret integrations into pkg/localsecrets and pkg/remotesecrets.
  • Refactor reconciliation into readable stack discovery, project loading, and dependency tracking flows.
  • Use the Docker Compose SDK model for recursive include and extends dependency discovery.
  • Update CLI configuration and documentation for local and remote secrets.

Why

  • Support offline, Git-managed secrets while preserving selective stack reconciliation.
  • Make the reconciliation code easier to understand and unit test in a follow-up PR.

Validation

  • go test ./... (repository currently has no test files)
  • go vet ./...
  • go build ./...
  • task lint

Closes #36

…nal providers

- Implement in-process age passphrase decryption for *.age files
- Support shared root, stack-specific, and included directory secret files
- Direct in-memory environment injection into compose projects
- Add *.age files to git diff dependency change detection
- Add deprecation warning for external secrets managers
- Update documentation and build ldflags

Closes #36
@veerendra2 veerendra2 linked an issue Aug 22, 2026 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Secrets are currently injected after Compose project load (breaking ${VAR} interpolation/back-compat) and shared Age secrets can be skipped on external-provider errors, both of which can cause incorrect deployments.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds offline, in-repo secrets support by introducing an Age (filippo.io/age) decryption client and integrating secrets loading into the reconciler flows (sync/health/image updates), while deprecating external secrets providers (Bitwarden/Infisical) and updating documentation accordingly.

Changes:

  • Added pkg/agesecrets for decrypting *.age dotenv files (armored or binary) using a passphrase.
  • Integrated shared + stack/included-directory Age secrets into reconciliation and change detection; external secrets providers marked deprecated.
  • Updated CLI/config plumbing and docs to reflect Age secrets usage and deprecation messaging.
File summaries
File Description
Taskfile.yml Adds -s -w to build ldflags.
README.md Updates feature table to highlight Age secrets + deprecation notice.
pkg/secretsmanager/client.go Renames package and deprecates provider help text.
pkg/secretsmanager/bitwarden.go Renames package to secretsmanager.
pkg/secretsmanager/infisical.go Renames package to secretsmanager.
pkg/agesecrets/age.go New Age decryption + dotenv parsing implementation.
mkdocs.yml Adds Age secrets guide; marks old provider guides deprecated.
internal/reconcile/update.go Loads shared secrets and loads projects via secrets-aware loader.
internal/reconcile/sync.go Loads shared secrets; tracks *.age in dependency file paths.
internal/reconcile/reconcile.go Adds ageClient to reconciler construction.
internal/reconcile/prune.go Switches to loadStackConfig naming.
internal/reconcile/health.go Loads shared secrets and loads projects via secrets-aware loader.
internal/reconcile/discover.go Adds shared/stack/include Age secrets loading + env injection.
internal/reconcile/config.go Renames config loader to loadStackConfig (envs/startup_order only).
go.mod Adds filippo.io/age dependency.
go.sum Adds checksums for filippo.io/age and transitive deps.
docs/Introduction.md Updates sync steps and dependency list to include Age secrets.
docs/index.md Updates docs landing page feature table for Age secrets.
docs/how-to-guides/AgeSecrets.md New Age secrets setup/how-to guide.
docs/how-to-guides/Bitwarden.md Marks Bitwarden guide deprecated; points to Age.
docs/how-to-guides/Infisical.md Marks Infisical guide deprecated; points to Age.
docs/GettingStarted.md Adds AGE_PASSPHRASE docs and deprecates external providers.
cmd/composeflux/common.go Wires Age config into CLI and constructs the age client; logs deprecation warning.
Review details
  • Files reviewed: 22/23 changed files
  • Comments generated: 6
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/reconcile/discover.go Outdated
Comment thread internal/reconcile/discover.go Outdated
Comment thread internal/reconcile/discover.go Outdated
Comment thread internal/reconcile/discover.go Outdated
Comment thread pkg/agesecrets/age.go Outdated
Comment thread docs/how-to-guides/AgeSecrets.md Outdated
- Precompute and cache scrypt identity in age client to avoid repeated derivation
- Continue loading root age secrets when external secrets provider fails
- Add nil guard for ageClient in decryptAgeEnvs
- Remove placeholder comment in discover.go
- Clarify direct environment injection in age secrets docs
@veerendra2 veerendra2 added the build Builds dev docker image label Aug 22, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Secret-loading error handling and included-compose interpolation behavior currently allow partial/incorrect project loads that can break redeploy-on-secret-change and produce deployments with missing secrets.

Review details

Suppressed comments (5)

Previously missed (4) — in code that hasn't changed since the last review.

internal/reconcile/sync.go:70

  • loadSharedSecrets() failure is only logged and ignored, which can lead to deploying stacks without required shared secrets (e.g., when root *.age files exist but AGE_PASSPHRASE is missing/incorrect). This should be treated as a fatal error for the sync run so deployments don’t proceed with incomplete env.
	// Load shared secrets (external secrets manager + root *.age files)
	sharedAgeEnvs, sharedAgeFiles, err := r.loadSharedSecrets()
	if err != nil {
		slog.Warn("Failed to load shared secrets", "error", err)
	}

internal/reconcile/update.go:30

  • UpdateImages() continues after failing to load shared secrets, which can cause stacks to be skipped or evaluated with incomplete configuration. Consider failing fast here so image update checks don’t run against partially configured projects.
	// Load shared secrets (external secrets manager + root *.age files)
	sharedAgeEnvs, _, err := r.loadSharedSecrets()
	if err != nil {
		slog.Warn("Failed to load shared secrets for image updates", "error", err)
	}

internal/reconcile/health.go:39

  • Health reconcile proceeds after shared secrets load failure, which can cause repeated deploy attempts with missing required env (and possibly flap health checks). Treat failure to load root *.age secrets as fatal for this reconcile pass.
		// Load shared secrets (external secrets manager + root *.age files)
		sharedAgeEnvs, _, err := r.loadSharedSecrets()
		if err != nil {
			slog.Warn("Failed to load shared secrets for health reconcile", "error", err)
		}

internal/reconcile/discover.go:177

  • Root *.age file paths are built from r.gClient.Path() without normalizing to an absolute path. GitSync’s change detection compares dependency paths against an absolute repoPath, so a relative clone path can cause sharedAgeFiles to be treated as “outside the repo” and skipped, breaking redeploy-on-secret-change.
	// 2. Discover and decrypt root *.age files
	stacksRootDir := filepath.Join(r.gClient.Path(), r.stackPath)
	mergedSecrets, rootAgeFiles, err := r.decryptAgeEnvs(stacksRootDir, sharedSecrets)

internal/reconcile/discover.go:249

  • Included-directory *.age secrets are discovered only after the initial LoadProject() completes, so any ${VAR} interpolation inside included compose files won’t see those secrets. Reload the project once after discovering included-dir secrets so interpolation and path resolution are consistent with the decrypted env set.
	// Scan included compose file directories
	for _, composeFile := range project.ComposeFiles {
		dir := filepath.Dir(composeFile)
		if _, seen := seenDirs[dir]; seen {
			continue
  • Files reviewed: 22/23 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new include-directory age secret scan uses filepath.Dir(composeFile) without normalizing relative compose file paths, which can lead to scanning the wrong directory for secrets.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 22/23 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread internal/reconcile/discover.go Outdated
@veerendra2 veerendra2 changed the title feat(secrets): add Age offline secrets management feat: add offline secrets and refactor reconciliation Aug 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Dependency tracking has a confirmed path-normalization bug for optional relative env_file entries that can cause incorrect warnings and change-detection behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 38/40 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread internal/reconcile/dependencies.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Compose source-walk cache key currently retains plaintext secret values in-memory (via string concatenation) and should be changed to avoid persisting decrypted secrets.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

pkg/localsecrets/age.go:36

  • newAgeClient discards the error from age.NewScryptIdentity(passphrase). If identity initialization can fail (e.g., due to invalid scrypt params in future versions), ComposeFlux would only fail later at decrypt time (and potentially repeatedly), instead of failing fast during startup. Prefer propagating this error up through localsecrets.New(...) so misconfiguration is caught early.
  • Files reviewed: 42/44 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread internal/reconcile/sources.go
@veerendra2
veerendra2 marked this pull request as ready for review August 30, 2026 19:47
@veerendra2
veerendra2 merged commit b2ffa91 into main Aug 31, 2026
2 checks passed
@veerendra2
veerendra2 deleted the 36-offlinegit-secrets-management branch August 31, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Builds dev docker image

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Offline(Git) secrets management

2 participants