Skip to content

[dmt] feat: add remote image linting - #470

Merged
ldmonster merged 14 commits into
mainfrom
feat/add-remote-scope
Sep 4, 2026
Merged

[dmt] feat: add remote image linting#470
ldmonster merged 14 commits into
mainfrom
feat/add-remote-scope

Conversation

@fuldaxxx

@fuldaxxx fuldaxxx commented Aug 31, 2026

Copy link
Copy Markdown
Member

Description

Adds dmt lint remote — linting a module as it was published, not as it sits in a working tree:

dmt lint remote registry.example.com/deckhouse/my-module:v0.0.1 \
  --login <user> --password <pass>

One reference addresses both images a release produces: the bundle at <repo>:<tag> and the release metadata at <repo>/release:<tag>. dmt pulls each, unpacks it into a temporary directory and lints it as a module, printing findings and the summary exactly the way a local run does.

This is built on the scope mechanism from #468. Two scopes join static:

Scope Source Config section
static the committed source tree global.linters-settings
bundle the packaged module image, <repo>:<tag> remote.bundle
release the release metadata image, <repo>/release:<tag> remote.release

One run path, two sources

Where the modules come from is now the only thing that differs between the two commands. internal/manager owns everything after the modules exist — running the linters, printing, statistics, metrics, the --linter filter — behind a manager.Source:

type Source interface {
    ConfigDir() string
    Scopes() []scopes.Scope
    Targets(ctx, *config.RootConfig, *errors.LintRuleErrorsList) ([]Target, error)
    Close()
}

internal/staticlint implements it over a directory (the module discovery, values loading and validateModule moved there out of the manager), internal/remotelint over a registry reference. cmd/dmt's runLint takes a Source and is otherwise the same function for both. What that buys:

  • the remote run gets the framed summary, the --linter filter and the metrics send it did not have — the remote path used to collect findings metrics through IncDmtLinterErrorsCount and then never call Send, which looks like a working run right up until nobody can find its data. Both paths now end at metrics.Flush, which records the config sections the run actually linted with (linters-settings, or remote.bundle + remote.release);
  • a Target is a module and a scope, so a remote run reads one module from two images and the summary still counts one module (ModuleID), while the findings of the two images stay apart (ObjectID carries the scope name);
  • a source failure is reported after the findings, never instead of them.

--fix is rejected for lint remote: the tree is an extracted image thrown away at the end of the run, so a fix would only drop a finding from the report.

Per-scope configuration is independent

The two images carry different files and are linted by different rules, so each gets its own section of .dmtlint.yaml, and nothing from linters-settings reaches a remote scope. A section left out means the built-in severities, not the ones the source tree happens to be tuned to:

remote:
  bundle:
    module:
      rules:
        bundle-layout:
          impact: error
  release:
    module:
      rules:
        release-layout:
          impact: error

Scope.Settings is the only place that mapping lives, and remotelint hands the branch it returns to modules.NewRemoteModule rather than the whole root config — a remote scope has no way to reach the source tree's settings even by accident.

New rules

Both check presence rather than content — presence is a property of the scope, not of the file, so the rules that parse those files keep returning quietly when one is absent:

  • module/release-layoutmodule.yaml, version.json, changelog.yaml in the release image root;
  • module/bundle-layout.helmignore, Chart.yaml, images_digests.json, module.yaml, plus the charts/, docs/, openapi/ and templates/ directories. The list is the intersection of eight published CE bundles, not everything they carry: crds/, hooks/, monitoring/ and .werf/ appear in some and not others, so requiring them would fail modules that legitimately have nothing to put there.

Findings from a remote scope name module-relative paths. The extraction directory is removed once the run is over, so a finding that named the absolute path would point at nothing — docs/readme was changed the same way.

Credentials

--login / --passwordDMT_REGISTRY_LOGIN / DMT_REGISTRY_PASSWORD → Docker config (d8 dk cr login) → anonymous. Each field falls back on its own, so a CI job can keep the login in the pipeline definition and the password in a secret — which is also the reason the environment variables exist: a password passed as a flag lands in the process list and in the job's own command echo.

Invariant the scope tables must keep

A remote module comes from modules.NewRemoteModule, which skips the chart load and the render, so GetChart, GetObjectStore and GetValues are nil there. The release and bundle tables must not ask for a rule that reads them — TestRemoteScopesRunOverAnUnpackedImage runs both scopes over a real unpacked layout to catch a rule ID that cannot survive there, and asserts that no finding names the extraction directory.

go.mod pins docker/cli to v27.1.1 via replace: deckhouse/pkg/registry pulls v29, whose types.AuthConfig no longer converts from docker/docker's registry.AuthConfig, which breaks oras.land/oras-go reached through werf/nelm.

Based on #422 — the --remote flag, the extraction flow and the changelog rule come from there; this PR rebuilds them on top of the scope mechanism, adds the independent per-scope configuration and the layout rules, unifies the two run paths behind manager.Source, and hardens the extraction and the failure paths.

Why do we need it, and what problem does it solve?

Modules ship as OCI artifacts, but dmt could only lint a filesystem checkout. A release pipeline that has the built artifact and nothing else had no way to validate it, and neither did anyone auditing a module already in a registry — the artifact that actually ships was the one thing dmt never looked at.

The gap is not only "the same checks, remotely". A published module is a different shape from its source tree: the bundle is a rendered Helm package with images_digests.json and .helmignore, the release image is metadata Deckhouse reads to decide whether to install a version, and neither looks like the directory the module was built from. Some source-tree checks are meaningless there (markdownlint over rendered docs), and some checks matter only there (changelog.yaml and version.json are optional in a repo and mandatory in a release image). That is why this lands as two scopes with their own rule tables and their own config sections, rather than as a flag that reruns the static pass over an unpacked directory: a packaging mistake — a missing changelog.yaml, a bundle without images_digests.json — is exactly the class of bug that is invisible in the source tree and breaks on install.

@fuldaxxx
fuldaxxx requested review from ipaqsa and ldmonster August 31, 2026 08:15
@fuldaxxx fuldaxxx self-assigned this Aug 31, 2026
@fuldaxxx fuldaxxx added enhancement New feature or request go Pull requests that update go code labels Aug 31, 2026
@fuldaxxx fuldaxxx changed the title feat: add remote image linting [dmt] feat: add remote image linting Sep 1, 2026
fuldaxxx and others added 4 commits September 2, 2026 10:21
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Co-authored-by: Smyslov Maxim <maksim.smyslov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
@fuldaxxx
fuldaxxx force-pushed the feat/add-remote-scope branch from 80e7513 to 36a47d0 Compare September 2, 2026 07:21
@fuldaxxx
fuldaxxx marked this pull request as ready for review September 2, 2026 07:32
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
@fuldaxxx
fuldaxxx marked this pull request as draft September 2, 2026 08:18
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
@fuldaxxx
fuldaxxx marked this pull request as ready for review September 2, 2026 14:16
@fuldaxxx
fuldaxxx marked this pull request as draft September 2, 2026 14:20
Route static and remote sources through the manager for consistent
summaries, metrics, and linter filters. Resolve credentials from flags,
environment variables, Docker config, or anonymous access.

Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
Signed-off-by: Ruslan Gorbunov <ruslan.gorbunov@flant.com>
@fuldaxxx
fuldaxxx marked this pull request as ready for review September 3, 2026 09:10
@ldmonster
ldmonster merged commit 0330f6a into main Sep 4, 2026
8 checks passed
@ldmonster
ldmonster deleted the feat/add-remote-scope branch September 4, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants