Skip to content

fix: split Harness name and version in persona analytics (CLI-1771) - #7160

Closed
nick-y-snyk wants to merge 2 commits into
snyk:mainfrom
nick-y-snyk:fix/CLI-1771-harness-name-version-split
Closed

fix: split Harness name and version in persona analytics (CLI-1771)#7160
nick-y-snyk wants to merge 2 commits into
snyk:mainfrom
nick-y-snyk:fix/CLI-1771-harness-name-version-split

Conversation

@nick-y-snyk

@nick-y-snyk nick-y-snyk commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • persona.agent fused Claude Code's name, version, and role into one string (claude-code_2-1-233_agent), fragmenting it into ~101 distinct values in analytics and making "how many Claude Code sessions" impossible to query. This splits name and version into two fields.
  • Replaces the hand-ported detect-agent signature table (cliv2/internal/persona/agent) with the upstream github.com/vercel/detect-agent module — grows coverage 13 → 20 harnesses and fixes a live bug where plain-CLI OpenCode was undetected (our port matched the launcher's env var, not the one OpenCode itself sets).
  • Adds a harness-agnostic regex that splits a version-shaped suffix off any detected name (name[_/]version[_/role]), and canonicalises the remaining fragment against detect-agent's own KnownAgents vocabulary rather than a hand-written alias list.
  • New persona.agent_version is written only when a version is found; persona.agent always resolves to claude_code for Claude Code regardless of whether it was detected via AI_AGENT or via environment signature.
  • cursor-cli, codex, and every other already-clean harness are unaffected — no version-shaped suffix, so the regex doesn't match and the value passes through unchanged.

Exact emitted string, flagging ahead of release per the spec: Claude Code will report as claude_code (upstream's spelling), not claude-code.

Datadog: where the new field lands

Confirmed against live analytics-service logs (service:analytics-service "analytics payload"). Today:

custom.analytics-service.interaction.extension.persona.agent = "claude-code_2-1-220_harness"

After this change, agent_version is a new sibling key at the same nesting level, queryable as @analytics-service.interaction.extension.persona.agent_version:

persona:
    agent: claude_code
    agent_version: "2.1.220"
    interactive: false
    interactive_mode: 0

agent_version is omitted from the payload entirely (not an empty string) whenever no version is parsed out.

Blocker — needs OSS/legal sign-off before merge

github.com/vercel/detect-agent has no LICENSE file in its repo (confirmed via the GitHub API), despite tagged releases and external contributors. This needs sign-off before the dependency lands.

Every third-party Go dependency in cliv2 ships its license text embedded in the CLI binary: cliv2/scripts/prepare_licenses.go runs go-licenses save ./... and writes one LICENSE (or NOTICE/COPYING) file per module path under cliv2/internal/embedded/_data/licenses/, which cliv2/internal/embedded/file.go pulls into the binary via //go:embed _data and surfaces to end users as the OSS notices. This step runs as part of make configure in cliv2/Makefile.

Ran that script locally against this branch to see what it does with detect-agent: it doesn't fail, it silently skips it. Exit 0, no warning naming the package, and no github.com/vercel/detect-agent/ folder is created under internal/embedded/_data/licenses/ (confirmed via git status — zero diff in that tree). So this isn't a build break that would get caught in CI; without this callout, detect-agent's code would ship inside the CLI binary with no bundled license text at all, and nothing in the existing pipeline would flag it.

The script does have a manual-override path for cases go-licenses can't auto-detect (github.com/davecgh/go-spew, github.com/alexbrainman/sspi, github.com/pmezard/go-difflib, go.dev are each fetched from a hardcoded URL) — but those are cases where a license exists upstream and go-licenses just can't introspect it. detect-agent has no license text anywhere to point that mechanism at, so that path doesn't apply here.

For legal's context: this looks like a fixable oversight, not an unknown-license situation. github.com/vercel/detect-agent's package.json explicitly declares "license": "Apache-2.0"go-licenses (and every other Go license tool) only reads an actual LICENSE/COPYING/NOTICE file, so it can't see that declaration, and the repo has never committed one. Checked for an existing upstream issue/PR about it — there is none.

If sign-off is refused (or upstream doesn't add a LICENSE), the fallback is to keep the hand-port, apply the OpenCode fix directly, and implement the split/canonicalisation against a locally-maintained name list — the split logic itself is unaffected by that choice.

Test plan

  • go test ./... passes in cliv2 and cliv2-private
  • go vet ./... and golangci-lint run clean on changed packages
  • go mod tidy -diff clean in both cliv2 and cliv2-private
  • New test coverage in cliv2/internal/persona/persona_test.go (TestReport_Agent) at the single seam where persona values are recorded onto analytics — extends the existing fake-analytics-recorder test rather than a new harness, using t.Setenv per case (the upstream module reads the process environment directly, no injection point). Covers every real-world shape observed in production: fused underscore (both _agent and _harness roles), legacy slash format normalising to the same bucket, bare signature detection, an already-clean harness, a fused identifier with no version-shaped segment, an identifier that fails to canonicalise, a redacted (***) identifier, and a version-shaped run with no trailing role.
  • Verified against 14 days of production analytics-service logs: no harness other than Claude Code currently reports a version-shaped persona.agent value at any real volume, confirming the ticket's Out-of-Scope call.
  • Manual: after merge, re-run the distinct-persona.agent-value count and confirm Claude Code's ~101 buckets collapse to one, with persona.agent_version populated on those events.

Ref: CLI-1771

persona.agent fused Claude Code's name, version, and role into one string
(e.g. claude-code_2-1-233_agent), fragmenting it into ~96 distinct values in
analytics and making a simple "how many Claude Code sessions" query
impossible. Replace the hand-ported detect-agent signature table with the
upstream github.com/vercel/detect-agent module (13 -> 20 harnesses, also
fixes a live OpenCode detection bug), and split any version-shaped suffix
off the detected name into a new persona.agent_version field, canonicalising
the remaining name fragment against the module's own vocabulary rather than
a hand-written alias list. persona.agent now always reports "claude_code"
for Claude Code regardless of detection path.

Needs OSS/legal sign-off before merge: the upstream module ships no LICENSE
file (see PR description).
@snyk-io

snyk-io Bot commented Aug 20, 2026

Copy link
Copy Markdown

⚠️ Snyk checks are incomplete.

Status Scan Engine Critical High Medium Low Total (0)
⚠️ Open Source Security 0 0 0 0 See details
⚠️ Licenses 0 0 0 0 See details
Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@nick-y-snyk
nick-y-snyk marked this pull request as ready for review August 20, 2026 14:59
@nick-y-snyk
nick-y-snyk requested a review from a team as a code owner August 20, 2026 14:59
@snyk-pr-review-bot

Copy link
Copy Markdown

PR Reviewer Guide 🔍

🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Breaking Change 🟠 [major]

The canonical name for Claude Code is changed from claude-code to claude_code to match the upstream detect-agent module. This will break existing analytics queries, dashboards, and alerts that filter on the string claude-code. Ensure downstream consumers are updated before or immediately after this lands.

func SplitVersion(raw string) (name string, version string) {
	m := versionSuffix.FindStringSubmatch(raw)
	if m == nil {
		return raw, ""
	}

	fragment := m[versionSuffix.SubexpIndex("name")]
	canonical, ok := knownNames[normalize(fragment)]
	if !ok {
		return raw, ""
	}

	return canonical, strings.ReplaceAll(m[versionSuffix.SubexpIndex("version")], "-", ".")
}
Compliance Risk 🟠 [major]

The PR introduces github.com/vercel/detect-agent as a dependency. As noted in the PR description, this repository currently lacks a LICENSE file. Per standard OSS compliance, this is a blocker until legal sign-off is obtained or the upstream maintainer adds a license (typically MIT or Apache 2.0).

github.com/vercel/detect-agent v1.2.0
📚 Repository Context Analyzed

This review considered 17 relevant code sections from 11 files (average relevance: 0.85)

🤖 Repository instructions applied (from AGENTS.md)

@nick-y-snyk

Copy link
Copy Markdown
Contributor Author

Superseded by #7162, opened directly from snyk/cli instead of the fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant