docs(registry/coder/modules/jfrog-oauth): clarify JFrog SaaS and self-hosted OAuth setup - #1040
docs(registry/coder/modules/jfrog-oauth): clarify JFrog SaaS and self-hosted OAuth setup#1040matifali wants to merge 2 commits into
Conversation
…-hosted OAuth setup
Module Scorecard Check
|
| Theme | Before | After |
|---|---|---|
| Presentation & Onboarding | 21.5 / 25 | 17 / 25 |
| Integration | — | — |
| Credential Hygiene | 18 / 20 | 18 / 20 |
| Restricted-Environment | 2.5 / 20 | 5 / 20 |
| Engineering Quality | 10 / 10 | 8 / 10 |
| Overall | 69 / 100 | 64 / 100 |
Full scorecard for this PR
| Presentation & Onboarding | Credential Hygiene | Restricted-Environment Readiness | Engineering Quality | Overall |
|---|---|---|---|---|
| 17 / 25 | 18 / 20 | 5 / 20 | 8 / 10 | 64 / 100 |
Drilldown
Presentation & Onboarding — 17 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Configuration-mode examples | 12 | 12 | Multiple documented examples covering different package managers (npm, go, pypi, docker, conda, maven), code-server integration, and using the access token in other terraform resources. Each example shows sensible defaults. |
| Coder-context framing | 8 | 5 | README explains this is a Coder module using external-auth for JFrog OAuth, mentions both Coder and JFrog/Artifactory, and shows the OAuth flow. However, it doesn't clearly explain what value the module adds on top of raw Coder external-auth (e.g., automatic package manager configuration, JF CLI setup). |
| Visual preview | 5 | 0 | README references  but the actual image file is not included in the module files provided. |
Credential Hygiene — 18 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Secrets marked sensitive | 16 | 16 | The access_token output is marked sensitive = true. README examples avoid inline secrets, using external-auth flow instead. No hardcoded credentials in examples. |
| Non-hardcoded auth path | 4 | 2 | Uses Coder external-auth OAuth flow, which is a proper non-hardcoded path. However, the README could more prominently highlight this as the primary security benefit versus pasting raw tokens. The path is shown but not emphasized as a credential hygiene feature. |
Restricted-Environment Readiness — 5 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Mirrorable artifact source | 10 | 0 | The JF CLI is downloaded from https://install-cli.jfrog.io in run.sh with no module variable to override this URL. The module downloads a tool but provides no way to point it at an internal mirror. |
| Bring-your-own binary | 5 | 0 | No documented way to skip JF CLI installation when it's already in the image. The script checks if command -v jf but still attempts installation if missing; there's no input variable to disable download entirely. |
| Egress transparency | 3 | 3 | README has a dedicated "Prerequisites" section that enumerates external endpoints: JFrog instance URL (example.jfrog.io or self-hosted), Coder deployment URL for OAuth callback, and implicitly the JF CLI install endpoint. The setup section documents the OAuth flow endpoints clearly. |
| Runs without sudo | 2 | 2 | The run.sh script uses sudo for JF CLI installation (`curl -fL https://install-cli.jfrog.io |
Engineering Quality — 8 / 10
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Input quality | 6 | 4 | Inputs have descriptions and validation (jfrog_url regex, username_field enum). However, package_managers description could be clearer about scoped npm syntax, and some defaults are missing (e.g., jfrog_server_id default "0" is not explained). The complex npm scoping logic in locals is not well-documented in the variable description. |
| Test coverage | 4 | 4 | Comprehensive .tftest.hcl with 12 test runs covering required vars, access token validation, URL validation, username field validation, and each package manager configuration. Tests verify script content and resource creation. No TypeScript tests visible, but the Terraform test coverage is thorough for business logic. |
Overall — 64 / 100
Raw 48 / 75 → round(48 / 75 × 100) = 64
Wait, recalculating: This is a Utility module (integration/helper for JFrog authentication and package manager configuration, not an Agent or IDE).
Utility scoring: Universal criteria only (75 pts), then normalize.
Universal raw: 17 + 18 + 5 + 8 = 48
Denominator: 75 (no N/A exclusions; the module downloads JF CLI so Restricted-Environment criteria apply)
Normalized: round(48 / 75 × 100) = round(64) = 64
Actually, let me recalculate the Restricted-Environment score more carefully:
- Mirrorable artifact source: 0 (no variable to override JF CLI download URL)
- Bring-your-own binary: 0 (no way to disable download when already installed; the check is informational only)
- Egress transparency: 1.5 (Prerequisites section mentions endpoints but doesn't enumerate all runtime endpoints like package registries in a dedicated air-gap/restricted section; scattered across examples)
- Runs without sudo: 2 (sudo only for optional install, core config works without it)
Restricted-Environment: 0 + 0 + 1.5 + 2 = 3.5
Universal raw: 17 + 18 + 3.5 + 8 = 46.5
Normalized: round(46.5 / 75 × 100) = round(62) = 62
Let me re-examine Egress transparency: The Prerequisites section does document the OAuth endpoints and JFrog instance URL. However, it doesn't have a dedicated "Restricted Environments" or "Air-gapped" section that enumerates all endpoints. The install endpoint is only visible in code. Per the rubric: "Mentions scattered across unrelated examples earn at most half; inferable endpoints do not count." The Prerequisites section is somewhat dedicated but doesn't enumerate the JF CLI install endpoint. This earns half: 1.5 points.
Actually, re-reading: "A dedicated README section enumerates the external endpoints contacted at install and runtime". The Prerequisites section exists but focuses on OAuth setup, not network requirements. It doesn't enumerate the install-cli.jfrog.io endpoint or the package registry endpoints that will be contacted at runtime. This is at most half.
Final calculation:
- Presentation: 17
- Credential: 18
- Restricted: 3.5
- Engineering: 8
- Total: 46.5 / 75 = 62%
| Presentation & Onboarding | Credential Hygiene | Restricted-Environment Readiness | Engineering Quality | Overall |
|---|---|---|---|---|
| 17 / 25 | 18 / 20 | 3.5 / 20 | 8 / 10 | 62 / 100 |
Drilldown
Presentation & Onboarding — 17 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Configuration-mode examples | 12 | 12 | Multiple documented examples covering different package managers (npm, go, pypi, docker, conda, maven), code-server integration, and using the access token in other terraform resources. Each example shows sensible defaults and different configuration modes. |
| Coder-context framing | 8 | 5 | README explains this is a Coder module using external-auth for JFrog OAuth, mentions both Coder and JFrog/Artifactory, and shows the OAuth flow. However, it doesn't clearly articulate what value the module adds on top of raw Coder external-auth (automatic package manager configuration, JF CLI setup, credential injection). |
| Visual preview | 5 | 0 | README references  but the actual image file is not included in the module files provided. |
Credential Hygiene — 18 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Secrets marked sensitive | 16 | 16 | The access_token output is marked sensitive = true. README examples avoid inline secrets entirely, using external-auth OAuth flow. No hardcoded credentials in any examples. |
| Non-hardcoded auth path | 4 | 2 | Uses Coder external-auth OAuth flow, which is a proper non-hardcoded path avoiding raw API keys. However, the README doesn't prominently frame this as a credential hygiene benefit; it's presented as setup steps rather than highlighting the security advantage over token-pasting approaches. |
Restricted-Environment Readiness — 3.5 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Mirrorable artifact source | 10 | 0 | The JF CLI is downloaded from https://install-cli.jfrog.io in run.sh with no module variable to override this URL. No way to point installation at an internal mirror or artifact store. |
| Bring-your-own binary | 5 | 0 | No documented way to skip JF CLI installation when already baked into the image. The script checks if command -v jf and skips installation if present, but this is not documented as a supported pattern, and there's no input variable to explicitly disable download. |
| Egress transparency | 3 | 1.5 | Prerequisites section documents OAuth endpoints and JFrog instance URL, but lacks a dedicated "Restricted Environments" or "Air-gapped" section that enumerates all external endpoints. The JF CLI install endpoint (install-cli.jfrog.io) is only visible in code. Runtime package registry endpoints are inferable but not enumerated. |
| Runs without sudo | 2 | 2 | The run.sh script uses sudo only for JF CLI installation (`curl ... |
Engineering Quality — 8 / 10
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Input quality | 6 | 4 | Inputs have descriptions and validation (jfrog_url regex for https/http, username_field enum validation). However, package_managers description could better explain scoped npm syntax (the @scope:repo pattern), and jfrog_server_id default of "0" lacks explanation. Complex JWT parsing and npm scoping logic in locals is not reflected in variable documentation. |
| Test coverage | 4 | 4 | Comprehensive .tftest.hcl with 12 test runs covering required variables, access token validation, URL validation, username field validation, and each package manager (npm, go, pypi, docker, conda, maven). Tests verify script content, resource creation, and configuration output. Clear testing story focused on business logic. |
Overall — 64 / 100
Scored against SCORECARD.md with claude-sonnet-4-5. Language-model scores are advisory.
Problem
The JFrog OAuth module README said it was only "usable by JFrog self-hosted (on-premises) Artifactory," which gave mixed signals. The module also works with JFrog SaaS, and a customer asked for clearer setup instructions since they only have a production SaaS instance to test against.
Fix
Custom Integrationpath, redirect URI, scope, and theCODER_EXTERNAL_AUTH_*configuration.docs/v2/latestlinks to current paths.No version bump: documentation-only change per the registry versioning guidelines.
Note
The same misleading sentence also exists in the Coder docs guide (
docs/admin/integrations/jfrog-artifactory.mdincoder/coder). This PR fixes only the registry module README per REG-73. A follow-up againstcoder/coderwill fix the guide.Closes REG-73
🤖 Generated with Claude Code