Skip to content

[FEATURE] Early validation of prerequisites and external payloads #129

Description

@koydas

🎯 Goal

Fail fast with explicit, actionable errors when prerequisites are missing or external payloads are malformed — before any costly LLM call is made.

📍 Context

  • Repo: autonomous-dev-loop
  • Domain: Reliable fail-fast behavior
  • Component: scripts/lib/config.mjs, scripts/lib/prompts.mjs, scripts/auto_fix_pr.mjs

🚀 Description

Startup validation is partially in place but incomplete:

  • requireEnv() in config.mjs validates required secrets — good.
  • Prompt files (prompts/auto-fix-system.md, prompts/generation-user.md, etc.) are loaded on demand inside prompts.mjs with no existence check before the LLM call is attempted.
  • The GitHub event payload (GITHUB_EVENT_PATH) is parsed but its structure is not validated — a missing pull_request.number field is caught, but other required fields are not checked upfront.
  • LLM response payloads are parsed with a try/catch but error messages do not indicate which field failed or why.

The fix is a single validateStartup() call at the top of each entrypoint script, plus a validatePayload(schema, data) helper used at external boundaries.

🧩 Scope

In:

  • scripts/lib/config.mjs: add validateStartup() that checks all required secrets AND verifies that required prompt files exist on disk before returning.
  • scripts/lib/prompts.mjs: on loadPrompt(name), throw immediately with the file path if the file does not exist (currently lets readFileSync throw a generic ENOENT).
  • scripts/auto_fix_pr.mjs: validate the GitHub event payload structure after parsing:
    • Required: pull_request.number (already checked), pull_request.head.ref, repository (already via requireEnv).
  • LLM response validation: when parseJsonResponse fails, include the field path that was missing/invalid in the error message.
  • All validation errors must be classified as PERMANENT (once error taxonomy from issue [FEATURE] Unified TRANSIENT/PERMANENT/UNKNOWN error taxonomy #109 is in place).

Out:

  • Full redesign of global config format.
  • Runtime schema library (no ajv or similar — plain JS checks only).

🧪 Acceptance criteria

  • Functional

    • If a required secret is missing, the process exits with a clear message naming the variable before any network call is made.
    • If a prompt file does not exist, the process exits with the file path in the error message before the LLM call.
    • If the GitHub event payload is missing pull_request.number, the process exits immediately with a message identifying the missing field.
  • Edge cases

    • A prompt file that exists but is empty (0 bytes) is treated as invalid and throws with the file path.
    • A GitHub event payload that is valid JSON but not an object (e.g., null, array) is caught and throws with a descriptive message.
    • A truncated LLM response (valid JSON but missing content[0].text) produces an error message that names the missing path, not a generic "unexpected format".
  • Tests

    • Unit: validateStartup() throws with the variable name when ANTHROPIC_API_KEY is unset.
    • Unit: loadPrompt('nonexistent') throws with the resolved file path.
    • Unit: loadPrompt('empty-file') throws indicating the file is empty.
    • Unit: parseJsonResponse on a response missing content[0].text produces an error message containing content[0].text.
    • Unit: GitHub event payload null is rejected with a descriptive error, not a TypeError.

⚙️ Constraints

  • Do not hide root causes behind generic messages — every error must name the specific missing/invalid field or file.
  • No new runtime dependencies — plain fs.existsSync / fs.statSync for file checks.
  • Validation must complete synchronously before any await (so failures are caught before the first network call)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-devIssue validated and ready for automated implementation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions