Technical Debt Agent

A propose-only GitHub Actions bot that reviews Java pull requests with fine-tuned GPT models, classifies each changed hunk as a bug, vulnerability, or code smell, and posts one aggregated PR comment with concrete refactor suggestions — without ever touching your code or failing CI.

Problem / Motivation

Generic static analyzers flag technical debt with broad, rule-based heuristics and no explanation of why something matters or what to do about it. Reviewers end up doing that translation manually, PR after PR. This project wires two fine-tuned OpenAI models — one for classification, one for refactor suggestions — into a small autonomous agent that inspects a PR's Java diff, decides for itself which files and tools to invoke next, and reports findings the way a human reviewer would: inline, per-hunk, with a short "what to change and why."

Architecture

flowchart LR
    A["PR labeled td-agent"] --> B["GitHub Actions<br/>td_agent.yml"]
    B --> C["scripts/td_agent_pr.py"]
    C --> D["Agent loop<br/>agent/loop.py + planner.py"]
    D --> E["InspectDiff<br/>GitHub API diff"]
    E --> F["TDAnalyzeFile<br/>td_engine/analyzer.py"]
    F --> G["TDRefactorFile<br/>td_engine/refactorer.py"]
    G --> H["PostPRComment"]
    G --> I["CreateOrUpdateCheck"]
    H --> J["Single aggregated<br/>PR comment"]
    I --> K["Technical Debt Agent check run<br/>status: success"]

Loading

The agent loop is plan → act → observe: on each step a planner LLM (or a deterministic fallback if no planner model is configured) decides which registered tool to call next — InspectDiff, TDAnalyzeFile, TDRefactorFile, RunTests, PostPRComment, or CreateOrUpdateCheck — based on a compact JSON summary of what's been analyzed so far. It stops once every changed Java file has been analyzed and suggestions exist for every detected issue.

Tech Stack

Layer	Tools / Technologies
Trigger	GitHub Actions (pull_request, push), td-agent label gate
Classification LLM	OpenAI fine-tuned GPT-3.5 (TD_CLASSIFICATION_MODEL_ID)
Refactor LLM	OpenAI fine-tuned GPT-3.5 (TD_REFACTOR_MODEL_ID)
Planner LLM	OpenAI GPT-4.1 (TD_PLANNER_MODEL_ID), with a deterministic fallback planner
Agent orchestration	Custom Python plan-act-observe loop (agent/)
GitHub integration	GitHub REST API via requests (diffs, PR comments, check runs)
Target language	Java (.java hunks; sample_java/ fixture for manual testing)
Testing / CI	pytest, ruff, GitHub Actions (ci.yml)

Setup and Run

# 1. Clone
git clone https://github.com/GENAI-CODEDEBT/Code_debt_agentic.git
cd Code_debt_agentic

# 2. Set up environment
python -m venv .venv
source .venv/bin/activate   # or .venv\Scripts\activate on Windows
pip install -e ".[dev]"     # or: pip install -r requirements.txt

# 3. Configure environment variables
cp .env.example .env
# fill in .env with your own OpenAI key and model IDs — see .env.example

Enabling the agent on a real PR

1. Add these as repository secrets (Settings → Secrets and variables → Actions): OPENAI_API_KEY, TD_CLASSIFICATION_MODEL_ID, TD_REFACTOR_MODEL_ID, TD_PLANNER_MODEL_ID. GITHUB_TOKEN is provided automatically. TD_AGENT_TEST_COMMAND is optional and can be left unset.
2. Open a PR that touches .java files and add the td-agent label — the workflow exits immediately without it.
3. Push changes or re-label the PR to re-trigger. The bot analyzes up to MAX_FILES_TO_ANALYZE changed Java files, comments with its findings, and creates a passing Technical Debt Agent check run.

Running the entrypoint locally

export GITHUB_EVENT_PATH=path/to/a/pull_request/event.json   # GitHub webhook payload shape
export GITHUB_EVENT_NAME=pull_request
export GITHUB_SHA=<commit-sha>
python -m scripts.td_agent_pr

Note: there is currently no dry-run flag — a local run still calls the real GitHub API for posting the comment and check run using whatever GITHUB_TOKEN you export, and the classification/refactor stages call the real OpenAI models. Point GITHUB_EVENT_PATH at a real or synthetic PR event payload for the repo/PR you have access to.

Sample Java fixture

sample_java/src/main/java/com/example/TechDebtDemo.java contains intentional issues (hard-coded secret, null dereference, swallowed exception, reference equality) for exercising the workflow end-to-end: branch it, tweak the file, open a PR, add the td-agent label.

Behavior

• Analyzes up to MAX_FILES_TO_ANALYZE (10) changed Java files per PR, capped at MAX_STEPS (6) planner iterations — see agent/config.py.
• Classifies each changed hunk as exactly one of BUG, VULNERABILITY, or CODE_SMELL, with a heuristic severity (MAJOR, CRITICAL, MINOR respectively).
• Generates a 1–3 sentence refactor suggestion per detected issue — explanation only, never a patch.
• Is propose-only and CI-safe by construction: every tool call and the entrypoint's main() catch their own exceptions, and the process always exits 0.

Key Challenges and What I Learned

• Hunk-accurate diff parsing without a diff library. agent/tools.py's _extract_hunks_from_patch walks GitHub's unified-diff patch text manually to recover per-hunk line ranges, so comments land on the exact lines GitHub shows in the PR view.
• Making an LLM-driven loop safe to run unattended in CI. The planner returns free-form JSON, so a malformed or missing response has to degrade to a deterministic fallback (_fallback_plan) rather than crash the workflow — and the entrypoint wraps everything so a model or API failure reports a clean summary instead of a red build.
• Balancing autonomy with a bounded budget. The agent decides its own next tool call, but max_steps and max_files_to_analyze keep a single PR run from running away in cost or latency.

Future Improvements

• Add an evals/ harness that scores classification accuracy against a labeled Java debt dataset
• Implement an actual local dry-run mode that skips the GitHub API calls
• Extend beyond Java to other languages via a pluggable hunk-language detector