Skip to content

Refactor tool infrastructure, per-request LLM configuration, and add LLM resilience - #17

Merged
Hemdan47 merged 8 commits into
mainfrom
dev
Jun 24, 2026
Merged

Refactor tool infrastructure, per-request LLM configuration, and add LLM resilience#17
Hemdan47 merged 8 commits into
mainfrom
dev

Conversation

@Hemdan47

Copy link
Copy Markdown
Collaborator

This PR introduces significant architectural improvements to the agent's tool infrastructure, LLM configuration, and error-handling resilience, enhancing flexibility, robustness, correctness, and maintainability.

Changes

Migrate to per-request LLM configuration via LiteLLM

  • Removed global config.py LLM configuration.
  • Introduced llm/factory.py with dynamic model instantiation per request.
  • Added llm/helpers.py with utility functions for LLM operations.
  • Updated API schemas to support per-request model configuration.
  • Modified health check endpoints to reflect new architecture.

Features

Add centralized Tenacity-based retry policy for LLM resilience

  • Introduced a custom _ResilientRunnable wrapper using Tenacity @retry to handle LLM execution errors.
  • Centralized shared Layer-2 tenacity configuration in retry_policy(), which handles retryable exceptions, exponential backoff with jitter, and emitting transient llm.error events.
  • Applied the centralized retry_policy to wrap the ReAct loop in the analyzer subgraph (run_analyzer), ensuring standard retries and notifications cover the entire agent invocation.

Replace hardcoded skip-lists with .gitignore-based filtering

  • Added new ignore_filter.py module using pathspec library for .gitignore parsing.
  • Glob, grep, and list tools now respect .gitignore patterns automatically.
  • Removes maintenance burden of hardcoded directory/file exclusions.
  • Always skips .git directories regardless of .gitignore settings.
  • Comprehensive test coverage for ignore filtering logic.

Configurable working directory for file tools

  • Converted glob, grep, and list tools to BaseTool classes.
  • Added working_directory parameter for flexible path scoping.
  • Improved tool consistency and reusability across different contexts.

LLM resilience and targeted exception handling

  • Implemented targeted LLM exception handling to make failures self-heal and visible instead of crashing opaquely.
  • Introduced classification of LLM failures into actionable categories (e.g., rate limits, bad requests, authentication errors, timeouts, service unavailable).
  • Introduced a new "llm.error" event to broadcast transient and fatal LLM errors to downstream clients.
  • Guarded I/O boundaries against system/disk issues (e.g., writer disk failures, glob/grep errors).
  • Added warning filter in pyproject.toml to ignore cosmetic Pydantic serialization warnings.

Bug Fixes

Prevent duplicate title emissions in planner

  • Fixed state management issues causing repeated title broadcasts.
  • Ensures cleaner streaming output during planning phase.

Classify mutating git subcommands correctly

  • Bash tool now properly categorizes git commands like add, commit, push as modify operations.
  • Improves permission request accuracy for git operations.

Graph Recursion Error Handling

  • Caught GraphRecursionError in the runner and emitted a clear "stuck in loop" event message (run.failed with step budget warning) instead of a generic crash.

Testing

  • Added 15 new unit test cases for the LLM factory (test_factory.py).
  • Added 7 new unit tests for the ignore filter functionality (test_ignore_filter.py).
  • Expanded unit tests for the glob, grep, and list tools to validate working_directory scoping, .gitignore filtering, and edge-case error handling (e.g., dangling symlinks and malformed regex).
  • Added unit tests for tenacity retry execution paths, event hooks, and fatal error propagation (test_resilience.py).
  • Introduced tests/integration/helpers.py to unify LLM config resolution and provider key extraction.
  • Refactored planner, generator, analyzer, and stream router integration tests to use the new unified helper, eliminating duplicated boilerplate.
  • Updated integration tests for streaming routes, planner, and analyzer.

Dependencies

  • Added pathspec dependency for .gitignore pattern matching (pyproject.toml).

Hemdan47 added 8 commits June 22, 2026 04:18
…gurable working_directory

Bug: These tools defaulted to os.getcwd() at call time, causing them to search the
agent's repo instead of the target project when the process cwd was set to the agent
directory. BashTool didn't have this issue because it already used the BaseTool class
pattern with injected working_directory.

Changes:
- Convert glob_tool, grep_tool, list_tool from @tool functions to BaseTool subclasses
- Add create_glob_tool(), create_grep_tool(), create_list_tool() factories
- Store working_directory as instance attribute; use as default when path omitted
- Update analyzer.py to construct tools with working_directory=project_path
Bug: The planner node emitted the title event but failed to persist it to state when branching to analyzer (Branch 1) or human interaction (Branch 2). On subsequent executions, Branch 3 would unconditionally overwrite the title field, potentially causing duplicate emissions or title changes during replans.

Fix: Track when title is newly generated with a flag and conditionally include it in state updates only for that execution. This ensures the title is emitted once, persisted immediately, and never overwritten in subsequent planner calls.
Problem:
Agent was coupled to environment variables for LLM config, preventing
per-request model selection and requiring server restart for changes.
Custom models only worked with OpenAI provider.

Solution:
- Replace env-based config with per-request LLMConfig schema
- Route all providers through LiteLLM (openai, anthropic, gemini, groq, openai_compatible)
- Add shared get_llm_from_config() helper in llm/helpers.py
- Update nodes to accept RunnableConfig and extract llm_config
- Remove langchain-{groq,google-genai,openai}, add langchain-litellm
…ring via pathspec

Bug: grep/glob/list tools had inconsistent hardcoded skip directories
(e.g., .pytest-cache hyphen vs .pytest_cache underscore in list). Glob had
no skip logic at all, walking into .git and .pytest-tmp unnecessarily.
Fix: shared _ignore.py module using pathspec.GitIgnoreSpec to honor
.gitignore semantics. Only .git is hardcoded because git never lists itself.
Make LLM failures self-heal and visible instead of crashing opaquely:
two-layer retry (silent LiteLLM num_retries + visible LangChain
.with_retry), error classification into actionable categories, and a
new "llm.error" event. Also guard I/O boundaries (writer disk failures,
glob/grep errors) and catch GraphRecursionError as a "stuck in loop"
event. Fixes two runtime import bugs and adds unit tests for the new code.
…acity policy.

- Replace LangChain's `.with_retry()` with a custom `_ResilientRunnable` wrapper using Tenacity `@retry`.
- Centralize shared Layer-2 tenacity configuration in `retry_policy()`, which handles retryable exceptions, exponential backoff/jitter, and emitting transient `llm.error` events.
- Apply the centralized `retry_policy` to wrap the ReAct loop in the analyzer subgraph (`run_analyzer`), ensuring standard retries and notifications cover the entire agent invocation.
- Introduce `tests/integration/helpers.py` to unify LLM config resolution and provider key extraction.
- Refactor planner, generator, analyzer, and stream router integration tests to use the new unified helper, eliminating duplicated boilerplate.
- Update unit tests in `test_resilience.py` to assert Tenacity retry execution paths, event hooks, and fatal error propagation.
@Hemdan47
Hemdan47 merged commit 93af553 into main Jun 24, 2026
2 checks passed
@Hemdan47
Hemdan47 deleted the dev branch June 26, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant