Skip to content

Extract shared knowledge base search and evaluation logic to submodule#85

Merged
JoeStech merged 6 commits into
mainfrom
extract-kb-search-logic
May 18, 2026
Merged

Extract shared knowledge base search and evaluation logic to submodule#85
JoeStech merged 6 commits into
mainfrom
extract-kb-search-logic

Conversation

@brikin01
Copy link
Copy Markdown
Collaborator

No description provided.

Copilot AI review requested due to automatic review settings May 15, 2026 13:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extracts the knowledge-base search utilities (previously in mcp-local/utils/search_utils.py) and retrieval evaluation helpers into a new standalone arm_kb_search Python package, and updates mcp-local/server.py, the Dockerfile, and the evaluation script to consume it.

Changes:

  • New arm_kb_search package with search, loaders, evaluation, config modules and a pyproject.toml.
  • mcp-local/server.py and Dockerfile updated to import from arm_kb_search; DISTANCE_THRESHOLD/K_RESULTS removed from mcp-local/utils/config.py.
  • embedding-generation/evaluate_retrieval.py replaced by evaluate_local_retrieval.py, which delegates to arm_kb_search.evaluation; shell script and README updated accordingly.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pyproject.toml Declares the new arm-kb-search package and its dependencies.
arm_kb_search/init.py Public API re-exports for the new submodule.
arm_kb_search/config.py Hosts moved DISTANCE_THRESHOLD/K_RESULTS constants.
arm_kb_search/loaders.py Index/metadata loader helpers moved out of search.py.
arm_kb_search/search.py Removes loader helpers and json/os imports.
arm_kb_search/evaluation.py New shared retrieval evaluation helpers with dataclasses.
mcp-local/server.py Switches imports to arm_kb_search.
mcp-local/utils/config.py Removes constants that moved to arm_kb_search.config.
mcp-local/Dockerfile Copies the arm_kb_search package into the image.
embedding-generation/evaluate_local_retrieval.py New evaluation entrypoint built on arm_kb_search.
embedding-generation/evaluate_retrieval.py Removed (replaced by new script).
embedding-generation/run-question-eval.sh Invokes the renamed evaluation script.
embedding-generation/README.md Updates the script name mentioned in docs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread embedding-generation/evaluate_retrieval.py
@brikin01 brikin01 requested a review from JoeStech May 15, 2026 14:23
Comment thread embedding-generation/requirements.txt
Comment thread embedding-generation/evaluate_retrieval.py
@brikin01
Copy link
Copy Markdown
Collaborator Author

brikin01 commented May 18, 2026

@JoeStech Addressed the comments and also updated the evaluation function to output both the summary and actually include the individual cases so that we can do more detailed breakdowns later.

@JoeStech JoeStech merged commit 73b0715 into main May 18, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants