fix(importers): source Hermes mining from state.db (skill mining was silently empty) - #142
Merged
Conversation
… path iter_hermes_sessions walked ~/.hermes/sessions/*.json, which modern hermes no longer writes (those files are request-error dumps with no 'messages' key), so both the skill importer and the tool-mining path silently got zero sessions. Read the canonical state.db first (the store our closed-loop validator already reads), falling back to the JSON path when the DB yields nothing. Mines all sessions regardless of sessions.source, strips Hermes's prepended model-switch note while keeping the real instruction, and decodes tool_calls to a list so the (session_id, messages) contract is unchanged for both consumers. Adds DB-sourcing tests + an autouse isolation fixture so a real ~/.hermes/state.db can't hijack the JSON-fixture tests.
jramos
force-pushed
the
fix/importer-state-db
branch
from
June 28, 2026 18:35
9bc538b to
44381c3
Compare
jramos
added a commit
that referenced
this pull request
Jul 6, 2026
…ne new item (#150) Incremental upstream review. Backlog steady at 72 open; six new PRs #142-#147. Five already covered -> SKIP; one genuinely-new latent bug -> new action item. - NousResearch#142: 5 of 6 fixes are covered clusters (GEPA/DSPy-3.2 compat x3, validate-full). The 6th is real and new here: find_skill via Path.rglob('SKILL.md') skips symlinked skill dirs on Python <3.13 (skill_sources.py:63/67/80). Promoted to a CHERRY-PICK action item (recommended). - #143: our #102 — state.db read-only importer already shipped; only cross-platform discovery extras. SKIP. - #144/#145: Sunwo0u HSE sanitized evidence packets, report-only, no mechanism. SKIP. - #146: extraction cluster — our skill_text is a @Property over signature.instructions, so its 'dead input field' premise is upstream-only. SKIP. - #147: compat + material_diff reporting subsumed by our behavioral deploy gate. SKIP. Review-log entry + new action-item row + 2026-07-06 snapshot delta.
jramos
added a commit
that referenced
this pull request
Jul 6, 2026
HermesSkillSource resolved SKILL.md via Path.rglob('SKILL.md') at three
sites. rglob refuses to descend into symlinked directories on Python <3.13
(the recurse_symlinks kwarg is 3.13-only), so a Hermes layout that symlinks
user-installed skills into the framework tree silently resolved 'not found'.
Replace all three sites with one private _iter_skill_files(root) helper:
os.walk(followlinks=True) + a (st_dev, st_ino) visited-set cycle guard
(prunes a symlink pointing back to an ancestor -> provably terminating) +
per-level sort() for deterministic first-match-wins. onerror/stat failures
are debug-logged and skipped (parity with the old silent skip). find_skill
walks once and runs both passes over the materialized list.
Scope held to HermesSkillSource: the flat ClaudeCode/LocalDir sources
already follow symlinks via is_file()/iterdir() (noted inline).
9 new symlink tests (correctness C1-C5, cycle/permission/dangling safety
S1-S3, determinism D1) with an os.symlink-unavailable skip-guard; the
symlinked-directory cases were red on the old rglob code. Full non-slow
suite green (1763 passed, +9), ruff clean.
Resolves the symlink item of NousResearch#142;
the other five #142 fixes are already-covered clusters. Marks the triage
action item DONE (row + review-log + snapshot delta).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Triage #102.
iter_hermes_sessions(shared by the skill importer and the tool-mining path) read~/.hermes/sessions/*.json, which modernhermesno longer writes — those files are request-error dumps with nomessageskey, so both paths silently mined zero sessions. This sources from the canonical~/.hermes/state.db(the store our closed-loop validator already reads viaparse_session_from_db).iter_hermes_sessions; falls back to the JSON path when the DB yields nothing (rows-based).sessions.source(every real row issource='cli'; filtering would re-create the empty bug).[Note: model was just switched …]note while keeping the real instruction (it's a prefix on the user turn, not a standalone message — dropping it would discard genuine examples).tool_callsTEXT→list so the(session_id, messages)contract is unchanged —session_mining.pyneeds no edit (guarded by a regression test).sqlite3.Errorabstains → JSON fallback.End-to-end: the skill importer now mines real user/assistant pairs from
state.dbwhere it returned 0 before.Test Plan
pytest tests/core/test_external_importers.py tests/tools/test_session_mining.py— DB pairing, source-agnostic mining, note prefix-strip (kept, not dropped), malformedtool_calls→None, DB-empty→JSON fallback, tool-path regression.~/.hermes/state.dbfrom hijacking the JSON-fixture / dry-run tests.Deferred follow-up (separate concern): the #26
RelevanceFilterlow-recall improvement.