#389 Move helper service discovery to CF catalog metadata

[somebloke1]

Summary

Draft PR for #389 Decision 2: make ContextForge registry/catalog service-offering metadata the ordinary product truth for helper service discovery.

This slice:

• Documents the service-offering Resource metadata contract.
• Changes helper runtime discovery to read ContextForge /resources, fetch full tagged /resources/{id} content, validate metadata, and match active offerings by explicit runtime.server_id plus virtual-server Resource association.
• Removes product fallback from CF tag/name heuristics and state-only/local-manifest resurrection.
• Passes the full CF-derived descriptor through enable/apply paths instead of a bare binding.
• Adds scripts/register_service_offerings.py to seed/refresh service-offering Resources from server-instances/*/instance.json as migration input only.
• Live-seeded the local ContextForge catalog for the 9 current offerings: context7, exa-search, github, mentality, openzeppelin-solidity-contracts, playwright, serena, ssh-tmux, web-search.
• Adds deterministic tests for invalid metadata, wrong schema, missing required fields, duplicate offering/binding conflicts, runtime mismatch, project-hash binding expansion, full resource fetch, catalog-unavailable behavior, and association preservation.
• Adds the known-service state-story runner scaffold for qwen-backed Pi/OpenCode acceptance.

Follow-up issue #391 tracks future mentality CWD/request-context isolation; this PR preserves the current shared/static mentality binding.

Validation

• python -m py_compile scripts/register_service_offerings.py scripts/project_init_common.py scripts/contextforge_helper_mcp.py
• .venv/bin/python -m py_compile scripts/contextforge_helper_mcp.py scripts/control_plane_project_init_helper.py scripts/project_init_common.py scripts/register_service_offerings.py tests/test_project_init_activation_workflow.py
• .venv/bin/python -m pytest tests/test_project_init_activation_workflow.py -q → 193 passed, 33 subtests passed
• .venv/bin/python -m pytest tests/test_project_init_activation_workflow.py -k 'service_offering or contextforge_registry_service_offerings' -q → 8 passed
• .venv/bin/python -m pytest tests -q → 1065 passed, 724 subtests passed
• git diff --check
• Live registrar: PYTHONPATH=scripts python scripts/register_service_offerings.py → 9 updated, 9 associated, 0 errors
• Live helper readback: service_management_list(..., client_type='opencode') returned 9 CF metadata-derived services.

Note: unscoped repo-root pytest is not a meaningful gate in this checkout because it discovers copied tests under run/ and upstream ContextForge trees with optional dependency/name-collision failures. Repo-local deterministic gate is pytest tests.

Status

Ready for review on the deterministic + OpenCode acceptance slice. OpenCode/qwen3.6-a3b state-story acceptance is recorded in PR/issue comments at docker/client-harness/evidence/known-service-management-state-story/opencode/20260625T143313Z-serena-language-infer. Earlier Pi/qwen runs predate the final CF metadata catalog rows and remain obsolete/non-acceptance evidence; do not claim Pi acceptance until a fresh current-branch Pi run is completed, or record a fresh blocker.

Closes/continues #389.

[somebloke1]

Final local validation/evidence update for #389 Decision 2:

• Commit pushed: 6d43183 (Add semantic session observability for state-story runs).
• Accepted qwen/OpenCode evidence: docker/client-harness/evidence/known-service-management-state-story/opencode/20260625T143313Z-serena-language-infer.
• Accepted run session: ses_100ccdffeffe3atwTZDEQANgNK.
• Accepted known defect: assistant overperformed by proceeding without requiring an extra re-approval; reviewer/user accepted this as non-blocking.
• Added scripts/sess_obs.py plus runner sess-obs-stream.jsonl/~/.sess-obs.json support so future semantic/model-involved runs can be observed live with Prompt/Assistant/Thinking demarcations.

Final deterministic validation on the pushed branch:

.venv/bin/python -m py_compile scripts/contextforge_helper_mcp.py docker/client-harness/scripts/run-known-service-management-state-story.py scripts/sess_obs.py
.venv/bin/python -m pytest tests/test_project_init_activation_workflow.py tests/test_contextforge_docker_harness.py -q
# 297 passed, 1 warning, 33 subtests passed
.venv/bin/python -m pytest tests -q
# 1065 passed, 1 warning, 724 subtests passed
.git diff --check
# passed

[somebloke1]

Follow-up prompt/list strengthening pushed in 9c83300:

• Visible ContextForge service menus now number rows, e.g. 1. context7 - Available - ..., so service-name and selection-number affordances are aligned.
• Service-management list now ends with Reply with service names or numbers to enable them, or choose none. instead of an unrelated reload line.
• Regression added for numbered project-init and service-management visible list output.

Validation:

.venv/bin/python -m pytest tests/test_project_init_activation_workflow.py -q
# 194 passed, 1 warning, 33 subtests passed
.venv/bin/python -m py_compile scripts/contextforge_helper_mcp.py
# passed
git diff --check
# passed

[somebloke1]

Update for OpenCode wrapper/auth split-brain follow-up (commit 429c47c):

• Root cause fixed: project-local OpenCode wrapper generation could carry CONTEXTFORGE_BASE_URL=http://127.0.0.1:4445 without the matching client-scoped CONTEXTFORGE_CONFIG_ENV, causing wrapper fallback to repo-local/default credentials. OpenCode entries now preserve wrapper-specific overrides first, then fall back to CONTEXTFORGE_CLIENT_SCOPED_ENV / CONTEXTFORGE_ENV / CONTEXTFORGE_CONFIG_ENV; token cache/lock fallback is also propagated.
• cf_project_init_continue now reuses the same ContextForge service-offering catalog for menu parsing and proposal resolution, so manifest/local staged config cannot resurrect services such as time unless valid CF service-offering metadata exists.
• Wrapper bootstrap 401 diagnostics now include non-secret base URL, resolved env path/existence, token-cache path, and HTTP status/reason.

Validation:

• PYTHONDONTWRITEBYTECODE=1 .venv/bin/python -m py_compile scripts/control_plane_contextforge_binding.py scripts/contextforge_mcp_wrapper.py scripts/contextforge_helper_mcp.py scripts/control_plane_project_init_helper.py scripts/project_init_common.py
• .venv/bin/python -m pytest tests/test_project_init_activation_workflow.py -k 'opencode and wrapper' -q -> 4 passed
• .venv/bin/python -m pytest tests/test_project_init_activation_workflow.py -k 'service_management or contextforge_registry_service_offerings or project_init' -q -> 199 passed, 33 subtests passed
• .venv/bin/python -m pytest tests/test_contextforge_mcp_wrapper.py -q -> 11 passed
• .venv/bin/python -m pytest tests -q -> 1072 passed, 724 subtests passed
• git diff --check

Live/local probe:

• Fresh /home/dgk/workspace/test-new-proj-07 OpenCode project-init for context7 against http://127.0.0.1:4445 wrote opencode.json with matching CONTEXTFORGE_BASE_URL, CONTEXTFORGE_CONFIG_ENV=/home/dgk/.config/opencode/state/contextforge-project-init/contextforge-4445.env, and token cache/lock paths.
• The live CF catalog menu listed 9 metadata-backed services and did not list time.
• Direct wrapper bootstrap for context7_local_server with the generated env exited cleanly (wrapper_rc=0, contextforge_wrapper_stop, no 401/Unauthorized bootstrap error).

No new Pi/OpenCode semantic acceptance claim is made here; this is deterministic + live wrapper/auth verification only.