Skip to content

feat: gate provider usage candidates - #35

Merged
ictechgy merged 4 commits into
mainfrom
eval/provider-cost-evidence
Aug 13, 2026
Merged

feat: gate provider usage candidates#35
ictechgy merged 4 commits into
mainfrom
eval/provider-cost-evidence

Conversation

@ictechgy

Copy link
Copy Markdown
Owner

Summary

  • add an offline aggregate-only provider usage scorer for metered-cost and fixed-subscription quota evidence
  • reject no-op economic candidates whose exact command matches the baseline
  • keep Codex, agy, and Grok standard routes at medium in the cost-focused scaffolds
  • record the rejected Terra-to-Luna Codex qualification without retaining task or response data

Safety boundaries

  • no provider credentials, raw exports, task text, responses, or pair rows are persisted
  • subscription quota evidence cannot claim monthly bill reduction or feed the cost recommender
  • the Luna candidate remains unqualified after two new critical failures; built-in routes are unchanged

Verification

  • Python 3.10: 816 tests passed with ResourceWarning as error
  • Python 3.14: 816 tests passed with ResourceWarning as error
  • Ruff 0.16.2 check and format-check passed
  • strict mypy passed for 107 source files
  • compileall and git diff --check passed

Document the exact-command Codex token signal and the agy/Grok usage-metadata gap without retaining task or response content.
@ictechgy
ictechgy merged commit ce9a349 into main Aug 13, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant