Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,16 @@ concurrency:
jobs:
test:
name: test-${{ matrix.os }}-py${{ matrix.python }}
continue-on-error: ${{ matrix.advisory }}
strategy:
fail-fast: false
matrix:
include:
- { os: ubuntu-latest, python: "3.12", coverage: false }
- { os: ubuntu-latest, python: "3.13", coverage: false }
- { os: ubuntu-latest, python: "3.14", coverage: true }
- { os: macos-latest, python: "3.12", coverage: false }
- { os: macos-latest, python: "3.14", coverage: false }
- { os: ubuntu-latest, python: "3.12", coverage: false, advisory: false }
- { os: ubuntu-latest, python: "3.13", coverage: false, advisory: false }
- { os: ubuntu-latest, python: "3.14", coverage: true, advisory: false }
- { os: macos-latest, python: "3.12", coverage: false, advisory: true }
- { os: macos-latest, python: "3.14", coverage: false, advisory: true }
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/ocr-compatibility.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,16 @@ jobs:
if: ${{ !cancelled() }}
env:
GITHUB_TOKEN: ${{ github.token }}
QUALIFICATION_OUTCOME: ${{ steps.qualify.outcome }}
REPOSITORY: ${{ github.repository }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
input=(--status /tmp/ocr-compat/status.json)
if [ -f /tmp/ocr-compat/evidence.json ]; then
input=(--evidence /tmp/ocr-compat/evidence.json)
fi
case "${QUALIFICATION_OUTCOME}" in
success) input=(--evidence /tmp/ocr-compat/evidence.json) ;;
failure) input=(--status /tmp/ocr-compat/status.json) ;;
*) echo "unsupported qualification outcome" >&2; exit 1 ;;
esac
test -f "${input[1]}"
PYTHONPATH=src python scripts/ocr_compat.py upsert-issue \
"${input[@]}" \
--repository "${REPOSITORY}" \
Expand Down
366 changes: 365 additions & 1 deletion PLANS.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions changelog.d/139.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Made OCR compatibility failure publication authoritative and recoverable. The workflow now selects closed status or evidence from the actual qualification outcome, atomically replaces output handoffs, and can publish a failure status without loading unrelated support metadata. It still publishes the bounded issue and diagnostic artifact before returning a red job, while aggregate promotion remains blocked and raw diagnostics remain private.
1 change: 1 addition & 0 deletions changelog.d/140.bugfix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixed the GitLab example's stale positive `OCR_MAX_TOOLS` default by delegating with sentinel `0`. Added exact parsing for OCR's supported max-tools normalization notice and behavior-based compatibility evidence for numeric CLI boundaries, including the effective template-owned tool-loop value. The notice remains operator-only: raw OCR stderr, findings, result warnings, receipts, DLP, telemetry, lifecycle commands, and automatic-approval semantics are unchanged.
4 changes: 4 additions & 0 deletions changelog.d/142.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
GitLab technical details now make OCR review activity easier to explain without implying unsupported token attribution.

- **Added:** the existing inline `all OCR tool calls` line shows every non-zero counter from the closed OCR inspection, review-output, and toolkit context/evidence set instead of collapsing entries after the six most frequent tools.
- **Unchanged:** aggregate input/output/cache token usage remains separate because OCR does not report per-tool token consumption; raw call arguments, results, dynamic external MCP tool names, receipt v5, DLP, findings, severity, and automatic approval are unchanged.
1 change: 1 addition & 0 deletions changelog.d/143.maintenance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Keep both macOS endpoint CI jobs as visible best-effort compatibility diagnostics while retaining Linux, coverage, quality, security, dependency, package, and CodeQL checks as release-blocking gates.
73 changes: 73 additions & 0 deletions compatibility/evidence/ocr-1.10.0.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,78 @@
"result": "passed",
"wire_field": "max_completion_tokens"
},
"numeric_cli_probe": {
"max_tokens_budget": {
"cases": {
"invalid_below": {
"effective": null,
"input": -1,
"outcome": "rejected"
},
"minimum": {
"effective": 1,
"input": 1,
"outcome": "accepted"
},
"omitted": {
"effective": "unlimited",
"input": null,
"outcome": "accepted"
},
"representative": {
"effective": 30000,
"input": 30000,
"outcome": "accepted"
},
"sentinel": {
"effective": "unlimited",
"input": 0,
"outcome": "accepted"
}
},
"maximum": null,
"owner": "ocr-cli"
},
"max_tools": {
"cases": {
"invalid_below": {
"effective": null,
"input": -1,
"outcome": "rejected"
},
"minimum": {
"effective": 100,
"input": 50,
"outcome": "accepted"
},
"minimum_minus_one": {
"effective": 100,
"input": 49,
"outcome": "normalized",
"reported_normalization": 50
},
"omitted": {
"effective": 100,
"input": null,
"outcome": "accepted"
},
"representative": {
"effective": 101,
"input": 101,
"outcome": "accepted"
},
"sentinel": {
"effective": 100,
"input": 0,
"outcome": "accepted"
}
},
"maximum": null,
"owner": "ocr-template-or-higher-cli",
"reported_minimum": 50
},
"result": "passed"
},
"optional_capabilities": [
"llm_result_identity",
"per_run_model_override",
Expand All @@ -74,6 +146,7 @@
"--format",
"--from",
"--max-tokens-budget",
"--max-tools",
"--preview",
"--rule",
"--to"
Expand Down
4 changes: 2 additions & 2 deletions compatibility/ocr-support.json
Original file line number Diff line number Diff line change
Expand Up @@ -1181,8 +1181,8 @@
"semantic_grouping"
],
"evidence": "compatibility/evidence/ocr-1.10.0.json",
"evidence_sha256": "50f9668b68dbcc4dc83f379a55e81aa58bc7188d1be88d44abb32a509361e761",
"human_conclusion": "Reviewed OCR 1.9.10...1.10.0 source and hosted Linux wire evidence: semantic grouping, path-aware comments, default medium two-round review, private additive group metadata, output-file capability, git diagnostics, and the 16384 completion cap are compatible with toolkit boundaries; toolkit retains exclusive result-path ownership and explicit 4096 cap support.",
"evidence_sha256": "55fe857b240df00d69a7e70c8ac2d0a0bb2c12640b72e6aa9f270fa87063f8eb",
"human_conclusion": "Reviewed OCR 1.9.10...1.10.0 source, hosted Linux wire evidence, and exact Darwin numeric-boundary evidence: semantic grouping, path-aware comments, default medium two-round review, private additive group metadata, output-file capability, git diagnostics, the 16384 completion cap, and template-owned max-tools behavior are compatible with toolkit boundaries; toolkit retains exclusive result-path ownership, explicit 4096 cap support, and closed normalization projection.",
"published_at": "2026-08-25T05:45:43Z",
"release_url": "https://github.com/alibaba/open-code-review/releases/tag/v1.10.0",
"status": "tested",
Expand Down
6 changes: 5 additions & 1 deletion docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The versioned support contract lives in [`compatibility/ocr-support.json`](../co

The scheduled **OCR compatibility** workflow discovers stable upstream releases newer than the manifest monitoring floor. Drafts, prereleases, non-semantic tags, unexpected asset sets, oversized metadata or downloads, redirects outside the reviewed GitHub origins, and checksum disagreement fail closed. Every binary digest must agree with both GitHub release metadata and the upstream `sha256sum.txt`.

Candidate execution uses the verified Linux amd64 binary on an Ubuntu runner. The harness checks the reported version, the CLI flags consumed by the GitLab integration, range preview behavior, an actual JSON review through a deterministic local gateway, an aggregate-budget review that must preserve completed findings and emit budget-attributed partial coverage, and the additive JSON fields consumed by posting. For OCR 1.9.0 and later it also requires JSON preview without a session-store side effect and proves that additive comment `thinking` is accepted but not published to GitLab. Upstream source review separately verifies how OCR derives that field; the toolkit probe does not claim to reproduce a provider's private reasoning channel. Evidence permits unknown new fields but requires the fields the toolkit reads. Legacy result statuses and the versioned `ocr.run-manifest/v1` outcome are normalized through one shared toolkit contract; manifest coverage sets, failure classifications, terminal state, and budget attribution must agree before a result can be published.
Candidate execution uses the verified Linux amd64 binary on an Ubuntu runner. The harness checks the reported version, the CLI flags consumed by the GitLab integration, range preview behavior, an actual JSON review through a deterministic local gateway, an aggregate-budget review that must preserve completed findings and emit budget-attributed partial coverage, and the additive JSON fields consumed by posting. Toolkit-managed numeric OCR options are also exercised at omitted/default, sentinel, invalid-below-boundary, minimum minus one, minimum, representative, and maximum edges when bounded. The evidence records closed outcomes, recognized diagnostics, normalization, ownership, and effective values observed through real loop behavior; CLI help text is not accepted as runtime evidence. For OCR 1.9.0 and later the harness also requires JSON preview without a session-store side effect and proves that additive comment `thinking` is accepted but not published to GitLab. Upstream source review separately verifies how OCR derives that field; the toolkit probe does not claim to reproduce a provider's private reasoning channel. Evidence permits unknown new fields but requires the fields the toolkit reads. Legacy result statuses and the versioned `ocr.run-manifest/v1` outcome are normalized through one shared toolkit contract; manifest coverage sets, failure classifications, terminal state, and budget attribution must agree before a result can be published.

Built-in MCP qualification follows the protocol revisions supported by the recommended OCR release's exact MCP SDK. The current recommended release uses Go MCP SDK v1.6.1 and initiates revision `2025-11-25`; the evidence server also retains `2025-06-18`, `2025-03-26`, and `2024-11-05` for qualified older clients. For an unknown client revision the server follows MCP negotiation semantics by returning its current supported revision, leaving acceptance or termination to the client. Qualification exercises initialize, the initialized notification, ping, tool discovery, and bounded summary/list/get calls through the exact SDK rather than relying only on handcrafted JSON-RPC fixtures.

Expand All @@ -23,6 +23,8 @@ Classify a pin-only promotion and its machine or human qualification evidence as

Each OCR version owns one stable HTML marker and one qualification issue. The workflow performs a single upsert through bounded direct issue listing rather than GitHub's eventually consistent search index. Historical issues closed with the `duplicate` label remain as incident evidence but do not compete for canonical identity. Any other duplicate state fails closed instead of creating another issue; after operators select and reconcile the canonical issue, reruns update it in place.

The qualification step writes evidence, public issue text, and its closed status through same-directory atomic replacements. The step outcome is authoritative: a successful step publishes validated evidence, while a failed step publishes only the closed failure status even if retained evidence exists from work completed before the failure. Status-based recovery validates no support manifest because it does not consume support metadata. The workflow publishes the issue and diagnostic artifact before restoring the failed job outcome; dependent aggregation and promotion jobs therefore remain blocked. Raw exceptions and subprocess output stay in the private job log and never enter the status or issue projection.

Evidence records optional capabilities separately from required compatibility. OCR 1.8.7 and later expose per-run provider/model overrides and additive `llm` result identity; older tested releases remain valid without these fields. OCR 1.9.4 additionally prints its session ID only in human-readable terminal summaries; JSON output and the toolkit-consumed result contract are unchanged. OCR 1.9.5 makes the existing aggregate budget flag visible in `scan` JSON, while the toolkit remains on `review` and separately qualifies that review's partial manifest, completed findings, and budget-attributed coverage. OCR 1.9.8 adds Bedrock as an upstream provider, moves human-audience JSON/SARIF progress to stderr, and changes only upstream skill guidance to consume native severity. The toolkit configures none of the Bedrock credential boundary, invokes review with agent audience, and already consumes the same tolerant structured severity/category result fields. Session identity and scan-only output remain OCR-owned signals rather than toolkit metrics. Profile or telemetry work must require the capability it consumes rather than treating the recommended version alone as proof.

### OCR 1.9.9 — inherited predecessor
Expand All @@ -43,6 +45,8 @@ Group labels are model-produced and group keys contain sorted changed paths. Add

OCR 1.10.0 also adds `--output`; `ocr-ci review` rejects `--output`, `--output=...`, `-o`, and attached short forms before preview because the toolkit retains sole ownership of the private result descriptor and atomic validation/cleanup. Improved upstream git errors remain private OCR stderr and do not widen GitLab diagnostics. With no toolkit completion-cap override, the qualified OpenAI wire request now uses `max_completion_tokens=16384`, down from 58,888 in OCR 1.9.10. `OCR_LLM_MAX_COMPLETION_TOKENS=4096` remains an explicit operator workaround for gateways that reserve spending against the requested output cap.

Its max-tools CLI reports a minimum and normalization target of `50`, while the embedded review template owns `100` rounds and the CLI only raises that value. Exact-binary behavioral probes therefore observe effective `100` for omitted, sentinel `0`, `49`, and `50`, and effective `101` for explicit `101`; `--max-tools 49` still prints the `50` normalization target. Toolkit preview accepts only that exact diagnostic shape and emits a closed operator notice. It does not copy raw stderr into the OCR result or derive finding, receipt, DLP, telemetry, lifecycle, or approval authority from the notice. The public example now passes sentinel `0` instead of a stale release-specific positive value.

Deploy toolkit 0.8.2 directly with OCR 1.10.0; do not install OCR 1.9.10 as an intermediate step. The public Linux amd64 example verifies SHA-256 `f8f99ea071bed77dbcaa15fdd2083287bb8ae408d5928b3943ebe0788d191b6b`; the manifest records the complete hosted asset set and qualification evidence.

The contiguous 1.9.7 through 1.9.8 promotion combines the repository-owned hosted Linux qualification with adjacent upstream source review. It also repeats the required 1.9.8 contract probes through the checksum-verified Darwin arm64 artifact. Multi-tool MCP initialization, fixed tool discovery and dispatch, and owner-only session persistence remain compatible with the M5 design: the toolkit may expose several fixed tools from its one built-in process, while an isolated `HOME` contains OCR's `.opencodereview/sessions` state for deterministic cleanup. Gemini and Bedrock are optional upstream provider capabilities that the toolkit neither selects nor configures; mirror-aware installation is outside the toolkit-owned download path; native severity retains the structured result fields already consumed; and human-audience progress routing does not affect the toolkit's agent-audience invocation. Therefore this promotion requires no toolkit CI behavior adaptation beyond version and checksum pins. These source observations establish capability only; the M5 production-path tests remain responsible for proving the toolkit's composition and containment.
Expand Down
12 changes: 11 additions & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ These names belong to `examples/gitlab/ocr-review.gitlab-ci.yml`; they are shell
| **`OCR_TOOLKIT_CHECKSUMS_URL`** | Example pipeline | Yes | Release URL derived from `OCR_TOOLKIT_VERSION` | Toolkit `SHA256SUMS` URL. |
| `OCR_TOOLKIT_WHEEL` | Example shell | Computed | `open_code_review_toolkit-${OCR_TOOLKIT_VERSION}-py3-none-any.whl` | Exact wheel filename selected from the release. |
| `OCR_TOOLKIT_WHEEL_SHA256` | Example shell | Computed | Matching value from `SHA256SUMS` | Digest checked before installing the toolkit wheel. |
| `OCR_MAX_TOOLS` | Example pipeline / OCR CLI | No | `30` | Positive maximum OCR tool requests per review loop; the example owns and passes this bounded value explicitly. |
| `OCR_MAX_TOOLS` | Example pipeline / OCR CLI | No | `0` | Non-negative tool-call control passed to OCR. `0` delegates to the installed OCR template; a positive value can raise that template-owned loop limit, subject to OCR's runtime validation. |
| `OCR_MAX_TOKENS_BUDGET` | Example pipeline / OCR CLI | No | `0` | Non-negative aggregate OCR token ceiling; `0` is unlimited. |

## Dynamic adapter and MCP inputs
Expand Down Expand Up @@ -164,6 +164,16 @@ The toolkit publishes that run as partial and never treats it as clean or eligib
for automatic approval. The cap is approximate because already-running work may
finish and OCR accounts the provider-reported input plus output tokens.

`OCR_MAX_TOOLS` is independent of all three token controls. Leave the example
default at `0` so the installed OCR template owns the effective per-file
tool-call limit. For the currently qualified OCR, behavioral qualification
observes a CLI minimum and normalization target of `50`, while omitted, `0`,
and values through `100` all retain the template's effective `100` rounds; a
value of `101` raises the effective limit to `101`. The OCR help text is not the
source of truth for these values. A recognized normalization is emitted only as
a toolkit-authored CI notice; its raw stderr is not added to findings, result
warnings, receipts, DLP inputs, telemetry, or automatic-approval signals.

## Posting controls

`OCR_POST_MODE`, `OCR_STRICT_POSTING`, `OCR_EXIT_CODE`, `OCR_MAX_POST_COMMENTS`, `OCR_MAX_RESULT_BYTES`, `OCR_POST_ERROR_DETAILS`, `OCR_POST_EMOJI`, `OCR_POST_BADGES`, and `OCR_AUTO_APPROVE` control write behavior and bounded error reporting. Human replies to bot-created discussions prevent automated ownership actions on that discussion.
Expand Down
Loading
Loading