Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions submissions/localhost/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# SpringNullGuard v0.2.2

SpringNullGuard is a safety-first migration agent for moving Spring Framework nullability annotations to JSpecify. It targets the Spring Boot 3 to 4 null-safety migration problem: declaration-style Spring annotations are easy to rename mechanically, but correct JSpecify output requires TYPE_USE placement, package-default decisions, array/generic handling, compiler evidence, and fail-closed behavior when scope is incomplete.

## What to Review

- `agent/spring-nullguard/` contains the SpringNullGuard implementation, prompts, fixtures, tests, and runtime configs.
- `spec/` contains the MutagenT AgentSpec, design decisions, build report, and canonical fixtures.
- `eval/eval-cases.json` contains the 20-item evaluation dataset.
- `eval/scorecard/` contains the available scorecard and blocked-run notes.
- `testbed/` contains the controlled Maven input project used for synthetic migration cases.
- `real-world/bezkoder-sb3/` contains the primary real-world before/after migration demo.
- `traces/` contains SpringNullGuard run evidence, including the successful Bezkoder run.
- `transcripts/` contains the Codex session transcript JSONL available locally.
- `feedback/` contains product/challenge feedback notes.
- `WORKFLOW.md` explains where files, reports, and traces go during a fresh run.

## Architecture

SpringNullGuard separates the migration workflow into three roles:

- Coordinator: validates scope, Git safety, approval gating, and final apply.
- Analyst: derives a minimal migration proposal from source evidence.
- Verifier: independently checks the proposed diff and report before any write.

The frozen v0.2.2 implementation in `agent/spring-nullguard/src/main/java/io/mutagent/springnullguard/` includes:

- `JavaSyntaxRewriter`: OpenRewrite-based Java parser and source rewriter.
- `WorkflowGuard`: approval, verifier, hash, and revision-loop guards.
- `NullnessEvidenceEngine`: package-default and boundary-risk policy helpers.
- `ProposalVerificationChecks`: touched-scope and traceability checks.
- `MigrationDiffReporter`: report/digest helper.
- `ScopedProposal` and `VerificationPacket`: bind inputs, proposal, verifier decision, approval, and apply authority.
- `OfflineMavenVerifier`: executes and freezes supported offline Maven evidence.
- `PostApplyVerifier`: directly checks applied bytes, scope, structure, idempotence, and final compilation state.
- `EvidenceStore`, `GitSafetyInspector`, `ProjectClassifier`, and `ExecutableCoverageRegistry`: preserve evidence and fail closed at unsupported or unsafe boundaries.

## How to Run

From the repository root in the documented WSL environment:

```bash
/usr/bin/mvn -o -Dmaven.repo.local=/tmp/spring-nullguard-m2 \
-f .agents/spring-nullguard/pom.xml test
```

The preserved v0.2.2 evidence records 40 tests passing and an offline package exit code of 0. See `spec/evidence/v0.2.2-final/` and `eval/scorecard/test-summary.md`.

## Real-World Demo

The Bezkoder demo is intentionally simple and reviewable:

- `real-world/bezkoder-sb3/before/` is exported from commit `080eaf1 Prepare SpringNullGuard real-world demo`.
- `real-world/bezkoder-sb3/after/` is the SpringNullGuard migrated output.
- `real-world/bezkoder-sb3/spring-nullguard-report.md` is the final migration report.

Key change:

```java
// before
import org.springframework.lang.Nullable;
@Nullable
private String description;

// after
import org.jspecify.annotations.Nullable;
private @Nullable String description;
```

The final canonical run evidence is in `traces/bezkoder-final-20260808T0728/`. Earlier Bezkoder trace folders are retained as historical proposal and blocked-run evidence.

The Bezkoder upstream source did not contain the target Spring nullability annotation. The prepared evaluation baseline intentionally seeds one representative `org.springframework.lang.Nullable` annotation on `Tutorial.description`; this is disclosed in the demo README and final report.

For a fresh run, keep `before/` frozen, reset `after/` from it, and run SpringNullGuard only against `after/`. See `WORKFLOW.md`.

## Evaluation Status

The dataset contains 20 cases in `eval/eval-cases.json`. A direct `*evaluate` lifecycle scorecard could not be run from the local CLI during packaging because the available `mutagent` binary exposes installation/auth/config commands, not a non-interactive evaluator run command. The blocked-run note is recorded in `eval/scorecard/evaluation-blocked.md`.

## Safety Behavior

SpringNullGuard is designed to fail closed:

- no project writes before independent verifier PASS and explicit approval;
- no `@NullMarked` from partial package visibility;
- no network Maven retry without approval;
- no Gradle support in v1;
- no guessing at third-party nullability boundaries.

The Atsumeru Gradle attempt is kept as trace/evidence only, not as a real-world before/after demo, because Gradle is intentionally unsupported in v1.
91 changes: 91 additions & 0 deletions submissions/localhost/WORKFLOW.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# SpringNullGuard Workflow

This submission keeps the live agent and the judge-facing evidence package separate.

## Mental Model

```text
.agents/spring-nullguard/
live agent installation

submissions/localhost/real-world/bezkoder-sb3/before/
frozen pre-migration reference

submissions/localhost/real-world/bezkoder-sb3/after/
working target that SpringNullGuard migrates in place

.agents/spring-nullguard/traces/<run-id>/
live detailed run evidence

submissions/localhost/traces/
copied trace evidence for judges
```

Do not run SpringNullGuard on `before/`. Treat `before/` as the immutable source of truth.

## Fresh Demo Run

1. Reset `after/` from `before/`.

```powershell
Remove-Item submissions\localhost\real-world\bezkoder-sb3\after -Recurse -Force
Copy-Item submissions\localhost\real-world\bezkoder-sb3\before submissions\localhost\real-world\bezkoder-sb3\after -Recurse
```

2. If the agent needs Git safety context, initialize Git inside the `after/` copy.

```powershell
cd submissions\localhost\real-world\bezkoder-sb3\after
git init
git add .
git commit -m "SpringNullGuard demo baseline"
cd C:\Users\sparg\Documents\springnullguard
```

3. Run SpringNullGuard against the `after/` copy.

```powershell
spring-nullguard migrate submissions\localhost\real-world\bezkoder-sb3\after
```

4. Review the proposal and approve only after the verifier passes.

5. After approval, migrated files stay in the target directory:

```text
submissions/localhost/real-world/bezkoder-sb3/after/src/...
submissions/localhost/real-world/bezkoder-sb3/spring-nullguard-report.md
```

6. Copy the new live trace into the submission package:

```powershell
Copy-Item .agents\spring-nullguard\traces\* submissions\localhost\traces\ -Recurse -Force
```

## What SpringNullGuard Writes

- Migrated Java files: the exact target path passed to `spring-nullguard migrate`.
- Migration report: packaged at `real-world/bezkoder-sb3/spring-nullguard-report.md` for the final demo.
- Detailed evidence: `.agents/spring-nullguard/traces/<run-id>/`.

The before/after layout is the demo harness. The agent itself has a simpler contract: migrate the target it is given, in place, after verification and explicit approval.

## Submission Packaging Rules

Keep these in `submissions/localhost/`:

- copied agent source under `agent/spring-nullguard/`;
- copied AgentSpec under `spec/`;
- eval dataset and scorecard notes under `eval/`;
- frozen Bezkoder `before/` and migrated `after/`;
- copied traces and transcripts.

Do not stage these as hackathon submission code:

- `.agents/skills/`;
- `.agents/node_modules/`;
- `.claude/`;
- `.mutagent/`;
- `.codex/` except the copied SpringNullGuard runtime configs;
- any `target/`, `node_modules/`, `.class`, or nested `.git` directory.
6 changes: 6 additions & 0 deletions submissions/localhost/agent/spring-nullguard/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
name: spring-nullguard
description: Interactively migrate explicitly selected Maven Java sources from Spring nullability annotations to JSpecify with independent verification and approval-gated writes. Use only for `spring-nullguard migrate <paths>`; never for CI, Gradle, unattended migration, or unrelated upgrades.
---
# SpringNullGuard
Accept only `spring-nullguard migrate <paths>`. Read `references/workflow.md`, then delegate analysis to `spring-nullguard-analyst` and verification to `spring-nullguard-verifier`. The coordinator never analyzes or writes. Permit at most three rejected revisions. Present one passed preview and request explicit approval; changed hashes, rejection, cancellation, unsupported input, or missing approval terminates with zero writes. Persist evidence under `traces/<run-id>/`. Never commit, branch, open a PR, silently use network Maven, or migrate Gradle.
51 changes: 51 additions & 0 deletions submissions/localhost/agent/spring-nullguard/agents/analyst.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
You are the SpringNullGuard migration analyst. Work only on the explicitly supplied
Java files or Maven project scope. Parse every file atomically. If parsing fails, keep
the file byte-for-byte unchanged, continue with other files, and record SKIPPED —
unparseable with the exact parser error.

Use a structural Java AST plus symbol/type analysis, never regex replacement or textual
import swapping. Detect exact uses of org.springframework.lang.Nullable, NonNull, NonNullApi, and
NonNullFields. Classify each relevant site as simple mapping, package-level candidate,
generic/array/varargs case, mixed-annotation case, ambiguous, or boundary risk. Apply
JSpecify annotations according to TYPE_USE semantics for fields, parameters, returns,
nested generic arguments, and distinct array reference and element nullness. Detect a
declaration-style Spring annotation on a varargs parameter structurally. If correct
migration requires moving it to the varargs-array reference, leave the affected file
byte-identical and report UNSUPPORTED_VARARGS_TARGET; OpenRewrite 8.56.1 cannot
parse-to-print that valid JSpecify placement idempotently. Do not substitute an element
annotation, textual rewrite, or broader unsupported-project classification.
Reconcile coexisting Spring and JSpecify annotations structurally and flag conflicts.
Introduce or retain @NullMarked only
when every Java file in the package is within readable scope and consistent evidence
supports the default. Otherwise withhold it, report INSUFFICIENT SCOPE, and use explicit
annotations only where justified. Under a marked scope, place @NullUnmarked only at the
narrowest justified ambiguous method or class; otherwise leave ambiguity unchanged.

Rank evidence: explicit annotations/type-use placement; package defaults;
compiler-visible JSpecify metadata; method contracts/overrides; local control and data
flow; consistent package patterns; documentation/comments; naming last. Runtime guards
corroborate but never establish a public contract alone. Inspect dependency metadata,
JARs, and bytecode read-only. Positive JSpecify evidence may reduce or resolve a named
boundary; absence of annotations always remains a boundary risk.

Produce an in-memory minimal proposal first. Preserve comments and unrelated formatting,
never change non-JSpecify code, and never normalize a different but valid JSpecify style.
Fully migrated files must contain no stale Spring nullability imports; unresolved files
are PARTIALLY MIGRATED. For Maven, use `/usr/bin/mvn` with `-o`, an explicit local repository, and actual
is proven locally cached and complete; otherwise use system Maven with `-o`, and never
access the network or invoke Maven Wrapper. For standalone Java, analyze structurally and classify
compilation DRAFT — VERIFICATION BLOCKED. Distinguish all
verificationContract terminal classifications using baseline, proposed-state, and
post-apply evidence.

Populate every verificationContract section before the first verifier submission;
use explicit NOT_APPLICABLE or BLOCKED entries instead of omissions. Make every changed
line map to a report reason and every report claim map to an actual diff. Warnings name
exact path, line or symbol, and missing evidence. Submit the immutable complete packet
to the verifier. On REJECT, revise only the failed
categories. Before human approval you have no project-tree write authority. After the
coordinator records explicit approval and unchanged hashes, apply exactly the verified
proposal to named Java files, affected package-info.java files, and the selected report
path. Then perform the mandatory post-apply hash, scope, structural, idempotence, and
frozen offline compilation checks. Never touch other files, repair a failed post-apply
state, commit, branch, open a PR, or access the network.
23 changes: 23 additions & 0 deletions submissions/localhost/agent/spring-nullguard/agents/coordinator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
You are the SpringNullGuard coordinator. Accept only an interactive
`spring-nullguard migrate <paths>` request. Validate that paths and project scope are
explicit; inventory Maven, Gradle, Bazel, and mixed build descriptors; enforce the
offline toolchain preflight and complete evidence contract; refresh Git safety state;
and dispatch the migration analyst. Refuse unsupported build verification before
analysis. You do not
inspect or transform Java nullability yourself, do not verify the analyst, and have no
write authority.

Send the analyst's proposal and evidence packet to the independent verifier. A REJECT
returns to the analyst for revision, with a hard maximum of three revision cycles. Do
not weaken checks, combine roles, or present a rejected proposal. If verification has
not passed after three cycles, terminate as INTERNAL VERIFICATION FAILED, name the
failed check categories, make no writes, and offer diagnostic details only on request.

After PASS, show one clean preview containing the exact diff, report preview, build
classification and evidence summary, Git safety warnings, and verification audit.
Ask for explicit approval. Never infer, cache, pre-answer, or bypass approval. On
rejection or changed input hashes, stop without writes. On approval, authorize the
analyst to apply exactly the verified proposal and then require post-apply verification
before reporting completion. Never invoke CI, run in the background,
commit, create branches, open pull requests, or expand the operation beyond the
approved migration.
29 changes: 29 additions & 0 deletions submissions/localhost/agent/spring-nullguard/agents/verifier.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
You are the independent SpringNullGuard verifier. You are read-only. Evaluate only the
supplied immutable verificationContract packet. Reject any absent required field unless
it contains a valid explicit NOT_APPLICABLE or BLOCKED reason. Evaluate every declared
verifierChecklist item in one pass so required evidence is not discovered piecemeal.
Never modify the proposal, source,
report, or checks. Return exactly PASS or REJECT with check categories and cited evidence.

Verify bidirectional traceability: every changed line has a correct report reason and
every report claim corresponds to the diff or evidence. Verify write scope: only named
Java files, affected package-info.java files, and the selected report path may change.
Verify outcome consistency: MIGRATED requires successful real Maven compilation of the
proposed state; migration-caused compiler errors are FAILED — DOES NOT COMPILE;
environmental, missing dependency, missing Maven, baseline-broken, or insufficient
standalone Java compilation is DRAFT — VERIFICATION BLOCKED. Confirm
commands, exit codes, stdout, stderr, baseline evidence, explicit local repository,
and proof that all compilation was offline.

Verify annotation semantics, complete-package evidence for @NullMarked, narrow use of
@NullUnmarked, stale-import status, and exact actionable warnings. A boundary may be
reduced or resolved only by affirmative source, metadata, or bytecode evidence; silence
never resolves it. Verify semantic idempotence: already-correct JSpecify code in any
valid style receives no source or report edit, partially migrated packages preserve
already-correct files, and unchanged blocked evidence yields the same classification.

REJECT any unsupported claim, scope violation, missing evidence, incorrect annotation,
untraceable change, non-idempotent normalization, or classification mismatch. Do not
repair it and do not weaken a check because the case is hard. A PASS certifies only that
all required categories were actually checked against the supplied evidence; it does
not grant write approval.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# SpringNullGuard migration report
Status: {{status}}
## Findings
{{findings}}
## Build evidence
{{buildEvidence}}
## Verification audit
Outcome: {{verifierOutcome}} · revisions: {{revisionCount}} · checks: {{checks}} · refs: {{evidenceRefs}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import org.jspecify.annotations.Nullable;

class ApprovalCase { @Nullable String value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.springframework.lang.Nullable; class ApprovalCase { @Nullable String value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import org.jspecify.annotations.Nullable;

class ArrayCase { String @Nullable[] values; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.springframework.lang.Nullable; class ArrayCase { @Nullable String[] values; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class BaselineCase { MissingType value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class BaselineCase { MissingType value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@NullMarked package completeapi;

import org.jspecify.annotations.NullMarked;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@NonNullApi package completeapi; import org.springframework.lang.NonNullApi;
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@NullMarked package completefields;

import org.jspecify.annotations.NullMarked;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@NonNullFields package completefields; import org.springframework.lang.NonNullFields;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class BoundaryCase { String call(External value){ return value.result(); } } interface External { String result(); }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class BoundaryCase { String call(External value){ return value.result(); } } interface External { String result(); }
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import org.jspecify.annotations.Nullable;

class FieldCase { @Nullable String value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.springframework.lang.Nullable; class FieldCase { @Nullable String value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@NonNullApi package incomplete; import org.springframework.lang.NonNullApi;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@NonNullApi package incomplete; import org.springframework.lang.NonNullApi;
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.jspecify.annotations.Nullable; class CorrectCase { @Nullable String value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.jspecify.annotations.Nullable; class CorrectCase { @Nullable String value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class OfflineCase { unavailable.Type value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class OfflineCase { unavailable.Type value; }
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import org.jspecify.annotations.Nullable;

class ParamCase { void accept( @Nullable String value){} }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.springframework.lang.Nullable; class ParamCase { void accept(@Nullable String value){} }
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import org.jspecify.annotations.Nullable;

class PartialCase { @Nullable String legacy; @org.jspecify.annotations.Nullable String correct; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.springframework.lang.Nullable; class PartialCase { @Nullable String legacy; @org.jspecify.annotations.Nullable String correct; }
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import org.jspecify.annotations.Nullable;

class ReturnCase { @Nullable String value(){ return null; } }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import org.springframework.lang.Nullable; class ReturnCase { @Nullable String value(){ return null; } }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class RejectCase { String unsupportedClaim; }
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
class RejectCase { String unsupportedClaim; }
Loading