feat(evaluation): validate LongMemEval-V2 smoke inputs

evaluation/README.md

@@ -181,6 +181,52 @@ uv run --project evaluation powercontext-eval swebench-pro create-batch \
   --idempotency-key "stability-$(date -u +%Y%m%dT%H%M%SZ)"
 ```
 
+## LongMemEval-V2 smoke input validation
+
+The LongMemEval-V2 command currently validates a fixed smoke subset and writes
+its preflight artifacts. It does not yet invoke a PowerContext Memory adapter,
+a Reader, or a Judge.
+
+Prepare a detached upstream checkout at the pinned harness commit and download
+the matching LongMemEval-V2 data root outside this repository. The checked-in
+small-tier lock and smoke manifest fix the data revision, three core data-file
+hashes, ten source-ordered questions, all five published abilities, and both
+published domains. The dataset lock has this shape:
+
+```json
+{
+  "schema": "powercontext.longmemeval-v2-dataset-lock.v1",
+  "upstream": {
+    "repository": "https://github.com/xiaowu0162/LongMemEval-V2",
+    "harness_commit": "2cc8c540bdb87fe6761629b585e727e1c4704520"
+  },
+  "dataset_revision": "DATASET_REVISION",
+  "tier": "small",
+  "files": {
+    "questions.jsonl": "SHA256",
+    "trajectories.jsonl": "SHA256",
+    "haystacks/lme_v2_small.json": "SHA256"
+  }
+}
+```
+
+The smoke manifest fixes question IDs, their upstream source order, and coverage
+of the published ability categories. Run the preflight against a new output
+directory:
+
+```bash
+uv run --project evaluation powercontext-eval longmemeval-v2 smoke \
+  --harness-root /path/to/LongMemEval-V2 \
+  --data-root /path/to/longmemeval-v2-data \
+  --dataset-lock evaluation/locks/longmemeval-v2-small-v1.dataset-lock.json \
+  --smoke-manifest evaluation/locks/longmemeval-v2-small-v1.smoke.json \
+  --output-dir /path/to/new-smoke-artifacts
+```
+
+The command refuses a harness checkout at a different commit, mismatched input
+hashes, invalid or incomplete smoke coverage, and an existing output directory.
+It writes `manifest.json` and `subset.json`, both labelled as a smoke subset.
+
 ## Configuration files
 
 Only the environment file configures the evaluation platform. The other files either belong to Codex or are

evaluation/locks/longmemeval-v2-small-v1.dataset-lock.json

@@ -0,0 +1,14 @@
+{
+  "schema": "powercontext.longmemeval-v2-dataset-lock.v1",
+  "upstream": {
+    "repository": "https://github.com/xiaowu0162/LongMemEval-V2",
+    "harness_commit": "2cc8c540bdb87fe6761629b585e727e1c4704520"
+  },
+  "dataset_revision": "f152293e235517d504809563c833d7190b8c713b",
+  "tier": "small",
+  "files": {
+    "questions.jsonl": "0a3ae5ebea938c24d7800e1e0b0828e08ae1646f939a53853b2b8cdc08e292b7",
+    "trajectories.jsonl": "363cec9a8e87aa8d9101ce4e600aadbf7031d674056ebe4f969e8424abc5f3c6",
+    "haystacks/lme_v2_small.json": "9b5301defb23a088a5f06e45ff8d5f35e569d78305a66d492046a9fff9b46593"
+  }
+}

evaluation/locks/longmemeval-v2-small-v1.smoke.json

@@ -0,0 +1,46 @@
+{
+  "schema": "powercontext.longmemeval-v2-smoke.v1",
+  "tier": "small",
+  "cases": [
+    {
+      "question_id": "01307e07",
+      "ability": "dynamic_state"
+    },
+    {
+      "question_id": "025db8ef",
+      "ability": "workflow_knowledge"
+    },
+    {
+      "question_id": "057a2d4d",
+      "ability": "static_state"
+    },
+    {
+      "question_id": "059974dd",
+      "ability": "premise_awareness"
+    },
+    {
+      "question_id": "18b91103",
+      "ability": "environment_gotchas"
+    },
+    {
+      "question_id": "00aa905a",
+      "ability": "dynamic_state"
+    },
+    {
+      "question_id": "01f6e679",
+      "ability": "premise_awareness"
+    },
+    {
+      "question_id": "0574c69a",
+      "ability": "environment_gotchas"
+    },
+    {
+      "question_id": "05cce9b3",
+      "ability": "static_state"
+    },
+    {
+      "question_id": "07ab3723",
+      "ability": "workflow_knowledge"
+    }
+  ]
+}

evaluation/src/powercontext_eval/benchmarks/longmemeval_v2/__init__.py

@@ -0,0 +1,15 @@
+# Copyright (c) 2026 OceanBase.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Pinned LongMemEval-V2 input validation and smoke preparation."""