Follow-up to #144. Blocked by #144.\n\n## Goal\n\nOptimize exact zero-point-aware XDNA INT8 to INT32 MATMUL throughput while preserving the released TOSA numerical contract, direct caller-buffer binding, and load-time compilation boundary.\n\nThe current implementation is a correct one-worker baseline. On the reference 1022:17f0 XDNA2 NPU with the v2026.08 toolchain, a 1x64x64 by 1x64x32 MATMUL with zero points [-2, 3] completes in 334.065 microseconds p50 and 343.723 microseconds p95 across 20 warmups and 200 samples, or 0.785 effective GOPS. Every INT32 output matches the shared exact oracle, all 660 measured bindings are direct, and submission performs zero explicit transfer bytes.\n\n## OpenVINO precedent and invariants\n\nRetain the same guest-visible arithmetic as OpenVINO:\n\n- calculate each term as (a - a_zero_point) * (b - b_zero_point);\n- accumulate exactly into INT32 with no floating conversion;\n- compile and plan at load_program or offline catalog time, never during submit;\n- bind caller-owned input/output allocations directly, with no hidden bounce buffer;\n- reject unsupported shapes instead of selecting a host fallback; and\n- publish reproducible ignored release-mode measurements rather than gating CI on wall time.\n\nOpenVINO can express widening and zero-point subtraction as provider graph nodes. XDNA must specialize equivalent arithmetic into AIE kernels. Worker striping, tile packing, native AIE instructions, FIFO topology, and a larger proven memory envelope are hardware-specific implementation details, not permission to change numerical results.\n\n## Investigation order\n\n1. Inspect Peano optimization remarks and generated AIE instructions to separate matrix-unit time from scalar INT8-to-INT16 adjustment and tile-packing time.\n2. Compare the current packed INT16 4x4x8 MMUL path with exact correction-term formulations that may retain more native INT8 work, proving overflow bounds for every admitted K and zero point.\n3. Reuse packed adjusted tiles across output tiles where legal instead of repeating scalar preparation.\n4. Overlap input DMA, adjustment/packing, matrix work, and output DMA with bounded FIFO tiles in flight.\n5. Stripe sufficiently large matrices across multiple AIE workers/cores while retaining a low-overhead one-core path for small tensors.\n6. Generalize the 16 KiB one-core envelope only after the multi-tile/multi-core memory plan is measured on metal.\n\n## Acceptance criteria\n\n- Extend the ignored release benchmark to cover the shared 2x3x2 case and representative tile-compatible 32x64x32 and 64x64x32 cases, with 20 warmups, 200 samples, p50/p95 admission and completion, effective GOPS, direct bindings, and explicit transfer bytes.\n- Record generated-code or trace evidence identifying the original bottleneck and why the selected design improves it.\n- Improve p50 completion throughput by at least 25% for 64x64x32 on the pinned reference machine without a material regression at 32x64x32; otherwise close with evidence showing the measured hardware/toolchain limit instead of merging complexity.\n- Preserve bit-exact outputs for the shared nonzero-zero-point corpus and expanded edge vectors covering -128, 127, asymmetric zero points, and the admitted K bound.\n- Preserve exact slot sizes, direct bindings, zero hidden staging, and zero submission-time transfer bytes.\n- Bound worker count, FIFO depth, retained allocations, and compiler-helper output; introduce no per-submit tensor-sized allocation or host arithmetic.\n- Keep artifact/cache identity sensitive to every generated-code, shape-policy, and topology change.\n- Run the full portable gates, complete XDNA hardware/fault suite, and before/after release benchmark; update performance documentation with the named device and toolchain.\n\n## Non-goals\n\n- Changing the guest-visible INT8 or zero-point contract.\n- Introducing approximate accumulation.\n- Adding a CPU fallback, submission-time repacking, or protocol change.\n- Treating the current microbenchmark as peak XDNA hardware throughput.
Follow-up to #144. Blocked by #144.\n\n## Goal\n\nOptimize exact zero-point-aware XDNA INT8 to INT32 MATMUL throughput while preserving the released TOSA numerical contract, direct caller-buffer binding, and load-time compilation boundary.\n\nThe current implementation is a correct one-worker baseline. On the reference 1022:17f0 XDNA2 NPU with the v2026.08 toolchain, a 1x64x64 by 1x64x32 MATMUL with zero points [-2, 3] completes in 334.065 microseconds p50 and 343.723 microseconds p95 across 20 warmups and 200 samples, or 0.785 effective GOPS. Every INT32 output matches the shared exact oracle, all 660 measured bindings are direct, and submission performs zero explicit transfer bytes.\n\n## OpenVINO precedent and invariants\n\nRetain the same guest-visible arithmetic as OpenVINO:\n\n- calculate each term as (a - a_zero_point) * (b - b_zero_point);\n- accumulate exactly into INT32 with no floating conversion;\n- compile and plan at load_program or offline catalog time, never during submit;\n- bind caller-owned input/output allocations directly, with no hidden bounce buffer;\n- reject unsupported shapes instead of selecting a host fallback; and\n- publish reproducible ignored release-mode measurements rather than gating CI on wall time.\n\nOpenVINO can express widening and zero-point subtraction as provider graph nodes. XDNA must specialize equivalent arithmetic into AIE kernels. Worker striping, tile packing, native AIE instructions, FIFO topology, and a larger proven memory envelope are hardware-specific implementation details, not permission to change numerical results.\n\n## Investigation order\n\n1. Inspect Peano optimization remarks and generated AIE instructions to separate matrix-unit time from scalar INT8-to-INT16 adjustment and tile-packing time.\n2. Compare the current packed INT16 4x4x8 MMUL path with exact correction-term formulations that may retain more native INT8 work, proving overflow bounds for every admitted K and zero point.\n3. Reuse packed adjusted tiles across output tiles where legal instead of repeating scalar preparation.\n4. Overlap input DMA, adjustment/packing, matrix work, and output DMA with bounded FIFO tiles in flight.\n5. Stripe sufficiently large matrices across multiple AIE workers/cores while retaining a low-overhead one-core path for small tensors.\n6. Generalize the 16 KiB one-core envelope only after the multi-tile/multi-core memory plan is measured on metal.\n\n## Acceptance criteria\n\n- Extend the ignored release benchmark to cover the shared 2x3x2 case and representative tile-compatible 32x64x32 and 64x64x32 cases, with 20 warmups, 200 samples, p50/p95 admission and completion, effective GOPS, direct bindings, and explicit transfer bytes.\n- Record generated-code or trace evidence identifying the original bottleneck and why the selected design improves it.\n- Improve p50 completion throughput by at least 25% for 64x64x32 on the pinned reference machine without a material regression at 32x64x32; otherwise close with evidence showing the measured hardware/toolchain limit instead of merging complexity.\n- Preserve bit-exact outputs for the shared nonzero-zero-point corpus and expanded edge vectors covering -128, 127, asymmetric zero points, and the admitted K bound.\n- Preserve exact slot sizes, direct bindings, zero hidden staging, and zero submission-time transfer bytes.\n- Bound worker count, FIFO depth, retained allocations, and compiler-helper output; introduce no per-submit tensor-sized allocation or host arithmetic.\n- Keep artifact/cache identity sensitive to every generated-code, shape-policy, and topology change.\n- Run the full portable gates, complete XDNA hardware/fault suite, and before/after release benchmark; update performance documentation with the named device and toolchain.\n\n## Non-goals\n\n- Changing the guest-visible INT8 or zero-point contract.\n- Introducing approximate accumulation.\n- Adding a CPU fallback, submission-time repacking, or protocol change.\n- Treating the current microbenchmark as peak XDNA hardware throughput.