feat(compiler): add opt-in EVM compiler observation

docs/changes/2026-07-28-evm-compiler-observation/README.md

@@ -0,0 +1,98 @@
+# Change: Add opt-in EVM compiler observation
+
+- **Status**: Implemented
+- **Date**: 2026-07-28
+- **Tier**: Full
+
+## Overview
+
+Add an opt-in, structured observation record for the EVM multipass compiler.
+The record attributes compile time and intermediate-representation growth
+across frontend, MIR, CgIR, Phi elimination, register allocation, machine-code
+emission, and code publication.
+
+## Motivation
+
+Total JIT time and mapped executable size cannot explain which compiler stage
+caused a regression. Diagnosing stack-SSA compile growth required reproducible
+per-stage timing, before/after IR snapshots, and counters for stack lifting,
+Phi elimination, register allocation, fallback coverage, and emitted bytes.
+
+The observation must remain disabled by default and must separate its own
+measurement overhead from compiler work.
+
+## Impact
+
+### Affected Modules
+
+- `compiler`: phase timers, MIR/CgIR snapshots, lift/Phi/RA counters, structured
+  record emission, and feature-coverage aggregation.
+- `runtime`: retain actual emitted code bytes and dynamic-entry fallback counts
+  on the EVM module.
+- `tools`: collect paired S0/S1 records for a deduplicated fixture set.
+- `tests`: validate parsing, fingerprint selection, comparison, and summary
+  behavior of the observation runner.
+
+### Affected Contracts
+
+Setting `DTVM_EVM_COMPILER_OBSERVE` to a non-empty value other than `0` emits
+one `[DTVM_EVM_COMPILER_OBSERVATION]` JSON line per attempted EVM compilation.
+No record is emitted when the variable is unset or `0`.
+
+`EVMModule::getEmittedJITCodeSize()` reports actual emitted bytes, while
+`getJITCodeSize()` continues to report the executable mapping size.
+
+### Compatibility
+
+The feature is additive. Default compilation, generated code, and EVM execution
+semantics are unchanged.
+
+## Implementation Plan
+
+### Phase 1: Capture compiler structure
+
+- [x] Add opt-in phase timing with explicit observation-overhead accounting.
+- [x] Capture MIR and CgIR snapshots around lowering, Phi elimination, and
+  register allocation.
+- [x] Capture stack-lift, Phi-copy, spill, fallback, feature-coverage, and
+  emitted-code metrics.
+
+### Phase 2: Emit and consume structured records
+
+- [x] Emit schema-versioned records with deterministic bytecode fingerprints.
+- [x] Add an atomic, resumable S0/S1 collection and comparison tool.
+- [x] Add unit tests for parsing, target selection, comparison, and aggregation.
+
+### Phase 3: Verify
+
+- [x] Update affected module specifications.
+- [x] Run formatting checks.
+- [x] Build stack-SSA off and on configurations.
+- [x] Run frontend and Python tool tests.
+- [x] Verify observation-off emits no records and both modes preserve execution
+  output.
+
+### Validation
+
+- Strict `clang-format` dry run on every changed C++ source and header.
+- `git diff --check`.
+- Stack-SSA off and on release multipass builds with LLVM 15.
+- `evmJitFrontendTests`: 108/108 passed in each configuration.
+- `python3 -m unittest tests/tools/test_run_ssa_compiler_observation.py`:
+  4/4 passed.
+- `double_mod_origin.evm.hex`: successful execution with identical output in
+  both configurations; zero observation records by default and exactly one
+  schema-versioned record when enabled.
+
+## Compatibility Notes
+
+Consumers should treat unknown JSON fields as forward-compatible additions and
+key records by `schema_version`.
+
+## Risks
+
+- Observation can perturb timings. The record reports observation overhead
+  separately, and performance measurements must run with observation disabled.
+- Snapshot walks add work only when observation is enabled.
+- Logging-only feature counters are reported as zero when the corresponding
+  build instrumentation is unavailable.

docs/changes/README.md

@@ -53,6 +53,7 @@ Typical triggers:
 | 2026-04-14 | [from-raw-pointer-safety-checks](2026-04-14-from-raw-pointer-safety-checks/README.md) | Accepted | Light | Add null/alignment safety checks to `from_raw_pointer` in Rust bindings |
 | 2026-05-13 | [evm-ngram-macro-ops](2026-05-13-evm-ngram-macro-ops/README.md) | Implemented | Full | Initial EVM n-gram macro-op lowering and specialized keccak helpers for multipass JIT |
 | 2026-07-21 | [evm-memory-alias-and-expansion](2026-07-21-evm-memory-alias-and-expansion/README.md) | Implemented | Full | Stronger memory alias proofs, wider precheck/expansion coverage, DSE, load forwarding, grouping, and MCOPY roadmap |
+| 2026-07-28 | [evm-compiler-observation](2026-07-28-evm-compiler-observation/README.md) | Implemented | Full | Add opt-in structured EVM compiler phase and IR observations |
 
 Each active proposal lives in its own subdirectory. Browse `docs/changes/*/README.md`
 to see all current proposals, or use:

docs/modules/compiler/spec.md

@@ -189,6 +189,16 @@ and destination ends for expansion elision while retaining the original
 
 - Compilation start/end times are recorded in `utils::StatisticPhase::JITCompilation`
 - When `ZEN_ENABLE_LINUX_PERF` is enabled, perf JIT dump symbols (e.g., `EVMBB*`) are emitted for generated blocks
+- `DTVM_EVM_COMPILER_OBSERVE=1` emits one schema-versioned JSON record per EVM
+  module. It reports phase times, explicitly separated observation overhead,
+  MIR/CgIR snapshots, stack-lift/Phi/register-allocation counters, feature
+  coverage, and emitted code bytes. The record names the existing stack-SSA and
+  memory-plan build gates. In a build with
+  `ZEN_ENABLE_MULTIPASS_JIT_LOGGING`, feature coverage includes aggregate
+  memory-plan and arithmetic-path counters; without logging those counters
+  remain zero.
+- Observation is disabled by default and must not influence generated code,
+  compiler decisions, or deterministic execution.
 
 ---
 

docs/modules/runtime/spec.md

@@ -95,8 +95,11 @@ runtime propagates errors via `common::Error` / `common::ErrorCode`; common runt
 ## Compatibility Strategy
 
 1. **Config**: `RuntimeConfig` controls RunMode (Interp / Singlepass / Multipass), WASI, statistics, JIT thread count; changes require `validate()`
-2. **Compile options**: Behavior controlled by `ZEN_ENABLE_EVM`, `ZEN_ENABLE_JIT`, `ZEN_ENABLE_SINGLEPASS_JIT`, `ZEN_ENABLE_MULTIPASS_JIT`, `ZEN_ENABLE_BUILTIN_WASI`, `ZEN_ENABLE_CPU_EXCEPTION`, `ZEN_ENABLE_VIRTUAL_STACK`, `ZEN_ENABLE_DWASM`, etc.
-3. **ABI**: entrypoint assembly depends on `Instance`, `MemoryInstance` layout; layout changes require offset updates in assembly
+2. **EVM JIT code size**: `getEmittedJITCodeSize()` reports actual emitted
+   machine-code bytes; `getJITCodeSize()` retains the executable mapping size
+   used for runtime address bounds.
+3. **Compile options**: Behavior controlled by `ZEN_ENABLE_EVM`, `ZEN_ENABLE_JIT`, `ZEN_ENABLE_SINGLEPASS_JIT`, `ZEN_ENABLE_MULTIPASS_JIT`, `ZEN_ENABLE_BUILTIN_WASI`, `ZEN_ENABLE_CPU_EXCEPTION`, `ZEN_ENABLE_VIRTUAL_STACK`, `ZEN_ENABLE_DWASM`, etc.
+4. **ABI**: entrypoint assembly depends on `Instance`, `MemoryInstance` layout; layout changes require offset updates in assembly
 
 ## Cross-References
 

docs/modules/tools/spec.md

@@ -63,8 +63,17 @@ This module does not include: Build system (CMake), test framework (gtest/ctest)
 | Script | Responsibility |
 |--------|----------------|
 | function_selector.py | Compute Solidity function selector (keccak256 first 4 bytes) |
+| run_ssa_compiler_observation.py | Collect and compare target-bytecode compiler phase and IR observations for S0/S1 fixtures |
 | requirements.txt | Python dependencies (PyYAML, rlp, eth-hash, trie, etc.) |
 
+`run_ssa_compiler_observation.py` isolates each selected fixture, alternates
+S0/S1 process order, and enables the opt-in compiler observation record. It
+computes the target bytecode's deterministic FNV-1a fingerprint and requires
+exactly one matching record, preventing helper or subcall modules from being
+mistaken for the selected target. Output contains executable and fixture
+hashes, per-code structural ratios, dominant S1 phase counts, and an atomically
+resumable checkpoint.
+
 ## External Contracts
 
 | External Dependency | Use |

src/compiler/cgir/pass/interference_cache.h

@@ -14,6 +14,8 @@
 
 #pragma once
 
+#include <cstdint>
+
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Support/Compiler.h"
 #include <cassert>

src/compiler/cgir/pass/phi_elimination.cpp

@@ -21,7 +21,7 @@ struct CopyEdge {
 
 class PhiEliminationImpl {
 public:
-  void runOnCgFunction(CgFunction &MF) {
+  CgPhiElimination::Statistics runOnCgFunction(CgFunction &MF) {
     this->MF = &MF;
     MRI = &MF.getRegInfo();
     TII = &MF.getTargetInstrInfo();
@@ -35,6 +35,7 @@ class PhiEliminationImpl {
     for (CgBasicBlock *BB : Blocks) {
       eliminateBlockPhis(*BB);
     }
+    return Stats;
   }
 
 private:
@@ -49,6 +50,7 @@ class PhiEliminationImpl {
     if (Phis.empty()) {
       return;
     }
+    Stats.PhiInstructions += Phis.size();
 
     std::unordered_map<CgBasicBlock *, std::vector<CopyEdge>> EdgeCopies;
     EdgeCopies.reserve(BB.pred_size());
@@ -58,6 +60,7 @@ class PhiEliminationImpl {
                  (Phi->getNumOperands() % 2) == 1 && "invalid lowered PHI");
 
       const CgRegister DstReg = Phi->getOperand(0).getReg();
+      Stats.PhiIncomingEdges += (Phi->getNumOperands() - 1) / 2;
       for (unsigned OpIdx = 1; OpIdx < Phi->getNumOperands(); OpIdx += 2) {
         CgOperand &SrcOp = Phi->getOperand(OpIdx);
         CgOperand &PredOp = Phi->getOperand(OpIdx + 1);
@@ -68,7 +71,10 @@ class PhiEliminationImpl {
     }
 
     for (auto &[Pred, Copies] : EdgeCopies) {
+      Stats.CandidateEdgeCopies += Copies.size();
+      const size_t CandidateCount = Copies.size();
       eraseIdentityCopies(Copies);
+      Stats.IdentityEdgeCopies += CandidateCount - Copies.size();
       if (Copies.empty()) {
         continue;
       }
@@ -79,6 +85,7 @@ class PhiEliminationImpl {
       }
 
       CgBasicBlock *SplitBB = createSplitEdgeBlock(*Pred, BB);
+      ++Stats.SplitCriticalEdges;
       emitParallelCopies(*SplitBB, SplitBB->end(), Copies);
       addUnconditionalBranch(*SplitBB, BB);
     }
@@ -150,6 +157,7 @@ class PhiEliminationImpl {
     llvm::MutableArrayRef<CgOperand> OperandRef(Operands);
     MF->createCgInstruction(BB, InsertPt, TII->get(llvm::TargetOpcode::COPY),
                             OperandRef);
+    ++Stats.EmittedCopyInstructions;
   }
 
   void emitParallelCopies(CgBasicBlock &BB, CgBasicBlock::iterator InsertPt,
@@ -196,11 +204,12 @@ class PhiEliminationImpl {
   CgFunction *MF = nullptr;
   CgRegisterInfo *MRI = nullptr;
   const llvm::TargetInstrInfo *TII = nullptr;
+  CgPhiElimination::Statistics Stats;
 };
 
 } // namespace
 
-void CgPhiElimination::runOnCgFunction(CgFunction &MF) {
+CgPhiElimination::Statistics CgPhiElimination::runOnCgFunction(CgFunction &MF) {
   PhiEliminationImpl Impl;
-  Impl.runOnCgFunction(MF);
+  return Impl.runOnCgFunction(MF);
 }

src/compiler/cgir/pass/phi_elimination.h

@@ -4,14 +4,24 @@
 #pragma once
 
 #include "compiler/common/common_defs.h"
+#include <cstdint>
 
 namespace COMPILER {
 
 class CgFunction;
 
 class CgPhiElimination : public NonCopyable {
 public:
-  void runOnCgFunction(CgFunction &MF);
+  struct Statistics {
+    uint64_t PhiInstructions = 0;
+    uint64_t PhiIncomingEdges = 0;
+    uint64_t CandidateEdgeCopies = 0;
+    uint64_t IdentityEdgeCopies = 0;
+    uint64_t EmittedCopyInstructions = 0;
+    uint64_t SplitCriticalEdges = 0;
+  };
+
+  Statistics runOnCgFunction(CgFunction &MF);
 };
 
 } // namespace COMPILER