Feat/silicon path runtime - #1
Merged
Merged
Conversation
…ised Replace task 7's standalone --force-dynamic qexe (build_sim_qexe) with a QuRT-hosted shared object (build_sim_so), dlopen'd by the SDK's own prebuilt run_main_on_hexagon_sim under a real booted QuRT kernel. The standalone qexe could link and reach hexlib_iface_open but could never reach a successful start(): VTCM's manager needs real QuRT thread/clock primitives a standalone qexe cannot host. Under this build, hexlib_iface_start now succeeds and hexlib_iface_hwinfo reports arch=75, vtcm_size=8388608. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 7b replaced the standalone qexe with a QuRT-hosted module, because the qexe could never acquire VTCM and so could never reach a successful start(). It correctly left three stale references alone as out of its scope. Fixing them here, because Task 8's brief is the first thing its implementer reads and it was wrong on two counts. TASK 8's INTERFACES REWRITTEN. build_sim_qexe / run_sim_qexe / SIM_LINK_FLAGS / SIM_LINK_EXTRAS are gone; the interface is now build_sim_so + write_qurt_sim_configs + sim_qurt_command, and the sim boots a real QuRT kernel which dlopens the module. Two requirements added that were learned by building it: 1. THE SUBPROCESS MUST SET cwd. Under QuRT packaging relative fopen READS resolve through --usefs but WRITES land in the launching process's real cwd. Get it wrong and hexlib_out.bin appears elsewhere, so the backend reads a stale file or none -- which presents as a kernel bug. Independently reproduced during review, not taken on faith. 2. NOTHING IN TASK 8 MAY BE NAMED "FastRPC". The simulator omits the qaic stub, so calls bind directly to the skel and no marshalling happens. A FastrpcBackend would re-create precisely the false belief the section 0.1 correction removed. Renamed to hexlib/exec/dsp.py, DspSimBackend, test_dsp_sim.py, and the test module docstring now states what is and is not exercised. Also: the plan's architecture line and file-structure table now describe the module rather than a qexe; session.c's stale build_sim_qexe reference is corrected; and the spec's build table records WHY the qexe was deleted rather than just dropping it, plus the cwd trap in section 10.1 so it is found by someone reading the design rather than only by someone reading a C comment. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
It runs but can never acquire VTCM, so it can never reach a successful start(). Kept for the flags and the v68-against-v75 library facts, and so the reason it was abandoned is not rediscovered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… writes land .gitignore and build.py's write_qurt_sim_configs/sim_qurt_command docstring fixes from review (name-scoped osam.cfg/q6ss.cfg ignore rules, corrected claim, and the cwd requirement for sim_qurt_command's caller) landed in the shared working tree via the coordinator's own commits. This commit adds the test that holds the gitignore claim to account (git check-ignore, not a Python re-implementation) plus a bug it caught in my own test code: two @SDK tests were calling tc.run() with no cwd, so every full-suite run from the repo root wrote hexlib_rsp.bin/hexlib_out.bin into the repo root itself, silently modifying a tracked file. Fixed by running both under a small cwd contextmanager pinned to the test's own out_dir, per sim_qurt_command's own new docstring. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MY MISTAKE, and worth recording rather than quietly deleting. I ran `git add -A` in the coordinator while a subagent was mid-edit in the same working tree. Two consequences, both mine: 1. c5016e6 committed hexlib_out.bin (0 bytes) and hexlib_rsp.bin (32 bytes) -- simulator run artifacts, not source. Removed here and added to .gitignore along with hexlib_in.bin and hexlib_batch.bin, so the whole family is covered rather than just the two that happened to get caught. 2. c67654c, labelled as a docs commit, also carries that subagent's in-progress build.py, test and .gitignore edits. The CONTENT is correct and reviewed -- it is the .gitignore-enforcement fix and the cwd docstring that were asked for -- but it is attributed to the wrong commit and the wrong author. Not rewriting history to fix a label: the branch is shared with running agents, and surgery across three commits to correct attribution risks more than the mislabelling costs. Recording it instead. ROOT CAUSE, so it does not recur: `git add -A` in a tree with concurrent writers stages whatever happens to be there, including generated files and other people's half-finished work. Explicit paths only from here. WHY THE ARTIFACTS WERE IN THE REPO ROOT AT ALL is itself the finding: under QuRT packaging a sim run's relative fopen WRITES land in the launching process's cwd, not in --usefs. A full-suite run from the repo root therefore drops them here. The subagent found and fixed the two @SDK tests that were doing it by pinning cwd; this ignore block is the second layer, so the next such test cannot dirty the tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…r, and an unmapped fd refused
…ct what they claim
main.c gained the three device-farm-facing modes test_on_device.py was already written against: --unmapped (allocates rpcmem as usual but skips the hexlib_iface_mmap registration call, mirroring simhost.c, so the DSP's table lookup has nothing to find and must refuse with HEXLIB_DSP_ERR_UNMAPPED), a printed cycles_total after --self-test's PASS line, and --coherency-check (pre-writes a sentinel into the output buffer, runs scale_fp16 with factor=0.0, and reports cycles_total alongside the sentinel verdict so a dispatch bug, a coherency miss, and success are all distinguishable from stdout -- see the design doc's corrected §6.1). Ten new function-scoped offline tests in test_host_source.py cover the new argument parsing and structure; each was mutation-verified by temporarily breaking the corresponding behavior and confirming the test failed, then reverting.
…d hours ago The whole-branch review found docs/STATE.md and test_on_device.py's own docstrings still stating as CURRENT that hexlib_run has no --unmapped flag, never prints cycles_total, and cannot plant a sentinel. All three landed in f9edc3b. A reader following STATE.md would conclude stage 3 cannot run yet -- on exactly the page a human reads before deciding whether to spend non-renewable device minutes. The ruling to have Task 12 match test_on_device.py's assertions rather than edit them was right; leaving the surrounding prose was not. Rewritten rather than deleted, because the gaps are the reason the flags exist. ALSO FIXED: `python -m hexlib.device.qdc.run` is referenced as the step-6 command in both STATE.md and the plan, and NO SUCH MODULE EXISTS. The real entry point is `hexlib test scale_fp16 --device qdc --timeout-min 20 --yes`. A first-time operator would have got ModuleNotFoundError. Task 12's own report noticed this and it was never propagated -- so the note now says explicitly that no such module exists, not merely what the right command is. AND THE CORRECTION THAT MATTERS MOST, now recorded in the on-device test itself rather than only in the design doc: THE SENTINEL ALONE DOES NOT DISCRIMINATE. A silently no-op dispatch returning HEXLIB_DSP_OK is observationally identical to a coherency miss -- status OK, sentinel intact, both times. Only cycles_total, which a no-op cannot fake, separates them, and the three-state table is written out where whoever reads that test will see it. Two limits also stated rather than implied: riding on scale_fp16 is not kernel-independent, and only the DSP-write -> host-read direction is covered. STATE.md additionally now records the three whole-branch Criticals, including that the arch cross-check could never pass and that its test asserted the comparison EXISTED rather than that its operands were commensurable -- source assertions cannot see incommensurable operands, so any guard comparing two independently produced values needs a behavioural companion test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… a failed job exited 0 Critical 1: hexlib_open compared the skel's plain-decimal __HEXAGON_ARCH__ against the driver's BCD-nibble-packed ARCH_VER directly (75 != 0x8c75 == 35957, unconditionally), so every device session would have refused before measuring anything. Added hexlib_decode_bcd_arch (ported from llama.cpp's htpdrv_get_arch) and compare against its output instead. Critical 2: --coherency-check's "expected zero" test was a bit-exact compare against +0.0, but the self-test's own negative inputs make x * 0.0f == -0.0 in IEEE-754 -- healthy hardware was misreported as a coherency miss. Now compares magnitude (fabsf). Also added a real bit-exact check that surviving bytes are the sentinel before calling anything "unchanged", so a garbled/partial buffer gets its own distinct verdict (buffer_garbled) instead of being folded into a coherency claim. Critical 3: _qdc_submit returned 0 the moment results.xml was fetched, without ever parsing it -- a job whose tests all failed, or that collected zero tests, produced a green CLI. Now parses the JUnit XML and requires tests > 0, failures == 0, errors == 0, and that the fetched logs actually contain hexlib_run's own cycles_total= and --self-test PASS lines. Important 7: skel_vtcm.c requested HMX unconditionally while every session passes n_hmx = 0; now guarded by ctx->n_hmx > 0. Mutation-verified: reverting the Critical 1 decode (raw ARCH_VER compared directly) fails test_session_arch_decode.py and the rewritten test_host_source.py assertion; reverting Critical 3's parsing (return 0 unconditionally) fails 8 of 10 new test_cli_qdc_results.py tests, including the zero-tests case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ake one 75 of the encoder's 259 real-work plan steps take a q4_0 weight, and hexlib has no checkpoint loader -- so no q4_0 buffer could be produced at all. `RunnerSpec`'s own message says "hexlib does not quantize here", and that stays true of the serializer: a `RawTensor` is bytes somebody else quantized. This module is that somebody, for tests and for the end-to-end encoder run. It is not on any hot path. Transcribed from ggml-quants.c's `quantize_row_q4_0_ref` and ggml-common.h's `block_q4_0` (MIT), read in place from ../llama.cpp. New ATTRIBUTION row; nothing imported, nothing built against. THREE HAZARDS, each producing a plausible wrong answer rather than an error, and each now bound by a test that was verified to catch it: 1. THE NIBBLE PAIRING IS j WITH j+16, NOT j WITH j+1. The low nibble of byte j holds element j and the high nibble holds element j+16 -- the two halves of the block, not adjacent elements. Adjacent pairing round-trips to a PERMUTATION of the right values: right norm, right histogram, wrong everywhere. Mutation fails 7 tests. 2. `d` IS SIGNED and the divisor is -8. `max` keeps the sign of the largest-magnitude element, so a block whose extreme is positive stores a NEGATIVE scale; `amax / 8` flips the sign of every dequantized value in it. Asserted directly on the stored bytes, not only through a round trip. Mutation fails 5 tests. 3. THE ROUNDING IS TRUNCATION AFTER ADDING 8.5, and this one is why the mutation pass was worth running. `np.round(x + 8.0)` and `np.trunc(x + 8.5)` differ ONLY where x is exactly a half-integer -- numpy's round is banker's -- which random data never hits, so swapping them passed every test in the file including the byte-for-byte comparison. The test now constructs input that lands on exact halves: an extreme of -8.0 makes d exactly 1.0 (exact in fp16, so the scale's own round trip changes nothing) and therefore id = 1.0, so the scaled values ARE the inputs and the boundary is hit directly. It also asserts that the two rounding modes genuinely disagree on that input, so the check cannot go vacuous. THE ORACLE IS A SECOND IMPLEMENTATION, NOT GOLDEN BYTES. A committed blob would pin the output without saying what it means, and the failure here is not "the bytes changed" but "the bytes are a valid q4_0 encoding of the wrong thing". The scalar reference is written from the upstream loop statement by statement, in Python, with no numpy vectorisation, so it shares no code and no broadcasting with the implementation. TWO DETAILS THAT ARE EASY TO GET SUBTLY WRONG AND ARE HANDLED: The scale is round-tripped through fp16 BEFORE it is used to quantize. It is stored as fp16, so the dequantizer sees the narrowed value; scaling by the fp32 original would quantize against a scale that does not exist on the wire and make this function's own round trip look better than any kernel's can be. An all-zero block has d == 0. llama.cpp guards it with `id = d ? 1/d : 0`, and this uses `np.divide(where=...)` rather than `np.where(cond, 1/d, 0)` -- the latter evaluates both branches, so it computed an inf and emitted a divide-by-zero warning on every zero block before discarding it. Correct either way, but a warning that always fires is a warning nobody reads. The round-trip bound is real rather than a smoke test: 16 levels spanning [-8d, 7d] means a step of |d| and a worst case of about half a step, so the error is asserted under amax/8 PER BLOCK -- a global bound would be dominated by whichever block had the largest values -- and also asserted NOT to be trivially small, which would mean the dequantizer was reading back a stored copy. 13 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…SP path
This is the thing the per-kernel gates and the per-op simulator tests could not
say. Both of those drive ONE op at a hand-chosen shape. What that leaves uncovered
is everything that only exists between ops:
- a kernel correct in isolation and wrong on the shape the encoder hands it (the
layernorm gate ran R=4; the encoder needs R=256);
- a spec whose scalars fit the harness's shape and not the graph's;
- an op whose output feeds another kernel rather than an assertion, so a
systematic error is only visible after it has propagated;
- a variant routed to the wrong kernel for SOME of its ops -- `transpose` is two
kernels and this graph uses both, interleaved, which nothing else exercises.
Two runs of `interpreter.run` over the same compiled plan and the same feeds: one
with DSP backends for every kind that has a kernel, one with none so the reference
runs throughout. The comparison is over the encoder's DECLARED OUTPUTS, after every
intermediate has been through the DSP and back.
`interpreter_backends()` in exec/dsp.py is the new wiring. The variant is resolved
PER CALL from the op's own attrs, which is why it cannot be a dict of pre-bound
kernels: `transpose` is two kernels and which one an op needs is in its `perm`.
Raw inputs are quantized on the way in through the new `exec/quant.py`, since the
interpreter works in fp32 and the registry maps q4_0 to float32.
ITS DOCSTRING RECORDS THE TRAP IN ANY ACCURACY CLAIM MADE THIS WAY. Once
matmul_epilogue is registered, the DSP will compute with a 4-BIT weight while the
reference has the fp32 one, and 4-bit error is ~1/16 of each block's range --
orders of magnitude larger than any kernel bug worth hunting. A comparison against
the fp32 reference would measure the FORMAT, not the kernel. `dequantize_q4_0` of
the same bytes is what isolates the arithmetic, and that is why it exists.
ONE SIMULATOR LAUNCH PER OP, so this is a tiny config on purpose and the docstring
says so rather than implying a full-size run was done: 259 real-work ops at 256x256
is 259 QuRT boots plus patchify's ~2.0M cycles -- hours, for a signal a 58-step
graph gives in 75 seconds. The plan-walking driver that would fix it (one ELF, one
invoke, the whole batch) is separate work.
TWO THINGS THAT KEEP THE TEST FROM GOING QUIET. It asks `runner.SPECS` which kinds
have kernels instead of listing them, so a newly registered kernel moves ops onto
the DSP with no edit -- and it ASSERTS which kinds actually went to the DSP against
what `select()` says should have, so a spec that silently stops matching fails
loudly instead of falling back to the reference and comparing the reference against
itself. Consts are random rather than zeros or ones for the same reason: a zero
weight makes every matmul agree with anything.
A companion test pins the plan's shape with no SDK, so CI notices if the graph or
the pass pipeline changes what this compiles to. Without it the SDK-gated test
could start measuring something else and still pass. It also asserts both transpose
perms are present, so the routing claim cannot become vacuous.
MY OWN FIRST VERSION WAS BROKEN IN A WAY WORTH RECORDING. I paired the plan with the
PRE-fusion graph via `compile_graph` + `Compiled(...)`. That validates -- every
tensor a step names is declared -- and then fails at run time with
`KeyError: 'matmul_epilogue'` on the third step, because `interpreter.run` resolves
backends by walking `graph.ops` while `_run_op` looks them up by the PLAN step's
kind, and `matmul_epilogue` exists only after fusion. `compile_graph`'s docstring
says exactly this and names `compile_model` as the answer. The fix is one call; the
docstring in the test now explains it so the next person does not repeat it.
Currently 8 of 10 op kinds run on the DSP in this test. matmul and matmul_epilogue
still fall back, and the shape test names them as the two known gaps so a THIRD
gap cannot appear unnoticed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The encoder's 75 epilogue matmuls in one kernel: q4_0-quantized weights dequantized inline, fp32 accumulation across K, bias, then gelu (tanh or erf) applied after the bias -- gated at three shapes covering act=none, gelu_tanh and gelu_erf. HVX-compute, not HMX. The HMX path needs the SSR.XE extension-context bit set from inside kernel.c, and doing that from this standalone gate hung the simulator rather than faulting or proceeding. That is an environment difference from the reference projects, not a numerics problem, and it is recorded in kernel_api.h and spec.json rather than half-fixed. The widen-multiply-accumulate qf32 path here is real vector arithmetic and proves used_hvx_compute from the ELF. Five near-misses, all correctly rejected: bias applied after the activation instead of before, fp16 rather than fp32 accumulation, the two gelu variants swapped, the q4_0 scale read one block off, and the nibble order within a q4_0 byte reversed. gate PASS -- max abs err 2.44e-4, n_wrong 0, 14310406 cycles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…osts SIM_TIMEOUT_MAX_S 900 -> 1800. nearmiss_fp16_accumulate.c in matmul_fp16 needs 1195s. It was being killed at 900s and reported INCONCLUSIVE -- a near-miss that rejects correctly, scored as a gate failure, with a message guessing the kernel "may not terminate". It terminates: n_wrong 64, max_err 0.125, 185,300,776 simulated cycles against the real kernel's 995,714. The 186x is not pathological, it is what narrowing to fp16 on every multiply-add costs instead of once per output element, and that IS the bug the near-miss exists to model. Measured twice on an idle host (1195s standalone, 1218s in-gate). Its two siblings measured 825s and 862s -- they passed, but with 4-8% margin against the old ceiling, which is closer to a spurious failure than anyone had reason to know. The cost is that a genuine infinite loop now burns 30 minutes rather than 15. Accepted deliberately: the alternative was shrinking matmul_fp16's harness shape, and K=128 cannot move without destroying the adversarial element (one dominant product plus exactly 127 followers) that catches the fp16-accumulation bug at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`hexlib test` measures WALL time, which is a property of the kernel AND of whatever else the host is running. On a timeout the gate said "the kernel may not terminate" regardless. That sentence was wrong twice in one session on matmul_fp16: nearmiss_fp16_accumulate.c really did need 1195s against a 900s ceiling -- it terminates, and rejects correctly, given room. nearmiss_wrong_batch_stride.c had already PASSED in 825s, then ran past 1800s on a re-run because a game was launched 5 minutes into that stage. Byte-identical ELF, deterministic input, identical simulated work. Only the host changed. An hour went into re-running the gate to learn that. Simulated cycles would be the ideal evidence and are not available: the harness prints HEXLIB_KCYCLES at the END, so the stage that times out is exactly the stage with no cycle count. What can be measured while the process is alive is its CPU time. hexagon-sim is single-threaded, so on an unloaded host it accrues ~1.0 CPU-seconds per wall-second; 0.4 means the machine was taken away from it, and 0.98 means it was given everything and still did not finish -- which IS evidence about the code. SimLoadMonitor samples the hexagon-sim child by pid, so toolchain.run keeps its signature and its documented decoding behaviour, and every existing caller and monkeypatching test is untouched. psutil is optional and lazily imported (same reasoning as the QDC SDK): missing psutil degrades to "unmeasured", which reports the ambiguity rather than inventing a cause. Measured 0.95 share against a real unstarved child, so the 0.75 threshold sits well clear of ordinary scheduling noise. Three outcomes, three sentences, none of them a guess. 897 offline tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
QK^T and AV for the encoder's attention, batched over heads. No bias, no activation -- those belong to matmul_epilogue_fp16, which is a different op and a different kernel. HVX-compute, vectorised across the OUTPUT ROW rather than the K reduction: HVX has no horizontal-reduce worth using per output element here, so for a fixed (b, m) this accumulates C[m][:] += A[m][k] * B[k][:] over k, with A[m][k] splatted across a vector and B[k][:] a real vector load. Same shape as softmax_fp16's reduction with the vectorised and reduced axes swapped, because this op's reduction axis is not the last one. Accumulation is fp32 throughout, narrowed to fp16 exactly once at the end of each row's K loop. That single-rounding property is the one thing the baseline must not get wrong, since it is what every near-miss is judged against. HMX was attempted first and abandoned: the matrix-unit path needs the SSR.XE extension-context bit set from inside kernel.c, and setting it from this standalone gate's runtime hung the simulator rather than faulting or proceeding. An environment difference, not a numerics problem, recorded in kernel.c and spec.json rather than half-fixed. Three near-misses, all correctly rejected: fp16 rather than fp32 accumulation, B read as [N,K] instead of [K,N], and A's per-batch offset computed from m instead of b. gate PASS -- max abs err 9.77e-4, n_wrong 0, 995714 cycles, hvx-compute proven from the ELF. Needed the raised simulator ceiling from 9a2c99d: its fp16-accumulation near-miss costs 185,300,776 simulated cycles, 186x the kernel itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Stage 3 ran for the first time today. Two defects, both invisible to the offline suite, both found by submitting. 1. THE ARTIFACT TYPE WAS WRONG, so no job could be created at all. The zip uploaded as ArtifactType.TESTPACKAGE; QDC answers HTTP 400 "Appium tests requires 1 test script and at most 1 test package" -- the package is the OPTIONAL half, the script is the required one, and one package with zero scripts is not a valid Appium job. Now TESTSCRIPT, matching llama.cpp's own QDC runner against this same API. Costs no device minutes to get wrong: rejected before dispatch. 2. THE REPORT NEVER LEFT THE DEVICE. pytest.ini points --junitxml at the RELATIVE path TestLogs/results.xml, which lands wherever QDC's runner invoked pytest. QDC collects /data/local/tmp/QDC_logs -- exactly what utils.write_qdc_log writes to, and NOTHING called it with the report. Job 756124 reached Completed and returned one log file: a stale LauncherUI log from an unrelated job four days earlier. wait() polled its whole cap for a results.xml that may well have existed on the device. A conftest.py now copies it, in pytest_unconfigure (strictly later than the junitxml plugin's own sessionfinish). COPYING THE REFERENCE VERBATIM WOULD HAVE FAILED. llama.cpp writes a flat `results.xml`; QDC lists collected logs as `<job_id>/<name>`, and job.wait() matches a `TestLogs/results.xml` SUFFIX, which a flat name does not satisfy. Verified against _results_filename directly. write_qdc_log now creates the parent of its target rather than only QDC_LOG_DIR, which the nested name needs. Also: a qdc run stages a COPY of test_on_device.py under --out, so after two runs pytest found two files with one basename and refused to collect anything -- "Interrupted: 1 error during collection", the same zero-tests-run shape the root conftest.py already exists to prevent, by a second route. collect_ignore_glob covers it, and _work*/ is git-ignored so staged device binaries stop showing up as untracked. PROVEN: the auth header authenticates (it never had), the upload returns a uuid, and a job is accepted and reaches Completed. NOT PROVEN: that the report now arrives. That needs the next job. 897 offline tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he phone
utils.py argued at length that QDC provisions a Python ON the device and runs
the TestPackage there, so on-device paths could be named directly with no adb
prefix. It ended with "if that assumption is ever wrong, the fix belongs in
sh(), in one place." It was wrong. This is that one place.
WHAT SETTLED IT. llama.cpp's QDC runner -- working, on this same account and
the same TestFramework.APPIUM -- reaches the device only through adb:
run_adb_command is `adb shell`, its write_qdc_log does `adb push` into
/data/local/tmp/QDC_logs, and SCRIPTS_DIR (/qdc/appium) is a HOST path
holding the extracted zip. pytest runs on the runner.
WHAT THE OLD MODEL COST. Jobs 756124 and 756159 both reached Completed and
returned no logs of their own. write_qdc_log wrote /data/local/tmp/QDC_logs
ON THE RUNNER -- a directory QDC never collects, because it collects that
path from the PHONE. A report could be written perfectly and be invisible,
which is exactly what "no results.xml within the wait cap" looked like.
Worse, and quieter: test_binaries_are_present_and_executable ran `cp
hexlib_run libhexlib_skel.so`. On a Linux runner that SUCCEEDS -- it copies
an AArch64 binary between two host directories -- so the artifact "lands",
`ls -l` confirms it, and the failure surfaces later as an exec-format error
with nothing pointing back here.
sh() -> adb shell
push() -> adb push, new, from STAGE_DIR (this module's own dir,
not a hardcoded /qdc/appium)
write_qdc_log() -> adb push to the device, POSIX-joined so a Windows
runner cannot build TestLogs\results.xml
conftest.py -> opens the Appium session the framework expects, which
no hexlib job has ever established
Appium-Python-Client is pinned to llama.cpp's own version.
STILL NOT PROVEN: that any of this reaches us. Both prior jobs returned one
log file belonging to ANOTHER ACCOUNT's job (752055, 752090 -- 403/401 when
queried directly), relabelled under our job id. Whether that is QDC returning
foreign data or our jobs producing nothing is not decidable from here.
897 offline tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
THE DIAGNOSIS CAME FROM THIS ACCOUNT'S OWN WORKING JOBS. 744001 (hexbench)
and 743551 (llama.cpp) each publish 9-16 log files under a
`<job>/<subid>/` tree -- TestLogs/results.xml, TestLogs/install.txt,
appium_tests_stdout.txt, a screen recording, UserCollectedLogs/QDC_logs/*.
hexlib's three jobs (756124, 756159, 756206) published ONE file each, and it
was a LauncherUI log belonging to a different account's job. No subid tree at
all: the signature of a job whose test stage never started.
744001's own stdout gives the layout:
platform linux -- Python 3.11.12 <- the RUNNER, not the phone
rootdir: /qdc/appium
configfile: pytest.ini
tests/test_capprobe.py ... /qdc/appium/bin/capprobe
So: pytest.ini and requirements.txt at the root, tests under tests/,
binaries under bin/. hexlib staged everything flat. Every earlier fix on this
path -- the artifact type, the report copy, the adb model -- was necessary
and completely invisible, because nothing inside the tests could run.
Two of those earlier fixes were themselves wrong once the working jobs showed
their hand, and are corrected here:
* --junitxml is FLAT (`results.xml`). The framework publishes pytest's
report itself as <job>/<subid>/TestLogs/results.xml, which is what
job.RESULTS_MARKER matches. Both working jobs set exactly this.
* conftest's own pushed copy is flat too. Naming it TestLogs/results.xml
would give TWO collected names ending in that suffix, and
_qdc_check_results treats two matches as a failure by design -- the fix
would have manufactured the failure it was written to prevent.
The Appium fixture is now non-fatal. It is session-scoped and autouse, so
raising would error every test to obtain an object none of them use -- and
744001 ran against the phone over adb with no Appium session anywhere in its
stdout, so the session is plausibly unnecessary. Attempted, kept if it works,
recorded in the logs if not, never the reason a job reports nothing.
Also: wait() no longer dies on a transient API error. One HTTP 502 raised
straight out of it, through _qdc_submit, and killed the CLI -- discarding job
756206 after its minutes were spent, before anything was fetched. Errors are
now tolerated until the cap and COUNTED, so "the API was down throughout"
cannot masquerade as "the job produced nothing".
897 offline tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closed the known_gaps escape hatch (matmul, matmul_epilogue) in the shape-pinning test: it now asserts `not fallback` instead of `fallback <= known_gaps`, since Tasks 1-2 gave both kinds RunnerSpecs and every real-work op in the encoder now reaches the DSP. That change exposed a format mismatch, not a kernel bug: with matmul_epilogue dispatching, the DSP path multiplies by q4_0-quantized weights while the reference path used full-precision fp32 weights, so the two paths computed different functions. The end-to-end test failed at `merger.out: max relative error 0.1941` against its `rel < 0.05` tolerance -- roughly the q4_0 format's own ~1/16-of-range error, compounded through a 2-layer encoder. Fixed by pre-quantizing matmul_epilogue's weight consts in `_feeds` (round-tripped through quantize_q4_0/dequantize_q4_0 before either path runs), so both paths multiply by identical q4_0-quantized values. The DSP path's own on-the-wire quantization of an already-quantized array reproduces the same bytes, leaving only arithmetic and fp16 rounding between the two paths -- which is what the tolerance is for. Weight consts are identified narrowly, by walking the plan for matmul_epilogue steps and taking input index 1 (see opdefs/fused.py's `_infer`); activation and bias consts are left untouched. The `rel < 0.05` tolerance is unchanged. With the fix, the same end-to-end test passes at that tolerance: `pytest hexlib/tests/test_encoder_on_sim.py -q` -> 2 passed in 182.28s. Full offline suite unaffected: `pytest -q -m "not sdk"` -> 903 passed, 5 deselected, 401.53s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e that true
The vectorised column-block loop dereferenced an HVX_Vector* on both the B
row load and the C row store; that lowers to a 128-byte-ALIGNED vmem access,
correct only when N % 64 == 0. For any other N the hardware silently rounds
the address down instead of faulting, corrupting the first nvec64*64 columns
on both the read and (previously undiscovered) the write side. Switched both
to the repo's existing hvx_vmemu unaligned-load/store wrapper
(include/hexlib/hvx/hvx-base.h); no new wrapper added, no vendored header
touched.
spec.json's fallback claim ("a per-column scalar tail handles any N not a
multiple of 64") was false for the vectorised block; corrected, and the gate
shape widened from N=192 to N=200 (not a multiple of 64) so this can't
silently regress.
Added hexlib/tests/test_dsp_sim.py::test_matmul_is_correct_when_n_is_not_a_multiple_of_64
(N=96) as a fast (~seconds) regression guard, per Task 1's discovery.
1. spec.json's Bn/M/K/N ARE DECORATIVE. The gate's real shape is #defined as MM_B/MM_M/MM_K/MM_N in kernel_api.h, and nothing passes spec.json's values to the build. Task 6 was supposed to widen the gate to N=200 so the aligned-load bug fixed in cf25017 could never hide again; editing spec.json did not do that, and the file then CLAIMED a shape the gate does not run -- a fresh false claim in the file whose old false claim Task 6 existed to remove. Proved by changing N 192->200 and M 40->20 and finding the rebuilt near-miss ELFs byte-identical to the previous run's. Restored to the shape actually gated, with a note saying where the shape really lives. Widening it means editing MM_N and re-checking that the harness's adversarial element still works at the new N -- not done here. 2. hostload's diagnosis overclaimed. "The simulator was given 99% of one CPU, so the host was not starving it" reads as "the host was idle, so blame the code". A full CPU share only rules out DESCHEDULING. A process keeps accruing a CPU-second per wall-second while losing badly to memory-bandwidth contention, an SMT sibling, or cache pressure. Measured: a byte-identical near-miss ELF ran 1218s this afternoon and exceeded 1800s twice tonight, all at ~99% share; a game was running for the second of those. The message that shipped this morning would have sent the next reader hunting a kernel bug that is not there -- which is the same misattribution the feature was built to prevent, one level up. Task 6's gate is NOT green and its widening is NOT done; both remain open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hind kernel_api.h's MM_N (the real, #defined gate shape -- spec.json's params are descriptive only, see its own note) moves from 192 to 200. At N=192 every B row start (k*N*sizeof(hf) = k*384 bytes) happened to land on a 128-byte boundary for every k, so the aligned-HVX_Vector-dereference bug fixed in cf25017 was invisible to this gate even though it corrupted any real, non-multiple-of-64 shape. N=200 (200 = 3*64 + 8) breaks that coincidence and also exercises kernel.c's scalar tail in the same run. Verified before changing: the adversarial element (harness.c, C[1][5][7]) depends only on MM_K and MM_ADV_N=7, neither of which moved, so its numbers are unchanged; and the "all four distinct" argument that keeps nearmiss_wrong_batch_stride.c/nearmiss_transposed_operand.c from passing by accident holds at (3, 40, 128, 200) same as (3, 40, 128, 192) -- neither near-miss is HVX code, so it has no dependency on N's relationship to 64 either. Rewrote the harness.c header comment's N-multiple-of-64 paragraph, which no longer holds, to explain the new shape instead. Verified after: `pytest hexlib/tests/test_dsp_sim.py -k matmul` still 7 passed (batch path, unaffected by MM_N). Compiled kernel.c and all three nearmiss_*.c against the new shape via hexlib.build.build_kernel directly (compile+link only, no simulator) -- all four succeed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
STAGE 3 HAS RUN. Four Appium test-package jobs (756124, 756159, 756206,
756221) all reached Completed with result=UNSUCCESSFUL and published no logs
of their own, so nothing inside the package was ever observable. The
interactive session works, and this script is the recipe.
THREE THINGS ARE ALL REQUIRED and none is obvious from the SDK signature:
1. The key is QDC's, not yours. submit_session refuses your own key with
"could not be found or has not been created for user". QDC issues the
pair; ~/.ssh/qdc_id_<date>.pem. Probing showed the 2026-08-07 key
registered and the 2026-08-06 one not.
2. session_parameters=[SSHONLY] is what provisions SSH. Without it the
session reaches Running and never publishes sshConfigs -- two ten-minute
polls burned on that (756450, 756584).
3. It is an ADB TUNNEL, not a shell. sshConfigs returns
`ssh -L <port>:<host>:5037 -N sshtunnel@ssh.qdc.qualcomm.com`, which
forwards to the DEVICE'S ADB SERVER. Nothing runs remotely; everything
goes through a local `adb -P <port>`.
WHAT SILICON SAID, on SM8650 (Pineapple), device a652109b:
DECISIVE, and both were open questions:
* cycles_total=14267 in a user-mode unsigned PD -- NON-ZERO. STATE.md calls
this the most important thing a device job can report: SYSCFG.PCYCLEEN
cannot be set there, and a dead counter would have invalidated every cycle
figure stage 1 measured. It is alive.
* arch_ver 35957 (0x8c75), bit-identical to the simulator. job.py's fact 1
asserted this and had never checked it. unsigned_pd_support=1,
vtcm_total_bytes=8388608.
* The unmapped-fd refusal holds against real ION, not just the simulator's
mutation test: batch status 7, exit 4.
A REAL DEFECT no simulator test could catch:
* The skel builds as libhexlib_skel.so; FastRPC dlopens
libhexlib_iface_skel.so. Stage 1 links the skel directly rather than
loading it by name, so nothing offline can see it. hexlib_iface_open
failed rc -2147482618 until the file was pushed under both names. The
real fix belongs in runtime/build.py's _build_device_skel_so and is NOT
done here.
UNRESOLVED, and deliberately not called:
* --self-test returns 3859/4100 values not bit-exact; --coherency-check
exits 6 sentinel_unchanged. That is NOT a coherency verdict. main.c:94
says exit 6 is equally consistent with a kernel or generated entry
returning OK without writing its output, and its own table marks this row
NOT DISCRIMINATED. Settling it needs the skel-side echo op STATE.md
records as deferred.
Sessions bill for the whole timeout, not for what you use. complete_session
runs in a finally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Timestamp only -- the gate verdict, cycle count and near-miss rows are unchanged. Committed on its own so it is not noise inside the next change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`docs/` -- STATE.md, the design specs, the plans, the HVX tour, the research audits -- leaves the repository. The files stay on disk; the history purge is a separate mechanical step (`git filter-repo --invert-paths --path docs/`). THE PART THAT WAS NOT A ONE-LINE .gitignore CHANGE. `test_host_source.py`'s §6.1 coherency-table check read `docs/superpowers/specs/2026-08-10-silicon-path-runtime-design.md` at TEST time, as one of three parametrized sites. Untracking `docs/` leaves that test green on this machine -- the untracked copy is still on disk -- and red on every fresh clone, which is exactly the CI-only failure this suite exists to avoid. The design-spec site is dropped, leaving main.c and test_on_device.py, and the surrounding prose is corrected from "three copies" to two rather than left to describe a shape that no longer exists. Dropped, not made conditional: a `skipif` on the file's existence would be a check that silently protects nothing everywhere it actually runs, which is this project's own named failure mode. The remaining ~12 tracked files that cite a `docs/...` path do so in comments, as provenance for a decision. Those are left alone and the .gitignore entry records why: an unresolvable attribution is still worth more than none, but nothing tracked may DEPEND on a docs path at run time. Verified: hexlib/tests/test_host_source.py 28 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…one exit
259 ops in one blob instead of 259 blobs. `hexlib/exec/wholeplan.py` builds it;
`hexlib/tests/test_wholeplan.py` holds it to the invariants that only exist
once ops share a table.
NO C CHANGED, NO IDL CHANGED, NO NEW KERNEL. The wire already took a LIST of
ops (`wire.pack_batch(bufs, tensors, ops)`) and `skel_dispatch.c:171` already
looped `for (i = 0; i < hdr.n_ops; i++)` filling per-op status and cycles. The
only missing piece was a host-side builder, and this is it.
MEASURED, tiny config, one simulator launch: batch status OK, all 49 ops OK,
cycles_total=302,087,160 DSP-measured, and against the numpy reference
max relative error 1.1319e-03 with correlation 1.000000 -- the SAME figure the
49-launch per-op path produces. Same answer, 1/49th the launches.
At 256x256 the batch builds in 3.4 s: 259 ops, 49 reshapes elided, 512 wire
tensors (exactly MAX_TENSORS, zero headroom -- noted), 46,420-byte blob,
203.7 MB arena.
TWO THINGS THIS COST, both of which a per-op path never has to face:
* A CONST'S DTYPE COMES FROM ITS CONSUMER, NOT THE GRAPH. `pos_embed` is
fp32 in the graph and `add` declares both inputs fp16. `dsp.py` coerces at
call time (`np.ascontiguousarray(a, dtype=WIRE_DTYPE[dt])`); an arena has
no such moment, so it stages in the spec's dtype. Getting it wrong is
ERR_REQUIRES from the DSP -- the good failure, since the alternative is a
kernel reading fp32 bytes as fp16 and returning a shaped wrong answer.
Same for layout, which is per op-BUFFER (`spec.buf_layouts()`), not per
tensor: deriving it from the dtype sent q4_0_repacked for every quantized
weight and the DSP rejected it, correctly.
* REUSING THE PLAN'S VTCM OFFSETS AS ARENA ADDRESSES IS UNSAFE, AND THE
ALLOCATOR IS NOT AT FAULT. First attempt did exactly that and the output
correlation fell to 0.277 -- a wrong answer indistinguishable from a kernel
bug. Cause: this builder elides a reshape onto its input's storage, which
keeps the input live past the point the allocator was told it died, so
whatever legitimately owns that address in the meantime is overwritten. All
9 reshape pairs at the tiny config are assigned DIFFERENT slots by the plan.
Checked the allocator directly before blaming it: zero of its 88 slots
overlap in live range. `_refuse_unsafe_aliasing` now names the clashing
pairs instead of returning a corrupted arena, and BOTH facts are pinned --
the refusal, and the allocator's own disjointness, so the next reader
cannot mistake which one is broken.
The cost of not aliasing is memory: 203.7 MB rather than ~62 MB. Making it
safe means unifying reshape chains before the VTCM pass, which is a plan-pass
change and not this file's.
One test was written and deleted before commit: it called a
`wire.unpack_batch_header` that does not exist and `pytest.skip`ped on
`hasattr`, making it a test that could never run and never fail -- this
project's own named failure mode, in the file meant to catch it. It reads the
packed header directly now.
Verified: hexlib/tests/test_wholeplan.py 9 passed (8 offline in 1.8 s, plus the
single-invoke simulator run); ruff clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`_build_device_skel_so` linked `libhexlib_skel.so`. The URI qaic generates into `hexlib_iface.h` -- `hexlib_iface_URI`, which `session.c:163` hands to `remote_handle64_open` -- names the library after the IDL, so the device looks for `libhexlib_iface_skel.so`. On silicon `hexlib_iface_open` returned rc -2147482618 until the file was pushed under BOTH names by hand. That workaround was in a script; the defect was still in the build. NO SIMULATOR TEST COULD HAVE CAUGHT THIS, and that is the interesting part. Stage 1's build LINKS the skel directly -- `simhost.c` calls `hexlib_iface_open/_start/_invoke/...` as plain C functions, bound by the linker straight to `skel.c` -- so the FILENAME never participates in that path at all. A name only matters where something loads it BY name, and only a device does. The name is now DERIVED, not chosen: `device_skel_so_name()` builds it from the IDL stem, the same stem `run_qaic` already uses for `<stem>.h`/`_stub.c`/ `_skel.c`. One derivation, used by the linker, the artifact packer and the device test. `test_on_device.py` cannot import it -- it runs on the QDC runner where hexlib is not installed and imports `utils` as a flat module -- so it carries a literal `SKEL_SO`. `hexlib/tests/test_device_skel_so_name.py` reads that literal back and asserts it equals the derivation, because two independently written copies of a filename are exactly the drift that cost a session. It also asserts the IDL the stem names actually exists (a derivation from a stem naming no file would be a convention with nothing behind it) and that no live code still spells the old name as a string literal -- comments explaining the bug are deliberately allowed and several remain. THE LITERAL-SCAN TEST EARNED ITSELF IMMEDIATELY: it failed on first run against a `push`/`assert` pair in `test_on_device.py` that a search-and-replace had missed. That line would have shipped. Verified: 92 passed across test_device_skel_so_name.py, test_runtime_device_build.py, test_qdc.py and test_cli_device_flag.py -- including the SDK-gated tests that now link the real .so under the new name and read DT_SYMBOLIC, the Hexagon machine type and the skel symbols back out of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`run_batch_file` filled every buffer but the last from `--in` and wrote the last to `--out`. A whole-plan batch (hexlib/exec/wholeplan.py) is ONE buffer holding the weights, the activations, the graph input and the graph output at once -- so with n_bufs == 1 that convention left it never filled, and the DSP would have multiplied by whatever rpcmem handed back. The run would have succeeded and the output been garbage. The last buffer is now filled too when `--in` carries exactly its size. Exactly, not "as much as is left": a truncated `--in` is a refusal rather than a partly-staged arena. Passing nothing for it still means output-only, so the documented convention is unchanged for every existing caller. ONE BUFFER AND NOT TWO, DELIBERATELY. Splitting weights-in / activations-out would have fit the old convention with no C change at all, and it is wrong here: `simhost.c` patches the SAME fd into every buf_desc -- one rpcmem allocation for the whole batch -- so on the simulator two buffers share one address space while on a device they are separate allocations with independent offsets. The blob that runs on QDC has to be the blob the simulator already validated, byte for byte, or the rehearsal proves nothing. ALSO CLOSED, found while writing the above: a `--in` LONGER than the buffers it stages was silently accepted. That means the caller and the template disagree about the layout, and every byte that did land went to an offset derived from the same disagreement -- so it produced a full run and a plausible output file. It is now a refusal naming both sizes. Verified: hexlib/tests/test_host_source.py 28 passed (main.c's source assertions), and build_device_binary cross-compiles clean -- hexlib_run 29144 bytes, libhexlib_iface_skel.so 42608 bytes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…data
THE WHOLE ENCODER NOW RUNS ON SILICON AND MATCHES. SM8650 (Pineapple), 49 ops
in ONE invoke: max relative error 1.1319e-03, correlation 1.000000, cosine
0.99999967 against the numpy reference -- the SAME figure the simulator
produces from both its per-op and its single-invoke paths. Three transports,
one answer.
THE BUG. FastRPC keeps the two caches coherent for anything passed as an
invoke ARGUMENT. hexlib's data buffers are not arguments: they are mapped out
of band through fastrpc_mmap and named on the wire only by fd, so that no
address ever crosses between the processors. That design decision stands. Its
consequence is that FastRPC does not know the pages were written, and nothing
in this repository wrote them back -- zero cache calls existed in the skel.
THREE SYMPTOMS, ONE CAUSE, all measured on SM8650:
* --self-test: 3859 of 4100 fp16 values not bit-exact, against a simulator
giving exactly 0. Not garbage -- a PARTIAL write; the 241 correct values
were the lines that happened to get evicted. AFTER: PASS, 4100 bit-exact.
* --coherency-check: COHERENCY sentinel_unchanged, exit 6. The write never
reached DDR. AFTER: sentinel_overwritten, RC=0.
* the 49-op encoder: 179,124 arena bytes changed but merger.out -- the LAST
op's 1024 bytes -- came back all zero. AFTER: the figures above.
The third is what identified it, because it is ORDERED. Early writes landed
because later work evicted them; the final write was still in cache when the
invoke returned. Random corruption does not sort itself by age.
BOTH DIRECTIONS, and they are not the same operation. Invalidate before the
batch: the host has just written weights and inputs, and any line this DSP
holds from a PREVIOUS invoke on the same session is stale. Flush after: our
writes must reach memory before the host reads them. Flush alone works for
exactly one invoke per session and then silently computes on old data -- a
worse bug, because it needs two runs to appear.
The flush runs even when the batch FAILED: an op that died halfway still wrote
whatever it wrote, and leaving those lines in cache makes the wreckage
invisible from the host. It does not overwrite a real failure status.
Whole buffers rather than written ranges. qurt_memory.h warns the operation
takes the whole cache line either way -- "the contents of the adjoining buffer
can be flushed and invalidated if it falls in any of the cache line" -- so
partial ranges make neighbours a correctness question at every boundary. Narrow
it when a profile asks, not before.
HEXLIB_DSP_ERR_CACHE = 15 on both sides of the wire, its own status rather than
ERR_INTERNAL: every op ran and what the host reads may be STALE rather than
wrong, and nothing on the host can tell those apart without a code.
main.c now PRINTS the per-op results it was already receiving and never looked
at, plus cycles_total, and fails when fewer ops are reported than the template
carries. Their absence is why finding out whether op 74 had run meant diffing
659,712 arena bytes by hand.
skel_bufs.c carries a host-compiler fallback because
test_genentry_entry_probe.py compiles it with gcc and there is no qurt_memory.h
off-target. A fallback whose job is to do nothing is exactly what could
silently become what ships, and a source grep cannot tell which branch a build
took -- so test_device_cache_maintenance.py asserts qurt_mem_cache_clean is an
UNDEFINED symbol in the linked libhexlib_iface_skel.so, which is false the
moment the stub is what got compiled.
ALSO, and it is the first of its kind here: a real sim-vs-silicon cycle
comparison on identical work. Simulator 302,087,160 vs silicon 283,220,278 for
the same 49 ops -- the simulator is 6.7% high. Read it as a bare comparison,
not like-for-like: the simulator's batch path does not carry sim.py's
--timing --buspenalty 75 --busratio 2.
Every device DATA measurement taken before this is invalid. The cycle counts
are not: PCYCLE is a register read.
Verified: 921 passed before this change with 3 errors from the host probe,
which the guard fixes (test_genentry_entry_probe.py 9 passed);
test_device_cache_maintenance.py 4 passed including the SDK-gated artifact
check; test_wholeplan.py 9 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`--stage-dir` pushes a batch blob plus its arena, runs `hexlib_run --batch`, and pulls the arena back. That is what turned a QDC session from "run the scale_fp16 self-test" into "run the whole encoder", and it is the path that produced the 1.1319e-03 / correlation 1.000000 figure on SM8650. The arena is IN-OUT and the same file is both `--in` and what comes back: it carries weights and the graph input down, and the DSP writes every activation and the output into it. The push-under-both-names workaround is deleted. It existed because the build emitted libhexlib_skel.so while FastRPC dlopens the name in qaic's URI; `device_skel_so_name()` now derives the right one, so the script asks the build what it produced instead of guessing twice. If the workaround ever looks necessary again, the bug is in runtime/build.py and not here. `--self-tests` is opt-in rather than always-on. A session billed for its whole timeout should run what you came for; the self-test modes are a separate question and were being paid for every time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`_feeds_from_golden` lived in hexlib/tests/test_vision_oracle.py, keyed to that file's TINY_CFG and to an npz of random parameters. Every part of it -- the Conv3d flatten to [feat, embed], the fused-qkv split into three [H, H] blocks in q/k/v order, each nn.Linear transpose, the two folded tables -- is exactly what a REAL checkpoint needs. Writing a second copy for the checkpoint would have been a second chance to get a transpose wrong, and at hidden_size 768 every one of those matrices is square, so a forgotten transpose passes every shape check and returns a correctly-shaped wrong answer. `feeds_from_params(graph, cfg, get)` takes a getter instead of an npz and reads depth/hidden_size from the cfg instead of a module-level constant, so the same code serves the tiny golden and the 12-layer checkpoint. The test keeps its private aliases and supplies `image` itself -- that is a graph INPUT, not a const, and the mapping knows only about weights. THE PART A NAME-MATCHING LOADER WOULD GET WRONG, recorded in the module docstring because it is invisible from the parameter names: `pos_embed` and `rope_cos`/`rope_sin` are DERIVED, not copied. Upstream computes the positional embedding at run time as a bilinear resample of a learned grid, and the rotary tables from position ids; hexlib folds both to constants because the resolution is fixed at compile time. Matching names and copying tensors produces a graph that is complete, shape-correct, and wrong in three of its 203 consts. No behaviour change: the test calls the same code it used to contain. `pytest` was already an unused import in this file before the move; removed while here. Verified: test_vision_oracle.py + test_models_vit.py, 40 passed. The oracle's own gate still measures hexlib's eager path against transformers at 4.4703e-08. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MEASURED, not assumed. The shipped Qwen3.5-0.8B vision weights at 256x256,
against transformers, with fp32 arithmetic on BOTH sides so nothing but the
weight format differs:
q4_0 encoder output cosine 0.867606 max_rel 4.3170e-01
q8_0 encoder output cosine 0.999002 max_rel 7.9803e-02
for 1.89x the weight bytes (55.5 MB -> 105 MB). This commit is the format,
the quantizer and the wire registration; the kernel variant follows.
WHAT WAS TRIED FIRST AND REJECTED ON EVIDENCE.
* MIXED PRECISION. A sensitivity sweep on the real checkpoint says the error
is DIFFUSE, not concentrated: keeping the patch embedding, the merger AND
all of attention at 8 bits while the MLPs stay q4_0 still only reaches
0.913. Twelve layers of small errors compound; there is no dominant term to
target.
* SMALLER BLOCKS. block=8 spends 6 bits/value on more scales and reaches
0.937, where q8_0 spends 8.5 on mantissa and reaches 0.999. Bits buy more
as precision than as scale granularity here.
* A SCALE SEARCH inside q4_0's own 18-byte block. Worth 0.868 -> 0.896 for
zero extra bytes and NOT included here: quant.py's stated contract is
"matching llama.cpp block for block", and a search deliberately breaks it.
It also broke round-trip idempotence in my first attempt, which
test_encoder_on_sim depends on. It deserves its own change with that
compatibility question answered out loud.
A TINY-CONFIG SWEEP SAID THE OPPOSITE OF ALL OF THIS and nearly sent the work
the wrong way: at 2 layers with random weights it ranked the merger dominant,
put wq/wk at the eager-vs-torch noise floor, and made mixed precision look like
a 30x win. Two layers is not enough depth for diffuse error to compound, and
random normals have no outliers. The ranking was only trustworthy once the real
checkpoint was loaded.
THE QUANTIZER IS TRANSCRIBED FROM quantize_row_q8_0_ref (ggml-quants.c:276-299)
AND dequantize_row_q8_0 (:553-567), read in place from ../llama.cpp -- NOT
adapted from q4_0 above it. Three things differ and every one produces a
plausible wrong answer rather than an error, so each has its own test:
1. `d` IS POSITIVE. q4_0 stores signed_max/-8, so a block whose extreme is
positive gets a NEGATIVE scale -- it has a test asserting exactly that.
q8_0 stores amax/127, which has no sign. Carrying q4_0's trick negates
every value in the block.
2. THE ROUNDING IS roundf, half-away-from-zero. Not q4_0's trunc(x + 8.5)
(that exists only because q4_0 stores an unsigned nibble biased by 8) and
not np.round, which is banker's and disagrees on every exact .5.
3. NO BIAS AND NO NIBBLE PACKING. qs is a plain int8 per element in element
order, so none of q4_0's j-with-j+16 pairing applies.
Also clamped to [-128, 127]: amax/127 keeps |scaled| <= 127 in fp32, but `d` is
NARROWED to fp16 before use and narrowing downward makes the scaled value
slightly larger, so the extreme element can land on 128 and wrap to -128 --
flipping the sign of the largest element in the block. A 200-trial test over
six orders of magnitude covers it.
13 tests, including byte-for-byte agreement with a hand-transcribed scalar
reference that shares no code path with the implementation, and round-trip
idempotence (test_encoder_on_sim pre-quantizes its feeds so both paths multiply
identical values; a q8_0 that lost this would make that test FLAP rather than
fail).
TWO EXISTING TESTS FAILED AND BOTH WERE RIGHT TO. Strengthened rather than
merely updated:
* test_runtime_wire.py pinned `DTYPE_ID - WIRE_DTYPE == {"q4_0"}` as a
LITERAL. Now bound to WIRE_RAW -- the set that decides the same question
everywhere else -- plus a non-vacuity assertion. A literal is satisfied by
editing it; the binding is not.
* test_raw_q4_0_staging.py used the literal "q8_0" as its example of an
UNKNOWN dtype, so adding q8_0 turned it into an assertion that a supported
dtype raises. The stand-in is now chosen at run time from outside both
tables, which cannot go stale: the day it becomes known it stops being
picked.
test_graph_ir.py's exhaustive DTYPES pin also fired, which is that check
working: a dtype the graph accepts but the wire cannot carry is exactly what it
watches for.
genentry needed no change -- it selects `const unsigned char *` from WIRE_RAW
rather than from a literal, so q8_0 got the right C type for free.
Verified: 940 passed before this change; the four dtype-table files plus both
quantizers, 84 passed. ruff clean on every file touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
COMMITTED DELIBERATELY WITHOUT A RESULT.md, which every other kernel here has,
because it does not gate: the simulator faults before the harness prints a
verdict. What it found on the way is worth more than the code.
TWO REAL DEFECTS FOUND AND FIXED, both of which would have cost far more later:
1. `HMX_SET_BIAS` DOES NOT TAKE A BIAS TILE. It takes a 256-BYTE area, not a
32x32 one: `hmx_init_column_scales` (llama.cpp hmx-utils.h:19-23) writes one
HVX vector of per-COLUMN packed 32-bit words then one zero vector, and each
word is an fp16 PAIR -- low half a multiplicative scale, high half an
additive bias. That is what upstream's `Q6_V_vsplat_R(0x3c00)` means when
its comment reads "scale: 1.0, bias: 0.0 in FP16". Passing a 2048-byte
32x32 array, as the first version did, is a different operand entirely.
docs/hardware/hmx-int8.md recorded "the bias tile is a scale, not an
additive bias" from the earlier int8 probing. That was half the story: it
is BOTH, one pair per column.
2. TILE-SIZED ALIGNMENT. `mxmem` addresses a 2048-byte tile; the project's
standard HEXLIB_ALIGN is 128. The first fault was at badva0=04114a68, whose
low 11 bits are not zero.
WHERE IT STANDS. With both fixed it still faults, and the fault has MOVED to a
stack address (badva0=04115868) while the tiles are now 2048-aligned -- so it is
no longer the operands. Exception code 0x18 in ssr=80740018, ccr=00130000.
THE HYPOTHESIS THIS POINTS AT, and it is a re-plan rather than a bug fix: HMX is
not ENABLED in the standalone-ELF harness. Upstream runs HMX only inside a QuRT
protection domain with the unit acquired -- `HAP_compute_res_attr_set_hmx_param`
-- and every operand VTCM-resident (`vtcm_scales`, `vtcm_output_bufs`,
`htp_mm_hmx_vtcm_layout_build`). hexlib's own skel already has that branch and
has NEVER taken it: skel_vtcm.c:102 requests HMX only `if (ctx->n_hmx > 0)` and
session.c always passes 0, with a comment saying "this parameter is not
requested at all today. REVISIT THIS when an HMX kernel first lands."
If that is right, an HMX kernel CANNOT be gated through `hexlib test`'s
standalone path the way every HVX kernel here is, and has to be developed on the
QuRT-hosted batch path instead. That is a structural difference from every
kernel in this repository and is the next thing to settle -- before more kernel
code, not after.
The gate's own report is the honest one: "no verdict recovered from the
simulator: the harness never printed HEXLIB_VERDICT, so nothing was actually
checked. This is a failure, not a pass."
`nearmiss_split_packet.c` is kept even though nothing can run it yet: it
preserves the failure docs/hardware/hmx-int8.md spent four probe rounds on --
activation and weight loads issued as two packets instead of one, which does not
degrade the accumulator but CLEARS it, and reads as a hardware limitation rather
than a coding error.
Verified: 941 passed on the full suite (q8_0, committed separately, is green);
test_kerneldir.py + test_kernels.py 45 passed with this directory present.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit left this as a hypothesis: "HMX is probably not enabled in
the standalone-ELF harness". It is not a hypothesis. Reading
../llama.cpp/ggml/src/ggml-hexagon/htp gives the whole protocol, and a
standalone simulator ELF -- which is how EVERY other kernel here is gated --
has none of its four parts:
1. POWER, separate from HVX's own request and guarded on __HVX_ARCH__ >= 75:
HAP_power_set_HMX_v2 with set_power/power_up/set_clock, all three DCVS
corners at VCORNER_MAX and perf_mode HAP_CLK_PERF_HIGH (main.c:473-492).
2. ACQUISITION in the SAME compute-res attr as VTCM, not a separate one:
HAP_compute_res_attr_set_hmx_param(&attr, 1) before
HAP_compute_res_acquire (main.c:259-291).
3. AN EXPLICIT LOCK around every use: HAP_compute_res_hmx_lock(rctx) /
_hmx_unlock(rctx) (hmx-queue.c:17-30).
4. A DEDICATED THREAD owning that lock -- upstream queues all HMX work to
hmx_queue_thread, created only `if (n_hmx)` (main.c:386-394).
Whether the lock is thread-scoped, and so whether hexlib's single-threaded skel
can hold it inline instead of standing up a queue thread, is NOT settled and is
recorded as unsettled rather than guessed.
THE PIECES MAP ALMOST ONE-TO-ONE ONTO CODE THAT ALREADY EXISTS HERE, which is
why this is a short list rather than a redesign:
* llama.cpp's htp_iface_start(..., n_hvx, n_hmx, max_vmem) is the SAME
signature hexlib ported. n_hmx was always meant for exactly this, and
session.c passes 0 unconditionally.
* skel_vtcm.c:102 already carries requirement 2 --
`if (ctx->n_hmx > 0) HAP_compute_res_attr_set_hmx_param(&attr, 1);` --
written, commented "REVISIT THIS when an HMX kernel first lands", and never
once executed.
* Requirements 1 and 3 do not exist in hexlib at all.
CONSEQUENCE, and it is the useful part: an HMX kernel belongs on the
QuRT-hosted BATCH path, not on `hexlib test`'s standalone-ELF gate path. That
is a structural difference from all eleven existing kernels -- they are gated by
a program that boots, computes and prints a verdict with no protection domain
around it, and HMX cannot run in one. The kernel source committed previously is
probably fine; it was being run somewhere it can never work.
No code change here, only the record. Writing the four steps into hexlib before
knowing whether the lock needs its own thread would be building on the same kind
of assumption that produced the two defects in the previous commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ement
README.md, ROADMAP.md and CONTRIBUTING.md still described a repository with six
kernels, four of them dispatchable, 86 of 259 encoder ops covered, no full-size
PyTorch reference, and nothing ever executed on silicon. Every one of those is
now false. Since `docs/` is untracked and purged from history, the tracked files
are the only record a fresh clone gets, so they are the ones that have to be
right.
WHAT THE NUMBERS ARE NOW, all from the record rather than retyped from memory:
* TWELVE kernels through gates 1-5, with cycles and max abs error read out of
each `kernels/<name>/RESULT.md`: transpose_th 706, scale 886, add 1139,
cast 1176, rope_2d 1212, rmsnorm 2231, transpose_hd 5606, softmax 11292,
layernorm 111088, matmul 995714, patchify 2018331, matmul_epilogue
14310406.
* ALL 259 real-work ops dispatch. The other 49 of 308 steps are reshapes,
which need no kernel. Asked of `select()` by
test_encoder_dispatch_coverage.py, not counted -- the README had published a
counted number and the count was wrong three times.
* THE FULL-SIZE REFERENCE EXISTS. Real Qwen3.5-0.8B at 256x256 vs
transformers, fp32 both sides: 0.9999999999 full-precision, 0.999002 q8_0,
0.867606 q4_0, and fp16 ACTIVATIONS cost 0.999991 -- 15,000x less than the
weight format. README said "there is no full-size PyTorch reference yet".
* SILICON. The whole encoder in ONE FastRPC invoke on an SM8650: max rel
1.1319e-03, correlation 1.000000, cosine 0.99999967 -- the identical figure
the simulator gives from BOTH its per-op and single-invoke paths. Cycles
302,087,160 sim vs 283,220,278 silicon, 6.7% high.
THE THREE-WAY CHAIN IS NOW A TABLE, because that is the actual result and it was
nowhere in the tracked tree: PyTorch -> hexlib reference -> simulator (two
transports) -> silicon, each link compared against the one before it. Three
transports, one answer.
WHAT IS SAID JUST AS PLAINLY: every device run is the TINY config. The 256x256
encoder is a 46 KB blob over a 203.7 MB arena and has never run on a device. The
cycle comparison is two numbers under different flags, not a calibrated drift
figure. Gate 6 has a transport now but still no per-kernel drift record, so no
kernel may be marked gate-6 clear on the strength of the whole-encoder run --
CONTRIBUTING.md says so where it used to say silicon was unreachable.
kernels/hmx_matmul_fp16/README.md IS NEW, and it is the record that directory
was missing. It has no RESULT.md on purpose -- the simulator faults before the
harness prints a verdict, and hand-writing one would make an ungated kernel look
gated in the exact place a reader checks. The README carries the four things HMX
needs with their upstream line references, the two real defects already found
(HMX_SET_BIAS takes a 256-byte per-column fp16 scale/bias pair area, not a 32x32
tile; mxmem needs 2048-byte alignment, not HEXLIB_ALIGN's 128), why
nearmiss_split_packet.c is kept although nothing can run it, and the cheapest
next move.
ROADMAP.md's op-kind table is green in every row, and now says why `hexlib plan
--print` still lists all eleven kinds as NOT IMPLEMENTED: `OpDef.kernel` is a
separate registry that is still None everywhere and is NOT what decides
dispatch. That output is misleading rather than merely stale, so it is called
out where a reader will hit it.
No code changed. Verified: test_kerneldir.py + test_kernels.py 45 passed with
the new README in the kernel directory, and no test reads any of the four files
edited here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.