From db20899d1cadad90f8f160c9e96183ba93f8d98c Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Fri, 17 Jul 2026 09:13:34 +0200 Subject: [PATCH] release: prepare 0.36.1 Patch on 0.36.0, shipping against the same SKaiNET engine 0.36.0. Versioned 0.36.1 rather than 0.37.0 to keep the documented lock-step rule (a transformers X.Y.Z ships against engine X.Y.Z) intact; there is no engine 0.37.0 to ship against. Covers everything on develop since 0.36.0: the BGE embedding support from #229 (CLS pooling, retrieval prefixes, integer-buffer-safe loads) and the vec2text / T5 beam search from #228. - bump VERSION_NAME to 0.36.1; CHANGELOG: cut 0.36.1 from Unreleased, folding in the BGE entry that was sitting there alongside the new beam-search notes - README: 0.36.1 current-release notes (BGE + beam search), 0.36.0 demoted to "builds on", BOM snippet bump, BERT and T5/GTR architecture rows refreshed - docs: embedding-model-coverage plan retargeted 0.37.0 -> 0.36.1 (that release number is retired; Phase 2 rows now say "next release" rather than naming a version the engine line hasn't reached); dsl-vs-handcoded T5 row notes greedy + beam decode Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 22 +++++++++ README.md | 46 ++++++++++++++----- .../pages/explanation/dsl-vs-handcoded.adoc | 2 +- docs/specs/embedding-model-coverage.md | 21 +++++---- gradle.properties | 2 +- 5 files changed, 70 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index db0aa05..3c8c6f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.36.1] — 2026-07-17 + +Patch on **0.36.0** (same SKaiNET engine 0.36.0). Two additions: **BGE embedding models** on the +BERT DSL path (CLS pooling + retrieval prefixes), and **beam search** for the T5 decoder and the +vec2text inversion loop. Both are additive — existing consumers are untouched, and the vec2text +greedy path is unchanged when both beam widths are 1. + ### Added - **BGE embedding models** (`BAAI/bge-small-en-v1.5` and siblings) run on the BERT DSL path: @@ -28,6 +35,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 loader gains a tensor filter ([SKaiNET#822](https://github.com/SKaiNET-developers/SKaiNET/issues/822)). - Design + traceable plan: `docs/specs/embedding-model-coverage.md` (E5 multilingual follows in Phase 2 — Unigram tokenizer). +- **Token-level beam search on the T5 decoder.** `T5Runtime.generateBeam(memory, numBeams, + maxLength, lengthPenalty)` returns up to `numBeams` sequences, best-first by length-normalized + log-probability. It shares a new `decoderLastLogits()` step with greedy `generate`, and adds + `logSoftmax` plus linear top-k helpers. There is still no KV cache, so decode cost scales + roughly linearly with `numBeams`. +- **Sequence-level beam search across correction rounds.** `Vec2TextInverter.invert(..., + sequenceBeamWidth, tokenBeams)` and `invertEmbedding()` keep `beamWidth` hypotheses between + correction steps, ranked by cosine similarity to the target embedding — the oracle the beam + exploits. `InversionModel.invertBeam` / `CorrectorModel.correctBeam` expose the top-N candidates + from each stage. +- Verified end-to-end on real gtr-base weights: at one correction step, beam (sequence width 3, + token beams 3) improves cosine **0.765 → 0.818** over greedy on the round-trip test's example + sentence, with a visibly closer reconstruction. Covered by `Vec2TextRoundTripTest` + (`invert_beamBeatsGreedy`), which skips unless `VEC2TEXT_MODELS_DIR` is set. ## [0.36.0] — 2026-07-12 @@ -788,6 +809,7 @@ Version-aligned with **SKaiNET 0.21.0**. Last published transformers release before the engine-aligned version line. See `git log v0.16.0..0.18.0` for details. +[0.36.1]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.36.1 [0.36.0]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.36.0 [0.31.0]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.31.0 [0.30.0]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.30.0 diff --git a/README.md b/README.md index fecd08c..f99e09a 100644 --- a/README.md +++ b/README.md @@ -90,8 +90,8 @@ Honest status — see the project-status note at the top of this README. | **Qwen 2 / 3** | DSL + loaders present; runs through the shared decoder path. Early; Qwen3 RoPE / QK-norm fixes landed in 0.23.2. | | **Gemma 2 / 3 / 3n** | DSL + loaders present (Gemma 4 via the SafeTensors path); has the most test coverage, but not verified end-to-end. | | **Apertus** | DSL + loaders present; declared end-to-end in 0.23.1, still early. | -| **BERT** | Sentence embeddings on the DSL path (`bertNetwork()` + `BertEncoderRuntime`, eager or traced/fused) — verified against sentence-transformers on MongoDB/mdbr-leaf. One-call `BertEmbeddingModel.fromHuggingFace(...)` with built-in Hub download. No text generation, no tool calling. | -| **T5 / GTR** | Encoder-decoder runtime (hand-coded, batch 1, no KV cache) + `GtrEmbedder`, powering the **vec2text** embedding-inversion pipeline — verified with a real-weights gtr-base round-trip test. | +| **BERT** | Sentence embeddings on the DSL path (`bertNetwork()` + `BertEncoderRuntime`, eager or traced/fused) — verified against sentence-transformers on MongoDB/mdbr-leaf. One-call `BertEmbeddingModel.fromHuggingFace(...)` with built-in Hub download; MEAN or CLS pooling and retrieval prefixes cover LEAF, BGE and E5-style models. No text generation, no tool calling. | +| **T5 / GTR** | Encoder-decoder runtime (hand-coded, batch 1, no KV cache) + `GtrEmbedder`, powering the **vec2text** embedding-inversion pipeline, with greedy and beam-search decoding — verified with a real-weights gtr-base round-trip test. | | **Voxtral** | TTS / voice; architecture code only — no runtime facade or CLI yet. | ### Near term @@ -106,9 +106,33 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.36.0** (against **SKaiNET 0.36.0**) — **BERT is now completely -defined in the SKaiNET NN DSL**, and the deprecated hand-coded eager BERT stack is **removed -(BREAKING)** in the same release: +The current release is **0.36.1** (against **SKaiNET 0.36.0**) — a patch on 0.36.0 with two +additions, both purely additive for existing consumers. + +**BGE embedding models** (`BAAI/bge-small-en-v1.5` and siblings) now run on the BERT DSL path: + +- **CLS pooling** — `BertPooling { MEAN, CLS }`, auto-detected from the sentence-transformers + `1_Pooling/config.json`. Pooling stays outside the traced graph, so OPTIMIZED mode and + StableHLO export are unaffected. +- **Query/document asymmetry** — `EmbeddingModel` gains `embedQuery` / `embedDocument` / + `embedDocuments`, and `PrefixedEmbeddingModel` applies the per-model retrieval instruction + prefixes (E5 `query: `/`passage: `, BGE query instruction) that these models need to score + correctly. `fromHuggingFace` wires them automatically. +- Design notes: [embedding-model-coverage](docs/specs/embedding-model-coverage.md). + +**Beam search** for the T5 decoder and the vec2text inversion loop — vec2text's main quality lever: + +- `T5Runtime.generateBeam(...)` does token-level beam over the decoder, returning candidates + best-first by length-normalized log-probability. +- `Vec2TextInverter.invert(..., sequenceBeamWidth, tokenBeams)` adds a sequence-level beam that + keeps several hypotheses across correction rounds, ranked by cosine similarity to the target + embedding. +- Both are **off by default** — width 1 keeps the existing greedy behaviour, so this is a + drop-in upgrade. On the round-trip test's example sentence, one correction step with beam + (sequence width 3, token beams 3) improves cosine **0.765 → 0.818** over greedy. + +It builds on **0.36.0**, in which **BERT became completely defined in the SKaiNET NN DSL** and the +deprecated hand-coded eager BERT stack was **removed (BREAKING)**: - `bertNetwork()` is a numerically complete `tokens → hidden-states` encoder: the new `BertEmbeddings` module adds the absolute-position and token-type embeddings the DSL definition @@ -126,12 +150,12 @@ defined in the SKaiNET NN DSL**, and the deprecated hand-coded eager BERT stack [CHANGELOG](CHANGELOG.md) and the [BERT-as-DSL explanation](docs/modules/ROOT/pages/explanation/bert-dsl.adoc). -0.36.0 also adds a **T5 encoder-decoder** runtime (`llm-inference/t5`) with `GtrEmbedder`, and a -**vec2text embedding-inversion** pipeline (`llm-inference/vec2text`) that iteratively reconstructs -text from a GTR embedding — verified end-to-end against real -`sentence-transformers/gtr-t5-base` weights. +0.36.0 also added the **T5 encoder-decoder** runtime (`llm-inference/t5`) with `GtrEmbedder`, and +the **vec2text embedding-inversion** pipeline (`llm-inference/vec2text`) that iteratively +reconstructs text from a GTR embedding — verified end-to-end against real +`sentence-transformers/gtr-t5-base` weights. That is the pipeline 0.36.1's beam search extends. -It builds on **0.35.0**, which added **FunctionGemma** self-compiled from the SKaiNET NN DSL: a +Both build on **0.35.0**, which added **FunctionGemma** self-compiled from the SKaiNET NN DSL: a one-dependency function-calling sLLM (`skainet-transformers-runtime-kgemma`) with an eager one-line API (`FunctionGemma.fromGguf(gguf).call("turn the light on")` → `ToolCall(set_lights, {state="on"})`, runs anywhere on CPU, no iree) **and** a no-Python compiled edge export @@ -164,7 +188,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.36.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.36.1")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/docs/modules/ROOT/pages/explanation/dsl-vs-handcoded.adoc b/docs/modules/ROOT/pages/explanation/dsl-vs-handcoded.adoc index 0985f07..2a7645d 100644 --- a/docs/modules/ROOT/pages/explanation/dsl-vs-handcoded.adoc +++ b/docs/modules/ROOT/pages/explanation/dsl-vs-handcoded.adoc @@ -112,7 +112,7 @@ The goal is to extend the DSL to support these patterns over time. |T5 / GTR (vec2text) |_none_ -|Hand-coded encoder-decoder (`T5Runtime`); relative-position bias and the per-step decode loop are not DSL-expressible yet +|Hand-coded encoder-decoder (`T5Runtime`, greedy + beam decode); relative-position bias and the per-step decode loop are not DSL-expressible yet |Voxtral |`voxtralBackboneNetwork()` diff --git a/docs/specs/embedding-model-coverage.md b/docs/specs/embedding-model-coverage.md index f25bbe0..026afed 100644 --- a/docs/specs/embedding-model-coverage.md +++ b/docs/specs/embedding-model-coverage.md @@ -75,14 +75,14 @@ Gitflow: `feature/*` → `develop` → `release/x.y.z` → tag (publishes to Mav | ID | Work item | Repo / branch | Release | Status | |---|---|---|---|---| -| E5BGE-0 | This design doc | SK-tr `feature/embedding-pooling-profiles` | 0.37.0 | in progress | -| E5BGE-1 | `BertPooling` (MEAN/CLS) + `1_Pooling` detection + tests | SK-tr, same branch | 0.37.0 | in progress | -| E5BGE-2 | SPI `embedQuery`/`embedDocument` + `PrefixedEmbeddingModel` + profiles | SK-tr, same branch | 0.37.0 | in progress | -| E5BGE-3 | BGE end-to-end verification (CLS, 384 dims, prefix) + smoke entry | SK-tr, same branch | 0.37.0 | pending | -| E5BGE-4 | Unigram tokenizer spike → placement decision | SK-tr (or engine per CP-3) | 0.38.0 | pending | -| E5BGE-5 | Unigram impl + multilingual parity + special-token generalization | per CP-3 | 0.38.0 | pending | -| E5BGE-6 | E5 end-to-end verification + docs + benchmark matrix entry | SK-tr | 0.38.0 | pending | -| E5BGE-7 | leaf-cli: `embedQuery`/`embedDocument` wiring + dim recorded in index | SK-leaf `feature/*` → `develop` | after 0.37.0 | pending | +| E5BGE-0 | This design doc | SK-tr `feature/embedding-pooling-profiles` | 0.36.1 | done | +| E5BGE-1 | `BertPooling` (MEAN/CLS) + `1_Pooling` detection + tests | SK-tr, same branch | 0.36.1 | done | +| E5BGE-2 | SPI `embedQuery`/`embedDocument` + `PrefixedEmbeddingModel` + profiles | SK-tr, same branch | 0.36.1 | done | +| E5BGE-3 | BGE end-to-end verification (CLS, 384 dims, prefix) + smoke entry | SK-tr, same branch | 0.36.1 | pending | +| E5BGE-4 | Unigram tokenizer spike → placement decision | SK-tr (or engine per CP-3) | next release | pending | +| E5BGE-5 | Unigram impl + multilingual parity + special-token generalization | per CP-3 | next release | pending | +| E5BGE-6 | E5 end-to-end verification + docs + benchmark matrix entry | SK-tr | next release | pending | +| E5BGE-7 | leaf-cli: `embedQuery`/`embedDocument` wiring + dim recorded in index | SK-leaf `feature/*` → `develop` | after 0.36.1 | pending | ## Checkpoints (upstream-change gates) @@ -90,14 +90,15 @@ Gitflow: `feature/*` → `develop` → `release/x.y.z` → tag (publishes to Mav pooling/prefix unit tests pass. *Engine changes required: none.* - **CP-2 — BGE verified** (E5BGE-3): `fromHuggingFace("BAAI/bge-small-en-v1.5")` produces 384-dim CLS-pooled embeddings; parity vs sentence-transformers reference - vectors; PR to `develop`; ship as **0.37.0**. *Gate: confirm no engine release is on + vectors; PR to `develop`; ship as **0.36.1**. *Gate: confirm no engine release is on the critical path — if anything surfaced, stop and file engine issues first.* **Gate outcome (2026-07-14):** one engine gap surfaced — `SafeTensorsParametersLoader` cannot skip BGE's persisted I64 `embeddings.position_ids` buffer ([SKaiNET#822](https://github.com/SKaiNET-developers/SKaiNET/issues/822)). Bridged by the interim `FloatSafeTensorsLoader` in llm-providers (permute-handler pattern from 0.36.0: local, documented, dropped when the engine fix ships). **No engine release on - the 0.37.0 critical path.** + the 0.36.1 critical path** — which is why this ships as a patch on the 0.36.0 engine line + rather than as 0.37.0. - **CP-3 — Tokenizer placement decision** (E5BGE-4): spike Unigram-from-tokenizer.json; measure normalizer drift on a multilingual corpus. **Decision recorded here**: transformers-side vs engine-side. *This is the explicit "are upstream-engine changes diff --git a/gradle.properties b/gradle.properties index 5ecbcf4..8d70c4f 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.36.0 +VERSION_NAME=0.36.1 POM_DESCRIPTION=SKaiNET-transformers