Build a floppy-sized Windows LLM CLI

.github/PULL_REQUEST_SUMMARY.md

@@ -0,0 +1,3 @@
+# FloppyLLM implementation
+
+This branch adds a self-contained 32-bit Windows TinyStories CLI under the formatted 1.44 MB floppy limit, plus reproducible quantization, size enforcement, configurable token/context defaults, Windows smoke tests, and tagged releases.

.github/workflows/build.yml

@@ -0,0 +1,99 @@
+name: Build floppy executable
+
+on:
+  push:
+    branches: [main]
+    tags: ["v*"]
+  pull_request:
+  workflow_dispatch:
+    inputs:
+      default_tokens:
+        description: Default maximum new tokens
+        required: true
+        default: "128"
+      default_context:
+        description: Default context allocation (0 uses model's 256)
+        required: true
+        default: "0"
+
+permissions:
+  contents: write
+
+env:
+  LLVM_MINGW_VERSION: "20260616"
+  LLVM_MINGW_SHA256: "a1f7968b48ba8d949194d6dee6c76f3cd0f61cba91658599af2c2c834a55ab87"
+  DEFAULT_TOKENS: ${{ inputs.default_tokens || '128' }}
+  DEFAULT_CONTEXT: ${{ inputs.default_context || '0' }}
+
+jobs:
+  build:
+    runs-on: ubuntu-22.04
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install converter dependency
+        run: python -m pip install numpy
+
+      - name: Build verified Q8 model
+        run: make model PYTHON=python
+
+      - name: Install pinned Windows cross-compiler
+        run: |
+          url="https://github.com/mstorsjo/llvm-mingw/releases/download/${LLVM_MINGW_VERSION}/llvm-mingw-${LLVM_MINGW_VERSION}-msvcrt-ubuntu-22.04-x86_64.tar.xz"
+          curl -L --fail -o /tmp/llvm-mingw.tar.xz "$url"
+          echo "${LLVM_MINGW_SHA256}  /tmp/llvm-mingw.tar.xz" | sha256sum -c -
+          mkdir /tmp/llvm-mingw
+          tar -xJf /tmp/llvm-mingw.tar.xz -C /tmp/llvm-mingw --strip-components=1
+
+      - name: Build single-file Windows CLI
+        run: |
+          make windows \
+            PYTHON=python \
+            WINDOWS_CC=/tmp/llvm-mingw/bin/i686-w64-mingw32-clang \
+            DEFAULT_TOKENS="${DEFAULT_TOKENS}" \
+            DEFAULT_CONTEXT="${DEFAULT_CONTEXT}"
+
+      - name: Record checksum
+        run: sha256sum dist/floppy-llm.exe | tee dist/floppy-llm.exe.sha256
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: floppy-llm-windows
+          path: |
+            dist/floppy-llm.exe
+            dist/floppy-llm.exe.sha256
+
+      - name: Publish tagged release
+        if: startsWith(github.ref, 'refs/tags/v')
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh release create "$GITHUB_REF_NAME" \
+            dist/floppy-llm.exe \
+            dist/floppy-llm.exe.sha256 \
+            --title "FloppyLLM $GITHUB_REF_NAME" \
+            --generate-notes
+
+  smoke-test:
+    needs: build
+    runs-on: windows-latest
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: floppy-llm-windows
+          path: dist
+
+      - name: Run Windows executable
+        shell: pwsh
+        run: |
+          .\dist\floppy-llm.exe --version
+          $text = .\dist\floppy-llm.exe --prompt "Once upon a time" --temperature 0 --tokens 12
+          if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($text)) {
+            throw "generation smoke test failed"
+          }
+          Write-Output $text
+

.gitignore

@@ -0,0 +1,5 @@
+dist/
+model/*.bin
+*.pyc
+__pycache__/
+

BUILD_RESULT.md

@@ -0,0 +1,11 @@
+# Verified local build
+
+- Windows artifact: `floppy-llm.exe`
+- Format: PE32 console executable, Intel 80386
+- Model: tiny1m, 896,256 parameters, Q8_0 group 32
+- Size: 1,148,267 bytes
+- Floppy capacity: 1,474,560 bytes
+- Remaining: 326,293 bytes
+- SHA-256 of the locally verified build: `d9401ff6f14fb92e0691d3f7526dbd86fd226c9718e7a53075445d15aace72e4`
+
+GitHub Actions reproduces the model from pinned upstream hashes, builds the executable, enforces the size ceiling, and runs it on a Windows runner.

LICENSE

@@ -0,0 +1,23 @@
+MIT License
+
+Copyright (c) 2026 FloppyLLM contributors
+Copyright (c) 2023 Andrej
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

Makefile

@@ -0,0 +1,54 @@
+PYTHON ?= python3
+CC ?= cc
+WINDOWS_CC ?= i686-w64-mingw32-clang
+DEFAULT_TOKENS ?= 128
+DEFAULT_CONTEXT ?= 0
+FLOPPY_BYTES ?= 1474560
+
+CFLAGS = -Os -ffunction-sections -fdata-sections -Wall -Wextra
+DEFINES = -DDEFAULT_MAX_NEW_TOKENS=$(DEFAULT_TOKENS) -DDEFAULT_CONTEXT_OVERRIDE=$(DEFAULT_CONTEXT)
+WINDOWS_LDFLAGS = -Wl,--gc-sections -Wl,-s -Wl,--subsystem,console:4.0 \
+	-Wl,--major-os-version,4 -Wl,--minor-os-version,0 \
+	-Wl,--disable-dynamicbase -Wl,--disable-nxcompat
+
+.PHONY: all model native windows check clean
+
+all: native
+
+model: model/tiny1m-q8.bin model/tokenizer.bin
+
+model/model-f32.bin:
+	mkdir -p model
+	curl -L --fail -o $@ https://huggingface.co/shibatch/tiny1m/resolve/main/model.bin
+	printf '%s  %s\n' 80ba15bd7feed5f78c13c33bac83992617ff030661f16abbfed8fba0a6e335c1 $@ | sha256sum -c -
+
+model/tokenizer.bin:
+	mkdir -p model
+	curl -L --fail -o $@ https://huggingface.co/shibatch/tiny1m/resolve/main/tokenizer.bin
+	printf '%s  %s\n' 037cb335abb25d1fa9e8ecae30ed2a3a8ace9302862ebcdc05d51a6bbb10c312 $@ | sha256sum -c -
+
+model/tiny1m-q8.bin: model/model-f32.bin tools/legacy_to_q8.py
+	$(PYTHON) tools/legacy_to_q8.py $< $@ 32
+	printf '%s  %s\n' 43c16d0d0702c1f628f841cad10c01bfde4a3a94d7784221a2f6208853ce12c2 $@ | sha256sum -c -
+
+native: model
+	mkdir -p dist
+	$(CC) $(CFLAGS) $(DEFINES) -o dist/floppy-llm-runner src/floppy_llm.c -lm
+	$(PYTHON) tools/pack_overlay.py dist/floppy-llm-runner model/tiny1m-q8.bin model/tokenizer.bin dist/floppy-llm
+	chmod +x dist/floppy-llm
+	$(PYTHON) tools/check_size.py dist/floppy-llm $(FLOPPY_BYTES)
+
+windows: model
+	mkdir -p dist
+	$(WINDOWS_CC) $(CFLAGS) $(DEFINES) $(WINDOWS_LDFLAGS) \
+		-o dist/floppy-llm-runner.exe src/floppy_llm.c -lm
+	$(PYTHON) tools/pack_overlay.py dist/floppy-llm-runner.exe model/tiny1m-q8.bin model/tokenizer.bin dist/floppy-llm.exe
+	$(PYTHON) tools/check_size.py dist/floppy-llm.exe $(FLOPPY_BYTES)
+
+check: native
+	./dist/floppy-llm --version
+	./dist/floppy-llm --prompt "Once upon a time, there was a little robot named Pip." --temperature 0 --tokens 20
+
+clean:
+	$(RM) -r dist
+

README.md

@@ -1 +1,110 @@
-# floppy-llm
+# FloppyLLM
+
+A real 896K-parameter transformer, tokenizer, and native Windows CLI packed
+into one executable that fits on a 1.44 MB floppy disk.
+
+The release file is `floppy-llm.exe`. It does not need Python, a model file,
+or an installer beside it.
+
+## Run it
+
+On Windows:
+
+```bat
+floppy-llm.exe
+```
+
+That starts a small interactive story-completion prompt. For one-shot use:
+
+```bat
+floppy-llm.exe -i "Once upon a time, there was a tiny computer" -n 80
+```
+
+Useful options:
+
+```text
+-i, --prompt TEXT       complete TEXT and exit
+-n, --tokens N          maximum new tokens; 0 fills the context
+-c, --context N         runtime context allocation; trained default is 256
+-t, --temperature N     0 is deterministic; default is 0.8
+-p, --top-p N           default is 0.9
+-s, --seed N            sampling seed
+    --stats              print speed
+```
+
+`--context` controls the runtime token/cache limit and does not require a
+rebuild. A value beyond the trained 256-token context is allowed for
+experimentation, but output quality may get worse.
+
+## What fits
+
+The checked Windows build is 1,148,267 bytes:
+
+| Part | Bytes |
+| --- | ---: |
+| Q8/group-32 model | 1,055,488 |
+| tokenizer | 6,227 |
+| 32-bit Windows runner | 86,528 |
+| overlay footer | 24 |
+| **total** | **1,148,267** |
+| 1.44 MB formatted floppy capacity | 1,474,560 |
+| **free** | **326,293** |
+
+The executable reads the model and tokenizer appended to itself. At runtime it
+needs several megabytes of RAM for model data, activations, and the KV cache;
+only the disk footprint is floppy-sized.
+
+## Why this model
+
+The build uses [`shibatch/tiny1m`](https://huggingface.co/shibatch/tiny1m),
+trained on TinyStories. It has 896,256 parameters and is substantially more
+capable than the commonly demonstrated 260K-parameter `stories260K` model,
+while its llama2.c-compatible tokenizer and architecture let the whole
+Q8/group-32 system remain comfortably below 1.44 MB. In a deterministic
+sanity test, the Q8/group-32 build matched the float checkpoint's continuation.
+
+This is still a deliberately terrible, extremely narrow model. Expect short,
+simple children's-story prose—not factual answers or useful coding help.
+
+## Build
+
+The repository intentionally does not commit generated model binaries.
+`make model` downloads the upstream MIT model, verifies hashes, and produces
+the exact Q8 checkpoint.
+
+Native Linux reference build:
+
+```sh
+python3 -m pip install numpy
+make check
+```
+
+32-bit Windows build from Linux:
+
+```sh
+python3 -m pip install numpy
+make windows WINDOWS_CC=/path/to/i686-w64-mingw32-clang
+```
+
+Change compiled defaults if desired:
+
+```sh
+make windows DEFAULT_TOKENS=256 DEFAULT_CONTEXT=512
+```
+
+These only change defaults; users can always pass `-n` and `-c` at runtime.
+The GitHub Actions workflow exposes the same two values through
+**Run workflow** and refuses to upload anything over 1,474,560 bytes.
+
+## Old Windows status
+
+The release is a 32-bit i386 PE console program built against the old
+Windows-provided `msvcrt.dll`, with PE OS and subsystem versions set to 4.0.
+It is suitable for testing on modern 32/64-bit Windows first. Windows 95/98
+compatibility is an explicit experimental target: the imports are old Win32
+APIs, but this artifact has not yet been executed on physical Win9x hardware.
+CPU speed and RAM, rather than disk space, will be the practical constraint.
+
+## License
+
+MIT. See `LICENSE` and `THIRD_PARTY_NOTICES.md`.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,15 @@
+# Third-party notices
+
+The inference code is derived from Andrej Karpathy's
+[`llama2.c`](https://github.com/karpathy/llama2.c), under the MIT License.
+The original license is preserved in `licenses/llama2.c-MIT.txt`.
+
+The bundled model is [`shibatch/tiny1m`](https://huggingface.co/shibatch/tiny1m).
+Its model card declares the model and assets MIT licensed. It was trained on
+the TinyStories dataset.
+
+Release builds use the official
+[`llvm-mingw`](https://github.com/mstorsjo/llvm-mingw) toolchain to produce a
+32-bit PE executable linked against the Windows-provided `msvcrt.dll` and
+`kernel32.dll`.
+

licenses/llama2.c-MIT.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Andrej
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

model/README.md

@@ -0,0 +1,27 @@
+# Bundled model
+
+The build downloads [`shibatch/tiny1m`](https://huggingface.co/shibatch/tiny1m),
+an MIT-licensed Llama 2 model trained on TinyStories. Its architecture is:
+
+- 896,256 parameters
+- 512-token vocabulary
+- 128 hidden width, 4 layers, 2 attention heads
+- 352-wide feed-forward layers
+- 256-token trained context
+
+`tools/legacy_to_q8.py` converts the upstream 3,805,724-byte float checkpoint
+to llama2.c Q8_0 with groups of 32. The result is 1,055,488 bytes. The
+tokenizer is 6,227 bytes.
+
+Reproducibility hashes:
+
+| File | SHA-256 |
+| --- | --- |
+| Upstream `model.bin` | `80ba15bd7feed5f78c13c33bac83992617ff030661f16abbfed8fba0a6e335c1` |
+| Upstream `tokenizer.bin` | `037cb335abb25d1fa9e8ecae30ed2a3a8ace9302862ebcdc05d51a6bbb10c312` |
+| Converted `tiny1m-q8.bin` | `43c16d0d0702c1f628f841cad10c01bfde4a3a94d7784221a2f6208853ce12c2` |
+
+The model is a tiny story-completion model, not an instruction-following
+assistant. Its value here is that it is a real transformer that fits, runs,
+and sometimes produces recognizable simple prose.
+