docs: getting started, overview, command line, library API

README.md

@@ -10,6 +10,10 @@ Computer Vision ML inference in C++
 
 Based on [ggml](https://github.com/ggml-org/ggml) similar to the [llama.cpp](https://github.com/ggml-org/llama.cpp) project.
 
+New here? [**Getting started**](docs/getting-started.md) walks through a first run in five minutes.
+
+**Docs:** [Getting started](docs/getting-started.md) · [Overview](docs/overview.md) · [Command line](docs/using-the-cli.md) · [Library API](docs/using-the-library.md) · [Implementing a model](docs/model-implementation-guide.md)
+
 ### Features
 
 | Model                                    | Task                     | Backends    |

docs/getting-started.md

@@ -0,0 +1,101 @@
+# Getting Started
+
+In this tutorial you will cut an object out of a photo using vision.cpp. It takes about five
+minutes and needs nothing but the release package, one model file and one image — no build, no
+Python, no conversion.
+
+At the end you will have this:
+
+| | |
+| :--- | :--- |
+| `mask.png` | a black-and-white mask of the object |
+| `object.png` | the original photo with the background dimmed away |
+
+## Step 1 — Get the executable
+
+Download a [release package](https://github.com/Acly/vision.cpp/releases) and extract it. You
+will find `vision-cli` in the `bin` folder.
+
+Check that it runs:
+
+```sh
+vision-cli --help
+```
+
+You should see a list of commands: `sam`, `birefnet`, `depthany`, `migan`, `esrgan`.
+
+> If you would rather build from source, follow [Building](../README.md#building) first, then
+> come back here. `vision-cli` ends up in `build/bin`.
+
+## Step 2 — Get a model and an image
+
+The executable contains the network structure, but not the weights. Download them:
+
+```sh
+curl -L -O https://huggingface.co/Acly/BiRefNet-GGUF/resolve/main/BiRefNet-lite-F16.gguf
+```
+
+This is BiRefNet, a model that separates a subject from its background. The file is a
+[GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) — the weights and nothing else.
+
+For the input, use any photo with a clear subject. If you cloned the repository, there is one
+at `docs/media/input.jpg`. Put it next to the model file and call it `input.jpg`.
+
+## Step 3 — Run it
+
+```sh
+vision-cli birefnet -m BiRefNet-lite-F16.gguf -i input.jpg -o mask.png --composite object.png
+```
+
+The output tells you what it is doing:
+
+```
+Initializing backend... done (1.1 ms)
+- device: CPU - Intel(R) Core(TM) i3-14100
+Loading model weights from 'BiRefNet-lite-F16.gguf'... done (151.3 ms)
+- float type: f16
+- tensor layout: cwhn
+- model image size: 1024
+- inference image size: 1024x1024
+- flash attention: off
+Running inference... complete (5372.6 ms)
+-> mask saved to mask.png
+-> image composited and saved to object.png
+```
+
+Inference takes a few seconds on a desktop CPU. Loading the weights takes a fraction of a
+second — that number is the point of the project, and it is the same on any machine.
+
+## Step 4 — Look at the result
+
+Open `object.png`. The subject is untouched and the background has faded away.
+
+`mask.png` is what the model actually produced: white where the subject is, black elsewhere.
+Everything in `object.png` was computed from it.
+
+That is the whole loop. An executable that already knows the network, a `.gguf` that carries
+the weights, an image in, a result out.
+
+## Try one more
+
+The same executable runs the other built-in models. Only the command and the weights change:
+
+```sh
+curl -L -O https://huggingface.co/Acly/Real-ESRGAN-GGUF/resolve/main/RealESRGAN-x4plus_anime-6B-F16.gguf
+
+vision-cli esrgan -m RealESRGAN-x4plus_anime-6B-F16.gguf -i input.jpg -o upscaled.png
+```
+
+This one upscales the image four times. It works on tiles and takes noticeably longer — you will
+see it count them off.
+
+## Where to go next
+
+- [Overview](overview.md) — what the library is and why weights and structure are separate.
+- [Using the command line](using-the-cli.md) — every option, every built-in model.
+- [Using the library](using-the-library.md) — the same models from your own code.
+- [README](../README.md#features) — the other built-in models, and what each one does.
+- [Model implementation guide](model-implementation-guide.md) — when the model you want is not
+  in the list, and you want to add it.
+- [MMDetection detectors](mmdet-detectors.md) — running detectors whose structure is generated
+  rather than hand-written.

docs/overview.md

@@ -0,0 +1,111 @@
+# Overview
+
+vision.cpp is a C++ library for running computer-vision neural networks. It loads weights
+from a GGUF file, builds a compute graph with [ggml](https://github.com/ggml-org/ggml), and
+executes it on CPU or GPU. The result is a single native binary with no Python, no framework
+runtime, and no interchange-format interpreter.
+
+It is the same idea as [llama.cpp](https://github.com/ggml-org/llama.cpp), applied to vision
+models instead of language models.
+
+## The idea: a model is code, not a file
+
+Most inference stacks treat a model as data. You export a graph to ONNX or TorchScript, and
+a general-purpose runtime loads that graph, matches its operators against a kernel library, and
+interprets it. The runtime has to support every operator anyone might export, so it is large,
+and it has to plan the graph at startup, so loading takes time.
+
+vision.cpp splits the model in two:
+
+| Part | Form | Where it lives |
+| :--- | :--- | :--- |
+| Structure | C++ that builds a ggml graph | compiled into your binary |
+| Weights | GGUF tensors | a `.gguf` file loaded at run time |
+
+Nothing interprets a graph description at run time, because there is no graph description — the
+graph is the code you compiled. That is what makes the deployment small and start-up fast, and
+it is the trade-off at the centre of the project: adding a model that isn't supported yet
+means writing or generating code, not exporting a file.
+
+Weights stay external, so swapping checkpoints, changing precision, or quantising does not
+require rebuilding.
+
+## What you get
+
+- Self-contained. The only dependencies are ggml, `stb` for image I/O, and optionally
+  `fmt`. There is no Python in the runtime path.
+- CPU and GPU. CPU works everywhere; Vulkan covers NVIDIA, AMD and Intel from one build.
+- Small and quick to start. Deployment size and model-load time are explicit goals of the
+  project — see the [Performance](../README.md#performance) section for the current numbers.
+- Modular. The same primitives the built-in models are made of are public, so you can
+  assemble your own.
+
+## How it fits together
+
+The library is layered. Each layer is usable on its own; you can stop at whichever one matches
+how much control you need.
+
+| Layer | Header | What it gives you |
+| :--- | :--- | :--- |
+| Model APIs | `visp/vision.h` | Ready-made models — load, run, get a result. |
+| Image I/O | `visp/image.h` | Load, save, resize, tile, convert. |
+| Neural network layers | `visp/nn.h` | `conv_2d`, `group_norm`, attention, and other building blocks. |
+| Graph and backends | `visp/ml.h` | GGUF loading, weight transfer, graph construction, execution. |
+| Detection post-processing | `visp/postproc.h` | Anchors, decoding, NMS, RoIAlign, masks. |
+| Tracking | `visp/tracker.h` | ByteTrack association across frames. |
+
+Two front-ends are built on top:
+
+- `vision-cli` — a command-line tool for the built-in models
+  (`vision-cli sam -m MobileSAM-F16.gguf -i image.jpg -p 100 200 -o mask.png`).
+- Python bindings — `bindings/python`, for scripting and comparison against reference
+  implementations.
+
+## Running a model
+
+Most of the time there is nothing to add. The models in the
+[README](../README.md#features) are already implemented, so running one means downloading its
+weights and pointing at them:
+
+```sh
+vision-cli birefnet -m BiRefNet-lite-F16.gguf -i photo.jpg -o mask.png
+```
+
+Or from your own program, in three calls — pick a device, load the weights, compute. See
+[using the command line](using-the-cli.md) and [using the library](using-the-library.md).
+
+If you have your own checkpoint for one of those architectures, convert it with
+`scripts/convert.py`. The structure is already in the library; only the weights change.
+
+## Adding a model
+
+When the architecture is not implemented yet, it has to be written. The
+[model implementation guide](model-implementation-guide.md) walks through it: describe the
+network with the `nn.h` primitives, and provide a conversion function that turns the original
+checkpoint into GGUF. Every built-in model was added this way, and it gives the most control
+over layout and precision.
+
+The result is what the library loads: a `<Arch>_forward` function that builds a graph, plus a
+GGUF file of weights.
+
+For model families with hundreds of variants, hand-writing each one is not realistic and the
+C++ can be generated from a traced PyTorch module instead. The
+[MMDetection guide](mmdet-detectors.md) covers that case — the interface generated code must
+satisfy, and how to handle what tracing cannot capture.
+
+## Scope
+
+vision.cpp is an inference library. There is no training, no autograd, and no optimizer.
+
+It is also not a general model runtime: it does not aim to execute arbitrary exported graphs.
+Supported models are the ones that have been implemented or generated, which is why the model
+list in the [README](../README.md#features) is finite and why growing it is a code change.
+
+## Next
+
+- [Getting started](getting-started.md) — run a model end to end in five minutes.
+- [Using the command line](using-the-cli.md) — every built-in model, no code.
+- [Using the library](using-the-library.md) — the same models from C++ or Python.
+- [README](../README.md) — install, build, supported models, performance.
+- [Model implementation guide](model-implementation-guide.md) — write a model by hand.
+- [MMDetection detectors](mmdet-detectors.md) — run detectors from a compiled backbone.

docs/using-the-cli.md

@@ -0,0 +1,160 @@
+# Using the command line
+
+`vision-cli` runs every built-in model without writing any code. All you need is the executable
+and a `.gguf` weights file.
+
+If you have not run anything yet, start with [Getting started](getting-started.md).
+
+## The shape of a command
+
+```sh
+vision-cli <command> -m <weights.gguf> -i <input> -o <output>
+```
+
+The command selects the model, `-m` says which weights to load, `-i` and `-o` are files.
+
+| Command | Task | Input | Output |
+| :--- | :--- | :--- | :--- |
+| `birefnet` | Background removal | image | mask |
+| `sam` | Segment one object you point at | image + prompt | mask |
+| `depthany` | Depth estimation | image | depth map |
+| `migan` | Inpainting — fill a region | image + mask | image |
+| `esrgan` | Upscaling | image | larger image |
+
+## Options
+
+`-m, --model <file>`
+:   The `.gguf` weights. Required.
+
+`-i, --input <image> [<image> ...]`
+:   Input image. `migan` takes two — the image and the mask.
+
+`-o, --output <file>`
+:   Output file. Defaults to `output.png`.
+
+`-p, --prompt <x> [<y> ...]`
+:   Prompt for models that take one. `sam` accepts a point (`x y`) or a box
+    (`x1 y1 x2 y2`) in pixels, origin top-left.
+
+`-b, --backend <cpu|gpu>`
+:   Which device to run on. Defaults to automatic — GPU if the build has Vulkan and a device is
+    available, CPU otherwise.
+
+`--composite <file>`
+:   Also write the input image combined with the resulting mask, instead of the mask alone.
+
+`--tile <size>`
+:   Split large inputs into tiles of this size. Used by `esrgan` to keep memory bounded.
+
+`-h, --help`
+:   Print the command list and exit.
+
+## Getting weights
+
+Each model has its own GGUF repository. Download the file and pass it with `-m`.
+
+| Model | Weights |
+| :--- | :--- |
+| MobileSAM | [Acly/MobileSAM-GGUF](https://huggingface.co/Acly/MobileSAM-GGUF) |
+| BiRefNet | [Acly/BiRefNet-GGUF](https://huggingface.co/Acly/BiRefNet-GGUF) |
+| Depth-Anything V2 | [Acly/Depth-Anything-V2-GGUF](https://huggingface.co/Acly/Depth-Anything-V2-GGUF) |
+| MI-GAN | [Acly/MIGAN-GGUF](https://huggingface.co/Acly/MIGAN-GGUF) |
+| Real-ESRGAN | [Acly/Real-ESRGAN-GGUF](https://huggingface.co/Acly/Real-ESRGAN-GGUF) |
+
+Several variants are usually available per model — different sizes or resolutions. The
+executable reads which one it got from the file's metadata, so no extra flag is needed.
+
+## Remove a background
+
+```sh
+vision-cli birefnet -m BiRefNet-lite-F16.gguf -i photo.jpg -o mask.png --composite cutout.png
+```
+
+`mask.png` is white where the subject is. `cutout.png` is the photo with the background removed.
+
+## Segment one object
+
+Unlike background removal, this needs to be told which object. Give a point inside it:
+
+```sh
+vision-cli sam -m MobileSAM-F16.gguf -i photo.jpg -p 300 200 -o mask.png
+```
+
+or a box around it:
+
+```sh
+vision-cli sam -m MobileSAM-F16.gguf -i photo.jpg -p 420 120 650 430 -o mask.png
+```
+
+A box is usually more reliable when the object touches others.
+
+## Estimate depth
+
+```sh
+vision-cli depthany -m Depth-Anything-V2-Small-F16.gguf -i photo.jpg -o depth.png
+```
+
+The output is a single-channel image — bright is near, dark is far. Values are relative to the
+image, not metric distances.
+
+## Fill a region
+
+Inpainting takes two inputs: the image, and a mask marking what to replace.
+
+```sh
+vision-cli migan -m MIGAN-512-places2-F16.gguf -i photo.jpg mask.png -o filled.png
+```
+
+White in the mask is the region to fill. You can produce that mask with `birefnet` or `sam`,
+which makes removing an object a two-step operation.
+
+## Upscale
+
+```sh
+vision-cli esrgan -m RealESRGAN-x4plus_anime-6B-F16.gguf -i photo.jpg -o large.png
+```
+
+The scale factor comes from the weights — the model above is 4×. Large inputs are processed in
+tiles; you will see them counted off, and the whole run takes considerably longer than the other
+models.
+
+## Choosing a device
+
+```sh
+vision-cli birefnet -m BiRefNet-lite-F16.gguf -i photo.jpg -o mask.png -b gpu
+```
+
+GPU requires a build with Vulkan enabled — see [Building](../README.md#building). Without it,
+`-b gpu` has nothing to select and the run stays on CPU. The first two lines of output always
+name the device that was actually used.
+
+## Using your own weights
+
+If you have a checkpoint for an architecture the library already implements, convert it to GGUF
+rather than looking for a pre-made file.
+
+```sh
+uv run scripts/convert.py <arch> MyModel.pth
+```
+
+`<arch>` is one of `sam`, `sam3`, `birefnet`, `depth-anything`, `migan`, `esrgan`.
+The result lands in `models/`.
+
+| Option | Description |
+| :--- | :--- |
+| `-o, --output` | Output directory or file. Default `models`. |
+| `-q, --quantize f16` | Store float weights as f16 — roughly half the file size. |
+| `-l, --layout whcn\|cwhn` | Tensor layout for 2D operations. Leave unset unless you know you need the other one. |
+| `--model-name` | Name recorded in the file's metadata. |
+| `-v, --verbose` | Print every tensor as it is converted. |
+
+Conversion also rearranges and precomputes tensors, so it is not a pure format change — this is
+why a checkpoint cannot be loaded directly.
+
+This route only covers architectures that exist in the library. For anything else, see the
+[model implementation guide](model-implementation-guide.md).
+
+## Next
+
+- [Using the library](using-the-library.md) — the same models from your own C++ or Python code.
+- [Overview](overview.md) — why weights and structure are separate files.