0.5.0 - EXPERIMENTAL

AGENTS.md

@@ -11,7 +11,7 @@ cashet is a content-addressable compute cache with git semantics. Python functio
 - Install with HTTP server: `uv sync --extra server`
 - Install with all extras: `uv sync --all-extras`
 - Run tests: `uv run pytest tests/ -v`
-- Redis tests auto-detect: skipped if Redis isn't reachable at `localhost:6379`. Set `CASHET_REDIS=1` to skip auto-detection and force-enable.
+- Redis tests target `redis://localhost:6379/15` by default and FLUSH that database between tests. Override with `CASHET_TEST_REDIS_URL` (point it at a dedicated instance or database, never one holding data you care about). Skipped automatically if the target isn't reachable; set `CASHET_REDIS=1` to skip auto-detection and force-enable.
 - Start Redis for full suite: `docker run -d --name cashet-redis -p 6379:6379 redis:7-alpine`
 - Lint: `uv run ruff check src/ tests/`
 - Type check: `uv run pyright src/`
@@ -39,13 +39,15 @@ Protocol-based dependency injection via three pluggable protocols in `src/cashet
 - **Store** — metadata + blob storage. Default: `SQLiteStore` in `store.py`. Also: `RedisStore` and `AsyncRedisStore` in `redis_store.py`
 - **AsyncStore** — async protocol for IO-bound backends (Redis, S3, HTTP). Defined in `protocols.py`
 - **Executor** — runs functions. Default: `LocalExecutor` in `executor.py` (sync). Async variant: `AsyncLocalExecutor` in `async_executor.py`
-- **Serializer** — serialize/deserialize results. Default: `PickleSerializer` in `hashing.py`
+- **Serializer** — serialize/deserialize results. Default: `PickleSerializer` in `serializers.py` (re-exported from `hashing.py` for compatibility)
+
+Internal module layout: `store.py` holds only the public SQLite store classes; the engine lives in `_sqlite_core.py`, schema and migrations in `_sqlite_schema.py`, cross-process lock plumbing in `_locks.py`. `redis_store.py` holds the Redis store classes; the key scheme, commit codec, and index pipeline commands live in `_redis_codec.py`. Hash-prefix validation shared by both stores is in `_ids.py`.
 
 Data flow (sync): `Client.submit()` → `build_task_def()` hashes function source + args → `LocalExecutor.submit()` checks cache → runs if needed → `build_commit()` creates commit with parent lineage → blobs stored via `Store.put_blob()` with zlib compression (256B threshold).
 
 Async data flow: `AsyncClient.submit()` → `build_task_def()` → `AsyncLocalExecutor.submit()` checks cache via async store → runs function in thread via `asyncio.to_thread()` → stores result via async `AsyncStore.put_blob()`. Heartbeat and locking are asyncio-native.
 
-HTTP server: `client.serve(host, port, require_token=None)` exposes endpoints over HTTP. When `require_token` is set, all requests must include `Authorization: Bearer <token>`. Async server (`AsyncClient.serve()`) uses `create_async_app()` with native async handlers. Sync server (`Client.serve()`) uses `create_app()` which wraps sync Client calls in `asyncio.to_thread`.
+HTTP server: `client.serve(host, port, require_token=None)` exposes endpoints over HTTP. When `require_token` is set, all requests must include `Authorization: Bearer <token>`. One handler set serves both clients through an ops adapter in `server.py`: `create_async_app()` (from `AsyncClient.serve()`) calls `AsyncClient` natively, `create_app()` (from `Client.serve()`) wraps sync Client calls in `asyncio.to_thread`.
 
 ## Key design decisions
 
@@ -57,7 +59,7 @@ HTTP server: `client.serve(host, port, require_token=None)` exposes endpoints ov
 - `AsyncResultRef` is the async counterpart for `AsyncClient`.
 - IO-bound layers (Store, HTTP) are async. CPU-bound execution stays on threads via `asyncio.to_thread()`.
 - Redis blob data keys: `cashet:blob:data:{hash}`. Ref counters: `cashet:blob:ref:{hash}`. Per-fingerprint locks: `cashet:lock:{fingerprint}`.
-- Redis `delete_commit` uses a Lua script (`_DECR_DELETE_SCRIPT`) for atomic DECR + conditional DELETE of blob keys when ref count reaches zero, avoiding a TOCTOU race.
+- Redis `delete_commit` uses a Lua script (`DECR_DELETE_SCRIPT` in `_redis_codec.py`) for atomic DECR + conditional DELETE of blob keys when ref count reaches zero, avoiding a TOCTOU race.
 - Sync `Client` and async `AsyncClient` share common helpers extracted to `_client_base.py` (store dir resolution, task config, diff).
 
 ## Code style

CHANGELOG.md

@@ -1,5 +1,90 @@
 # Changelog
 
+## 0.5.0 - 12.7.2026.
+
+Experimental correctness and performance release. Cache hits are lock-free
+and roughly 10x faster; several data-integrity and cross-store consistency
+bugs are fixed. Read the Notes before upgrading shared stores.
+
+### Fixed
+- Blobs are written atomically (temp file, then rename). A crash mid-write
+  could leave a truncated file at the content-addressed path that the
+  deduplication check then trusted forever, permanently poisoning that hash.
+  `get_blob` also deletes a corrupt blob file so the next `put_blob` can store
+  the content again.
+- Redis fingerprint lookups return the newest completed commit, matching
+  SQLite. The per-fingerprint index was ordered by expiry with arbitrary tie
+  breaking, so after a force re-run later calls could keep serving the older
+  result. Legacy index entries are rescored in place on first lookup.
+- Garbage collection removes TTL-expired commits on both stores instead of
+  keeping them until they age past the access-time cutoff. Running and pending
+  commits are exempt from expiry eviction: their TTL clock starts at claim
+  time, so a task outliving its TTL must not be deleted mid-execution.
+- A store error during a heartbeat renewal no longer kills the lease loop
+  (which let another worker reclaim and double-run a live task) and can no
+  longer surface as an exception that destroys an already-computed result.
+  Failed renewals retry on a quarter of the normal interval, so a single
+  transient error cannot leave the claim stale before the next attempt.
+- Set and frozenset arguments are ordered by their stable serialized form
+  when hashing. Ordering by raw repr embedded memory addresses, so the same
+  logical set could hash differently in another process.
+- Per-fingerprint lock state no longer grows without bound: SQLite fingerprint
+  locks are striped across 256 paths, and Redis locks are created per
+  acquisition instead of being cached per fingerprint.
+
+### Performance
+Measured with `benchmarks/bench_hot_path.py` on one machine (medians,
+0.4.5 vs 0.5.0):
+- sync cache hit: 4.0 ms to 0.35 ms
+- async cache hit: 3.9 ms to 0.21 ms
+- hashing (`build_task_def`): 256 us to 10 us
+- cache miss (run + store): 8.2 ms to 2.7 ms
+
+- Cache hits take no fingerprint lock, never rewrite the commit, and skip the
+  redundant parent query on the claim path. SQLite hits inside the access-bump
+  window perform no writes at all; Redis hits still update the shared access
+  index with one ZADD, which single-instance Redis absorbs cheaply.
+- `last_accessed_at` is bumped at most once per hour per commit. Eviction
+  cutoffs are measured in days, so LRU eviction behavior is unchanged.
+- SQLite connections use `synchronous=NORMAL` under WAL: a power loss can
+  drop the newest commits, which a compute cache recomputes; the database
+  itself cannot corrupt.
+- Function source resolution and AST canonicalization are memoized per
+  function object. Globals, defaults, and closures are still hashed live, so
+  mutated module state keeps invalidating as before.
+
+### Changed
+- A cache hit no longer rewrites the stored commit's status to `cached`;
+  stored commits keep `completed` and `cashet log` shows them as such.
+- The HTTP server returns 422 with the final exception line (no traceback,
+  no server file paths) when a submitted task raises. 500 is reserved for
+  actual server errors.
+- CLI commands that operate on an existing store exit non-zero when the store
+  directory does not exist instead of silently creating an empty one;
+  `import` and `serve` still create it. A group-level `--store-dir` option
+  overrides `$CASHET_DIR` and the `./.cashet` default.
+- Redis tests target `redis://localhost:6379/15` by default and honor
+  `CASHET_TEST_REDIS_URL`. The suite flushes the target database, which
+  previously defaulted to a shared database 0.
+- Internal modules reorganized; public import paths are unchanged.
+  Serializers moved from `cashet.hashing` to `cashet.serializers` (the old
+  imports still work via re-exports). `store.py` keeps the public SQLite
+  classes and delegates to `_sqlite_core`, `_sqlite_schema`, and `_locks`;
+  `redis_store.py` keeps the Redis classes and delegates its key scheme and
+  codec to `_redis_codec`. The HTTP server's duplicated sync and async
+  handlers were unified into one set behind an ops adapter.
+
+### Notes
+- Argument hashes change for sets containing custom objects or single-element
+  tuples; affected entries recompute on first access.
+- The Redis per-fingerprint index is rescored lazily. Entries written by
+  older versions with a TTL can still order incorrectly until rewritten, and
+  commits written before 0.5.0 are absent from the new expiry index, so they
+  only age out by access time.
+- Blob writes create short-lived `*.tmp` files inside `objects/`. A failed
+  write removes its temp file, storage stats never count them, and garbage
+  collection sweeps any left behind by a crash once they are an hour old.
+
 ## 0.4.5 - 21.6.2026.
 
 ### Fixed

README.md

@@ -234,9 +234,10 @@ cashet export backup.tar.gz        # export commits and blobs to an archive
 cashet import backup.tar.gz        # import from an archive
 cashet stats                       # storage statistics
 cashet serve --host 127.0.0.1 --port 8000
+cashet --store-dir /path/to/store log   # any command against another store
 ```
 
-`cashet show`, `cashet get`, and `cashet rm` exit with a non-zero status when the commit is not found, so they compose in scripts.
+`cashet show`, `cashet get`, and `cashet rm` exit with a non-zero status when the commit is not found, so they compose in scripts. Commands that operate on an existing store exit non-zero when the store directory does not exist instead of creating an empty one; `import` and `serve` create it. `--store-dir` overrides `$CASHET_DIR` and the `./.cashet` default.
 
 ## Python API
 
@@ -362,7 +363,7 @@ client.serve(host="127.0.0.1", port=8000, require_token="secret123")
 | GET | `/stats` | Storage statistics |
 | POST | `/gc` | Run garbage collection |
 
-When `require_token` is set, every request needs an `Authorization: Bearer <token>` header. Request bodies are size-limited (default 500 MB). Use `AsyncClient.serve()` for the native async server.
+When `require_token` is set, every request needs an `Authorization: Bearer <token>` header. Request bodies are size-limited (default 500 MB). A task that raises returns 422 with the exception line; 500 means the server itself failed. Use `AsyncClient.serve()` for the native async server.
 
 > **Security.** `/submit` does not execute client-supplied Python by default. The legacy `func_source`, `func_b64`, `args_b64`, and `kwargs_b64` payloads require both `allow_remote_code=True` and a non-empty token (`cashet serve --require-token secret123 --allow-remote-code`). That mode deserializes and runs arbitrary Python, so expose it only to trusted clients.
 
@@ -447,7 +448,7 @@ Small results (under 1 KB) are stored inline in `meta.db` to avoid inode overhea
 
 ### Concurrency
 
-cashet is safe across threads, processes, and machines that share one store. Concurrent submissions of the same uncached task are deduplicated: the function runs exactly once and all callers get the same result. Cross-process claims use file locks (SQLite) or per-fingerprint Redis locks, with a heartbeat lease so a crashed worker's claim is reclaimed (default 5 minutes, configurable via `LocalExecutor(running_ttl=...)`).
+cashet is safe across threads, processes, and machines that share one store. Concurrent submissions of the same uncached task are deduplicated: the function runs exactly once and all callers get the same result. Cross-process claims use file locks (SQLite) or per-fingerprint Redis locks, with a heartbeat lease so a crashed worker's claim is reclaimed (default 5 minutes, configurable via `LocalExecutor(running_ttl=...)`). Cache hits take no locks. On SQLite a steady-state hit performs no writes at all; on Redis a hit updates the shared access index with a single ZADD, which is what lets every worker's hits feed one LRU.
 
 ### Notebooks
 
@@ -462,6 +463,7 @@ cashet resolves function source through `inspect.getsource`, then `dill.source.g
 
 Upgrading across a hash-format change does not corrupt anything; old entries simply miss and recompute on first access. To start clean, run `cashet clear` or point at a fresh store directory.
 
+- **0.4.5 to 0.5.0:** Argument hashes change for sets containing custom objects or single-element tuples; those entries recompute on first access. Cache hits no longer rewrite commit status to `cached`. The Redis fingerprint index is rescored to created_at ordering lazily on lookup; pre-0.5.0 TTL entries may order incorrectly until rewritten and are collected by access age only. SQLite opens with `synchronous=NORMAL`. The HTTP server returns 422 instead of 500 for task failures. CLI read commands require an existing store directory.
 - **0.4.4 to 0.4.5:** Hashing fixes (slotted-object state, referenced global containers, `ast.unparse` canonicalization) change function and argument cache keys, so results cached by earlier versions recompute on first access. The Redis tag index key scheme changed and is not migrated; rewrite affected commits to rebuild tag indexes. `import_archive` now returns `ImportResult(imported, skipped)` instead of a bare count.
 - **0.3.x to 0.4.0:** Added per-entry TTL and tag-based invalidation. SQLite auto-migrates; old caches stay readable.
 - **0.3.0 to 0.3.1:** Redis blob keys renamed to `cashet:blob:data:{hash}` and stats backfilled once. Clear Redis caches before upgrading if you rely on long-lived reuse.

benchmarks/bench_hot_path.py

@@ -0,0 +1,86 @@
+from __future__ import annotations
+
+import asyncio
+import statistics
+import tempfile
+import time
+from collections.abc import Callable
+from pathlib import Path
+from typing import Any
+
+from cashet import Client
+from cashet.async_client import AsyncClient
+from cashet.hashing import build_task_def
+
+ROUNDS = 300
+
+
+def transform(data: list[int], scale: float = 2.0) -> list[float]:
+    return [x * scale for x in data]
+
+
+def _timed(fn: Callable[[], Any], rounds: int = ROUNDS) -> tuple[float, float]:
+    fn()
+    samples: list[float] = []
+    for _ in range(rounds):
+        start = time.perf_counter()
+        fn()
+        samples.append((time.perf_counter() - start) * 1e6)
+    return statistics.median(samples), min(samples)
+
+
+def _report(label: str, median_us: float, min_us: float) -> None:
+    print(f"{label:<28} median {median_us:9.1f} us   min {min_us:9.1f} us")
+
+
+def bench_sync(root: Path) -> None:
+    with Client(store_dir=root / "sync") as client:
+        client.submit(transform, [1, 2, 3]).load()
+        _report(
+            "hash (build_task_def)",
+            *_timed(lambda: build_task_def(transform, ([1, 2, 3],), {})),
+        )
+        _report("sync hit (submit)", *_timed(lambda: client.submit(transform, [1, 2, 3])))
+        _report(
+            "sync hit (submit + load)",
+            *_timed(lambda: client.submit(transform, [1, 2, 3]).load()),
+        )
+
+
+def bench_async(root: Path) -> None:
+    async def run() -> None:
+        client = AsyncClient(store_dir=root / "async")
+        ref = await client.submit(transform, [1, 2, 3])
+        await ref.load()
+
+        samples: list[float] = []
+        for _ in range(ROUNDS):
+            start = time.perf_counter()
+            await client.submit(transform, [1, 2, 3])
+            samples.append((time.perf_counter() - start) * 1e6)
+        _report("async hit (submit)", statistics.median(samples), min(samples))
+        await client.close()
+
+    asyncio.run(run())
+
+
+def bench_miss(root: Path) -> None:
+    with Client(store_dir=root / "miss") as client:
+        samples: list[float] = []
+        for i in range(50):
+            start = time.perf_counter()
+            client.submit(transform, [i])
+            samples.append((time.perf_counter() - start) * 1e6)
+        _report("sync miss (run + store)", statistics.median(samples), min(samples))
+
+
+def main() -> None:
+    root = Path(tempfile.mkdtemp(prefix="cashet-bench-"))
+    print(f"store: {root}  rounds: {ROUNDS}")
+    bench_sync(root)
+    bench_async(root)
+    bench_miss(root)
+
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "cashet"
-version = "0.4.5"
+version = "0.5.0"
 description = "A Python memoization cache with Redis, async support, DAG pipelines, and an HTTP server"
 readme = "README.md"
 license = "MIT"

src/cashet/__init__.py

@@ -6,15 +6,15 @@
 from cashet.client import Client
 from cashet.dag import AsyncResultRef, ResultRef, TaskRef
 from cashet.executor import LocalExecutor
-from cashet.hashing import (
-    ClosureWarning,
+from cashet.hashing import ClosureWarning
+from cashet.models import Commit, ObjectRef, TaskDef, TaskError, TaskStatus
+from cashet.protocols import AsyncStore, Executor, Store
+from cashet.serializers import (
     JsonSerializer,
     PickleSerializer,
     SafePickleSerializer,
     Serializer,
 )
-from cashet.models import Commit, ObjectRef, TaskDef, TaskError, TaskStatus
-from cashet.protocols import AsyncStore, Executor, Store
 from cashet.store import AsyncSQLiteStore, SQLiteStore
 
 try:

src/cashet/_batch.py

@@ -7,9 +7,10 @@
 
 from cashet._client_base import resolve_task_config
 from cashet.dag import AsyncResultRef, TaskRef
-from cashet.hashing import Serializer, build_task_def
+from cashet.hashing import build_task_def
 from cashet.models import TaskError
 from cashet.protocols import AsyncStore
+from cashet.serializers import Serializer
 
 BatchKey = int | str
 NormalizedTask = tuple[

src/cashet/_client_base.py

@@ -6,9 +6,9 @@
 from pathlib import Path
 from typing import Any
 
-from cashet.hashing import Serializer
 from cashet.models import Commit, TaskStatus
 from cashet.protocols import AsyncStore
+from cashet.serializers import Serializer
 
 _DEFAULT_STORE_DIR = ".cashet"
 _CASHET_DIR_ENV = "CASHET_DIR"

src/cashet/_ids.py

@@ -0,0 +1,7 @@
+from __future__ import annotations
+
+
+def normalize_hash_prefix(hash: str) -> str | None:
+    if 0 < len(hash) <= 64 and all(c in "0123456789abcdefABCDEF" for c in hash):
+        return hash.lower()
+    return None

src/cashet/_locks.py

@@ -0,0 +1,49 @@
+from __future__ import annotations
+
+import asyncio
+import threading
+from dataclasses import dataclass
+from typing import Any
+
+
+@dataclass
+class SQLiteLockState:
+    thread_lock: threading.Lock
+    file_lock: Any
+
+
+SQLITE_LOCKS: dict[str, SQLiteLockState] = {}
+SQLITE_LOCKS_GUARD = threading.Lock()
+
+
+def sqlite_lock_state(lock_path: str) -> SQLiteLockState:
+    with SQLITE_LOCKS_GUARD:
+        state = SQLITE_LOCKS.get(lock_path)
+        if state is None:
+            from filelock import FileLock
+
+            state = SQLiteLockState(
+                thread_lock=threading.Lock(),
+                file_lock=FileLock(lock_path, timeout=30, thread_local=False),
+            )
+            SQLITE_LOCKS[lock_path] = state
+        return state
+
+
+class SQLiteFingerprintLock:
+    def __init__(self, lock_path: str) -> None:
+        self._state = sqlite_lock_state(lock_path)
+
+    async def __aenter__(self) -> None:
+        await asyncio.to_thread(self._state.thread_lock.acquire)
+        try:
+            await asyncio.to_thread(self._state.file_lock.acquire)
+        except Exception:
+            self._state.thread_lock.release()
+            raise
+
+    async def __aexit__(self, *args: Any) -> None:
+        try:
+            await asyncio.to_thread(self._state.file_lock.release)
+        finally:
+            self._state.thread_lock.release()