feat: introduce expiresInSeconds to .set() - #3
Conversation
|
One thing I've noticed reading this - just as an initial comment, is that this code is incredibly hard to follow. There are a lot of single-letter variable names. Reading this with absolutely zero context is incredibly difficult. |
|
Code read, I do agree that single char keys are making it slightly harder to read (my own code is usually more messy), but all in all the code is quite easy to follow if you ask me. I have ran the Jasmine tests a few times locally, every single time at least one or more tests fail, inconsistent result. |
|
Stress tested stower for concurrent/container reliability. Found that when 8+ processes flush to disk at the same time (e.g. rolling deploys, container restarts), some writes are silently dropped. The lock retry budget (~1.5s) gets exhausted and Happy to rewrite the script I wrote into a jasmine test if you'd find this beneficial |
Summary
Adds optional TTL support to stower and rewrites the disk I/O layer for multi-process safety. This is a semver-major release (v2.1.0 → v3.0.0) because the
set()signature changes and the on-disk JSON format now includes a reserved__expires__key.What changed
TTL / expiry (
expiresInSeconds)set(key, value, expiresInSeconds)accepts an optional 3rd argument — a TTL in seconds after which the key becomes invisible to reads__expires__is a reserved key in the JSON file used to persist TTL timestamps — blocked from user useget,exists,keys,values) respect expiryMulti-process safety overhaul
.lockfile mechanism withproper-lockfilefor robust cross-process mutual exclusion_dirtymap) — each process tracks which keys it changed since the last flush. On write, only dirty keys are merged on top of the current disk state, so concurrent writers don't clobber each otherload()now re-reads the file only when itsmtimehas changed and re-applies dirty keys on top of the freshly loaded state, preventing a reload from discarding pending in-memory writes<file>.<pid>.tmp) replace the shared.tmpfile to avoid cross-process collisionspersist()is now synchronoussleepSync()instead ofsetTimeout, sopersist()completes before returning. This simplifies startup in Docker environments where volume mounts may appear slightly after process start.SIGTERM handler
process.on('SIGTERM', ...)added alongside the existingSIGINThandler sodocker stopflushes pending writesFiles changed
index.jsproper-lockfile, mtime-based reload, atomic per-PID temp files,mergeAndSerialize/commitToDisksplittests/expires.spec.js__expires__reservation, disk pruningtests/multiprocess.spec.jstests/helpers/worker.jstests/methods.spec.jsREADME.mdpackage.jsonproper-lockfileas a production dependency.gitignorecache/*.jsonfiles from testsHow to review
index.js(set,get,remove,exists,keys,values,clear) — each is intentionally shortload(),reapplyDirtyKeys(),mergeAndSerialize(),commitToDisk(),write()tests/expires.spec.jsfor TTL behavior andtests/multiprocess.spec.jsfor concurrency guarantees