equationalapplications · equationalapplications · Aug 9, 2026 · Aug 9, 2026 · Aug 9, 2026 · Aug 9, 2026
diff --git a/README.md b/README.md
@@ -1,19 +1,22 @@
 # sqlite-s3-agent-tutorial
 
 A working example of the **SQLite-as-a-database-for-an-agent-on-AWS, rehydrated-by-S3**
-pattern: a Discord bot that checks the weather and Bitcoin price once a day, asks an LLM
-(Amazon Bedrock) to turn the raw value into a friendly message, posts it to a Discord
-webhook, and remembers what it already posted — all state lives in a single SQLite file
-in S3. No database server, no VPC. The same file also doubles as a vector database: each
-posted message gets embedded (Titan Text Embeddings V2) and searched with `sqlite-vec`,
-so the bot can mention the closest past result — see
-[docs/08-rag-vector-search.md](docs/08-rag-vector-search.md).
+pattern: a Discord bot that checks the weather and Bitcoin price on a schedule, asks an LLM
+(Amazon Bedrock) to turn the day's readings into a friendly message plus a closing haiku,
+and posts it to a Discord webhook — all state lives in a single SQLite file in S3. No
+database server, no VPC. The same file also doubles as a vector database: each tick's
+message gets embedded (Titan Text Embeddings V2) and searched with `sqlite-vec`, so the
+bot can mention the closest past result — see
+[docs/08-rag-vector-search.md](docs/08-rag-vector-search.md). There is no dedup — every
+tick posts, deliberately, to keep the tutorial's control flow simple; see
+[docs/03-schema.md](docs/03-schema.md).
 
 ## Quick start
 
 ```bash
 npm install
 npm test
+npm run typecheck
 
 # Put your webhook URL in an untracked .env (see docs/06-discord-webhook-setup.md),
 # then source it and run the writer — keeping the URL out of shell history.
@@ -30,11 +33,16 @@ real thing:
 export AWS_PROFILE=your-profile
 # Source the webhook URL from an untracked file rather than echoing it inline —
 # `infra/stack.ts` reads DISCORD_WEBHOOK_URL at synth time and throws if it is unset.
+# `.env.discord` is a separate file from the local-run `.env` above so a deploy never
+# accidentally picks up other local-only vars (e.g. a test `DB_PATH` override) from
+# the file meant for `npm run local-fetch`.
 set -a; . ./.env.discord; set +a   # .env.discord is gitignored
 npm run deploy
 npm run smoke
 ```
 
+`npm run smoke` is read-only and safe to run any time, including while a loop tick is in flight — it never invokes `fetch`, never posts to Discord, and never calls Bedrock. It probes the status Function URL with SigV4 and asserts the URL actually requires it.
+
 Before your first deploy, ensure your AWS account in `us-east-1` has an active AWS
 Marketplace subscription for `zai.glm-4.7-flash` (Bedrock enables foundation-model access
 by default in commercial Regions once the Marketplace subscription is in place; the legacy
@@ -61,7 +69,7 @@ haiku. If a past message in the corpus is close enough, the LLM's pre-suffix out
 is mechanically appended with a `Reminds me of: <past message>` line. All three
 scripts read the rule name from the `LoopRuleName` stack output and call the
 EventBridge API directly using the same AWS CLI credentials the smoke script
-already requires. `loop-status` is read-only — print the rule's current
+already requires. `loop-status` is read-only — it prints the rule's current
 `ENABLED`/`DISABLED` state plus its schedule expression and ARN.
 
 **Stop the loop when you're done** — `loop-stop.sh` disables the EventBridge rule so
@@ -72,40 +80,122 @@ confirm the state, then re-run `loop-stop.sh` if you want the loop to stay off.
 [docs/07-budget-protection.md](docs/07-budget-protection.md) for the per-day
 Bedrock call rate at 5-min cadence.
 
+To verify the reader side of the loop (no Discord post, no Bedrock call), run
+`npm run smoke` — it checks the status endpoint and confirms it is SigV4-protected.
+
 ## Triggering a fetch on demand
 
-The daily `fetch` run is normally EventBridge's job, but you can also trigger one over
-HTTP via the same Function URL the `status` op uses. This is off by default — set
-`FETCH_TRIGGER_TOKEN` before deploying (`export FETCH_TRIGGER_TOKEN=...` before
-`npm run deploy`, alongside `DISCORD_WEBHOOK_URL`), then:
+The scheduled `fetch` run is normally EventBridge's job, but you can also trigger one over
+HTTP via the same Function URL the `status` op uses. The Function URL is locked to
+AWS_IAM — your CLI credentials must be authorized against the same-account URL grant
+the stack synthesizes (smoke-status-iam design §3.1) before the request reaches the
+handler, so the request must be SigV4-signed the same way `scripts/smoke.sh` signs its
+status probe (a plain `curl` without `--aws-sigv4` gets `403` from the URL itself). This
+is off by default — set `FETCH_TRIGGER_TOKEN` before deploying (`export
+FETCH_TRIGGER_TOKEN=...` before `npm run deploy`, alongside `DISCORD_WEBHOOK_URL`), then
+the token goes on the query string (`?token=...`) and `op` goes in the JSON body —
+`resolveOp` in `src/handler.ts` only reads `op` from the top-level payload or the JSON
+`body`, and the token check reads `queryStringParameters.token`, so putting the token in
+the body or `op` on the query string will not work. Reusing the same `netrc` machinery
+as the smoke script handles the access key / secret pair cleanly; `aws configure
+export-credentials --format process` resolves SSO / session credentials too, and the
+`X-Amz-Security-Token` header is added when the resolved credentials include a session
+token:
 
 ```bash
-curl -X POST "$FUNCTION_URL?token=$FETCH_TRIGGER_TOKEN" --data '{"op":"fetch"}'
+FUNCTION_URL=$(aws cloudformation describe-stacks \
+  --profile "$AWS_PROFILE" --region "$AWS_REGION" \
+  --stack-name SqliteS3AgentTutorial \
+  --query "Stacks[0].Outputs[?OutputKey=='AgentFunctionUrl'].OutputValue" --output text)
+
+# Build a 0600 netrc file from the resolved credential chain (env vars, SSO,
+# credential_process, etc.). Unlike `aws configure get`, this handles every
+# profile type the CLI supports.
+NETRC=$(mktemp); chmod 600 "$NETRC"
+FUNCTION_HOST=$(echo "$FUNCTION_URL" | sed -E 's#^https?://([^/]+).*#\1#')
+CREDENTIALS=$(aws configure export-credentials --profile "$AWS_PROFILE" --format process)
+printf 'machine %s login %s password %s\n' \
+  "$FUNCTION_HOST" \
+  "$(jq -r '.AccessKeyId' <<<"$CREDENTIALS")" \
+  "$(jq -r '.SecretAccessKey' <<<"$CREDENTIALS")" > "$NETRC"
+
+# Session token (SSO / assumed-role) is required by SigV4 when present.
+SESSION_TOKEN=$(jq -r '.SessionToken // empty' <<<"$CREDENTIALS")
+if [ -n "$SESSION_TOKEN" ]; then
+  TOKEN_FILE=$(mktemp); chmod 600 "$TOKEN_FILE"
+  printf 'X-Amz-Security-Token: %s\n' "$SESSION_TOKEN" > "$TOKEN_FILE"
+  SESSION_HEADER=(--header @"$TOKEN_FILE")
+fi
+
+curl -X POST "$FUNCTION_URL?token=$FETCH_TRIGGER_TOKEN" \
+  --aws-sigv4 "aws:amz:$AWS_REGION:lambda" \
+  --netrc-file "$NETRC" \
+  --header 'Content-Type: application/json' \
+  "${SESSION_HEADER[@]}" \
+  --data '{"op":"fetch"}'
 ```
 
-Without a matching token, an HTTP-triggered `fetch` request is rejected with 403; the
-scheduled EventBridge fetch is unaffected either way. Since anyone with a valid token can
-run this repeatedly (a real Bedrock call and Discord post each time), see
+Without a matching token *or* an authorized same-account IAM principal, an HTTP-triggered
+`fetch` request is rejected with 403; the scheduled EventBridge fetch is unaffected either
+way. Both checks are required — the token alone no longer suffices, and the IAM grant
+alone without a token is treated the same as no token. Since an authorized caller with a
+valid token can run this repeatedly (a real Bedrock call and Discord post each time), see
 [docs/07-budget-protection.md](docs/07-budget-protection.md) before relying on this in a
 deploy you leave running unattended.
 
+## ⚠️ Concurrency Limitations & Scaling Up
+
+The **SQLite-rehydrated-by-S3** pattern operates under a strict **Single-Writer / Low-Concurrency** constraint. Because Amazon S3 does not support partial file locking, standard POSIX filesystem locks (`WAL` mode, `IMMEDIATE` transactions) are completely blind to concurrent AWS Lambda execution containers.
+
+### The Split-Brain Risk
+If two Lambda functions invoke concurrently and attempt to mutate state:
+1. **Lambda A** and **Lambda B** both download the same original database file from S3.
+2. Both modify their local copy in `/tmp`.
+3. Whichever Lambda finishes last will execute `PutObject` and overwrite the other's changes completely. This results in **silent data loss** (lost updates) and state divergence.
+
+---
+
+### How to Scale Beyond Single-Writer Concurrency
+
+If your agent outgrows a single-writer schedule and requires concurrent read/write access, choose one of the following paths depending on your infrastructure preferences:
+
+#### 1. EFS Mount: The Zero-Server Alternative (Single-Writer Only With Care)
+If you want to keep using SQLite without managing a traditional database server, attach an **Amazon EFS (Elastic File System)** to your Lambda function.
+* **How it works:** AWS mounts an EFS network drive directly to `/mnt/storage` inside your Lambda container.
+* **The Benefit:** A single Lambda can `hydrate-from-EFS` instead of S3, eliminating the per-invocation S3 download.
+* **Trade-off:** Requires moving your Lambda function into a VPC, which introduces minimal network configuration overhead. **EFS is not a true multi-writer substitute for a relational database.** EFS exposes NFSv4 advisory locking only; SQLite's `WAL` mode requires POSIX shared memory that no network filesystem provides, and concurrent writers across multiple Lambda hosts risk 'database is locked' errors and (in failure cases) corruption. The hydrating-Lambda pattern (one writer at a time, S3 as the truth) is the only SQLite-on-Lambda shape the tutorial guarantees. If you genuinely need concurrent writers, skip EFS and pick a client/server database.
+
+#### 2. Litestream / Litefs: The Replication Stream
+[Litestream](https://litestream.io) runs a background sidecar process alongside SQLite that continuously streams WAL (Write-Ahead Log) frames to an S3 bucket every second.
+* **How it works:** Instead of pulling/pushing a giant database file, it replicates granular changes.
+* **The Benefit:** Drastically reduces S3 network I/O, protects against data loss down to the second, and scales read concurrency beautifully.
+* **Trade-off:** Best suited for long-running containers (ECS Fargate) rather than short-lived, ephemeral Lambda functions.
+
+#### 3. Shift to an Architectural Serverless DB
+When cross-agent transactional consistency becomes a core app requirement, migrate the SQLite relational schema and vector lookups into dedicated cloud-native databases:
+* **Relational Data:** Migrate to **Amazon Aurora Serverless v2 (PostgreSQL/MySQL)** or **DynamoDB**.
+* **Vector Engine:** If using `sqlite-vec` for RAG, migrate those embeddings into **Amazon OpenSearch Serverless**, **pgvector** (on Aurora), or **Pinecone**.
+
 ## What's here
 
 | Doc | Covers |
 |---|---|
 | [docs/01-architecture.md](docs/01-architecture.md) | The pattern, in prose: one Lambda, two ops, one bucket |
 | [docs/02-rehydration.md](docs/02-rehydration.md) | Bootstrap, conditional writes, version-cached reads |
-| [docs/03-schema.md](docs/03-schema.md) | Why three tables, not one |
+| [docs/03-schema.md](docs/03-schema.md) | The tables, and why there's no dedup |
 | [docs/04-extending.md](docs/04-extending.md) | Adding a third source |
 | [docs/05-from-tutorial-to-prod.md](docs/05-from-tutorial-to-prod.md) | What changes if you outgrow this |
+| [docs/06-discord-webhook-setup.md](docs/06-discord-webhook-setup.md) | Creating and configuring the Discord webhook |
 | [docs/07-budget-protection.md](docs/07-budget-protection.md) | Setting up an AWS Budget alert, and what could actually drive cost up |
 | [docs/08-rag-vector-search.md](docs/08-rag-vector-search.md) | SQLite as a vector database too: sqlite-vec + Titan embeddings |
 | [docs/09-lesson-script.md](docs/09-lesson-script.md) | A 10-lesson script for teaching the RAG extension (frame, check-in questions, expected reasoning) |
+| [docs/bedrock-model-comparison.md](docs/bedrock-model-comparison.md) | Why `zai.glm-4.7-flash` is the default, and alternatives |
 
 ## Cost
 
-At one Discord post per day, `zai.glm-4.7-flash` costs under $0.02/year. See
-[docs/bedrock-model-comparison.md](docs/bedrock-model-comparison.md) for alternatives.
-For a spending backstop against misconfiguration (e.g. a leaked on-demand fetch trigger
-token — see the on-demand trigger below), see
-[docs/07-budget-protection.md](docs/07-budget-protection.md).
+At the default 5-minute loop cadence (288 ticks/day, 1 Converse + 1 Titan call per tick),
+`zai.glm-4.7-flash` runs roughly $0.02–$0.04/day — see
+[docs/07-budget-protection.md](docs/07-budget-protection.md) for the full breakdown and
+what else can drive cost up (e.g. a leaked on-demand fetch trigger token — see the
+on-demand trigger below). See [docs/bedrock-model-comparison.md](docs/bedrock-model-comparison.md)
+for alternative models and their pricing.
diff --git a/docs/01-architecture.md b/docs/01-architecture.md
@@ -1,19 +1,29 @@
 # Architecture
 
-One Lambda function. Two operations, read as `event.op`: `fetch` (the writer, run daily by
-EventBridge) and `status` (the reader, exposed by a Function URL). Both read and write the
-same single SQLite file that lives durably in one S3 object, but each keeps its own
-transient copy in `/tmp` for the lifetime of the execution environment: the writer at
-`${DB_PATH}` (default `/tmp/memory.db`), the reader at `${DB_PATH}.reader` (default
-`/tmp/memory.db.reader`). Warm Lambda invocations share that `/tmp`, which is exactly what
-lets the status reader reuse its cached database handle (see
-[docs/02-rehydration.md](02-rehydration.md)); cold starts discard it and rehydrate from S3.
+One Lambda function. Two operations, read as `event.op`: `fetch` (the writer, run on a
+5-minute EventBridge schedule — see the README's Loop mode section) and `status` (the
+reader, exposed by a Function URL locked
+to `authType: AWS_IAM`). Both read and write the same single SQLite file that lives
+durably in one S3 object, but each keeps its own transient copy in `/tmp` for the
+lifetime of the execution environment: the writer at `${DB_PATH}` (default
+`/tmp/memory.db`), the reader at `${DB_PATH}.reader` (default `/tmp/memory.db.reader`).
+Warm Lambda invocations share that `/tmp`, which is exactly what lets the status reader
+reuse its cached database handle (see [docs/02-rehydration.md](02-rehydration.md)); cold
+starts discard it and rehydrate from S3.
+
+The Function URL's `AWS_IAM` auth means a status read (or the on-demand HTTP fetch
+trigger) requires a SigV4-signed request from a principal the stack grants access to —
+by default, any principal in the deploying account. The on-demand `FETCH_TRIGGER_TOKEN`
+documented in the README is an application-level defense-in-depth check layered on top
+of that IAM grant, not a substitute for it.
 
 ## Why one file in S3 instead of a database server
 
 A database server (RDS, DynamoDB) needs to exist continuously, whether or not anything is
-happening. This bot runs once a day. Provisioning a server for a workload that is asleep
-99.9% of the time is the wrong trade — you're paying for uptime a cron job doesn't need.
+happening. This bot runs on a 5-minute schedule, and each tick's actual work — a couple of
+API calls, a Bedrock round trip, a Discord post — takes a few seconds. Provisioning a
+server for a workload that is asleep the overwhelming majority of the time is the wrong
+trade — you're paying for uptime a cron job doesn't need.
 SQLite has no server: it's a file format and a library. The only question the "SQLite for
 a stateful Lambda" pattern has to answer is "where does the file live between
 invocations, given Lambda's `/tmp` doesn't survive a cold start?" S3 is the answer:
@@ -43,13 +53,15 @@ protects against an out-of-band `aws s3 cp` — belt and suspenders.
 
 ## Bedrock calls: formatting and embedding
 
-Before formatting, the writer makes a small Bedrock round trip — Titan Text Embeddings
-V2, via `InvokeModel` rather than `Converse` — to embed the raw fetched value and search
-a `sqlite-vec` table inside the same `memory.db` file for the closest same-source past
-notification, so the formatter can fold that match into its prompt. Only after the
-formatted message is posted to Discord does the writer embed and store *that* posted
-notification, via the same Titan call, for future lookups; see
-[docs/08-rag-vector-search.md](08-rag-vector-search.md).
+Each tick makes exactly two Bedrock calls. First, `Converse` against the chat model
+formats all of that tick's readings into one combined message (a friendly comment plus a
+closing haiku) — the LLM is never told about past history. Second, Titan Text Embeddings
+V2, via `InvokeModel` rather than `Converse`, embeds that formatted output once; the
+writer reuses the same vector both to search `agent_embeddings` (a `sqlite-vec` table
+inside `memory.db`) for the closest past tick, across all sources, and to store this
+tick's own vector for future lookups. If a match is found, its text is appended
+mechanically as a "Reminds me of" suffix *after* the Discord post is built — the LLM
+never sees or influences it; see [docs/08-rag-vector-search.md](08-rag-vector-search.md).
 
 ## EventBridge's payload
 

diff --git a/docs/02-rehydration.md b/docs/02-rehydration.md
@@ -46,6 +46,14 @@ against an unchanged snapshot. Re-downloading the whole SQLite file on every req
 work, but it's wasted I/O on a warm Lambda container that already has last version on
 disk.
 
+The reader is reached via the Function URL, which `infra/stack.ts` locks to
+`authType: AWS_IAM` and grants to the deploying account. A status read therefore requires
+a SigV4-signed request from a same-account principal — `curl` without signing (or a
+browser, which can't sign) gets `403` from the URL itself before the handler ever runs.
+The retry-aware `npm run smoke` is the tutorial's end-to-end check that both halves of
+this hold: the unsigned probe returns `403`, the signed probe returns `200` with the
+documented schema.
+
 Instead, the reader's state lives in a closure returned by `createStatusReader` — that
 closure holds the last snapshot's S3 ETag and the open read-only SQLite handle.
 `src/handler.ts` keeps a module-scope `Map<dbPath, StatusReader>` and looks up (or creates)