Skip to content

feat(bin): add durable typed external-event ingress - #2795

Closed
hsuperman wants to merge 16 commits into
kunchenguid:mainfrom
hsuperman:fm/linear-event-intake
Closed

feat(bin): add durable typed external-event ingress#2795
hsuperman wants to merge 16 commits into
kunchenguid:mainfrom
hsuperman:fm/linear-event-intake

Conversation

@hsuperman

@hsuperman hsuperman commented Aug 22, 2026

Copy link
Copy Markdown

Intent

Deliver PR #2795 only as the smallest durable, upgrade-safe typed external-event ingress foundation for future Linear bug detection. The repository-owned behavior is a stable local or operator-controlled SSH ingestion boundary for authenticated external producers, not a complete Linear polling or routing system. Event and webhook bytes are untrusted hints, never captain instructions, approval, or authorization; wake-time handling must query Linear authoritatively before creating, routing, or dispatching work, and mutable initiative/domain ownership stays in existing secondmate routing. Retain bounded private capture before payload-free wake publication, exact canonical Linear delivery identity linear:@, durable idempotent deduplication and coalescing across independent external webhook and reconciliation producers, restart replay, handled acknowledgement behavior, and executable regressions for those guarantees. Remove bin/fm-linear-reconcile.sh, its tests, inventory and verification entries, and every claim that Firstmate ships or owns authoritative Linear polling, workspace enumeration, hourly scheduling, or full end-to-end secondmate routing. The external operator contract requires an authenticated immediate Linear webhook or automation plus an external operator-owned initiative-wide hourly reconciliation scan invoking ingest-linear; periodic fleet sweeps or manual external scheduler invocation remain interim coverage. Active homes can observe queued events through their existing watcher or away supervisor, while a completely idle home cannot promptly consume them through a supported repository boundary. Do not add a watcher, timer, daemon, supervision owner, public listener, polling hack, retry loop, synthetic captain message, or another producer. Tracked operator docs describe only current supported behavior and setup, never contribution or upstream-PR delivery guidance. PR intent, risk, testing, and attestation must describe this as a safe partial foundation and explicitly state that automatic Linear detection, reconciliation, and routing remain external and are not proven; make no overnight guarantee. Validate the complete final head and report whether this narrowed partial foundation itself is merge-ready or still held. Do not merge.

What Changed

  • Add a bounded, private typed external-event ingress for authenticated local or operator-controlled SSH producers, including canonical Linear delivery identities.
  • Durably deduplicate and coalesce event revisions, publish payload-free wakes after capture, and support restart replay and handled acknowledgements through the existing process-event machinery.
  • Treat unhandled results as supervision work and document the untrusted-hint boundary, authoritative wake-time Linear lookup, and externally owned detection and reconciliation requirements.

Risk Assessment

✅ Low: The change is a well-bounded typed-ingress foundation, retains the required durability and trust-boundary behavior, and removes the forbidden repository-owned Linear reconciliation producer and related claims.

Testing

Focused ingress and process-event regressions passed, and the CLI transcript demonstrates canonical identity, private bounded capture, deduplication, restart replay, and handled acknowledgement as an operator experiences them. No visual artifact was appropriate because this is a CLI-only boundary; the broader guard test exposed an unrelated pre-existing copy mismatch.

Evidence: Linear event ingress end-to-end CLI transcript

Source: Linear event ingress end-to-end CLI transcript

Operator invokes authenticated local/SSH boundary twice for the same Linear revision:
accepted: event-81d07b6d34ef02a56f1fcb2bc72a863a6b9fcf3d3f7f6458e1eee6f8c7 1 consumption=active-supervisor-required
duplicate: event-81d07b6d34ef02a56f1fcb2bc72a863a6b9fcf3d3f7f6458e1eee6f8c7 1 consumption=active-supervisor-required

Persisted typed metadata:
source=linear
delivery=linear:4a1bc793-6f51-4d52-91c0-6d8b76ee2a40@2026-08-22T10:00:00.000Z

Payload-free wake queue:
check: procevent:event-81d07b6d34ef02a56f1fcb2bc72a863a6b9fcf3d3f7f6458e1eee6f8c7:1

Privacy/authority checks:
durable-result-mode=600
payload-leaked-to-wake=no
captain-inbox-created=no

Simulated restart replay:
check: procevent:event-81d07b6d34ef02a56f1fcb2bc72a863a6b9fcf3d3f7f6458e1eee6f8c7:1

Handled acknowledgement and retry:
duplicate: event-81d07b6d34ef02a56f1fcb2bc72a863a6b9fcf3d3f7f6458e1eee6f8c7 1 consumption=active-supervisor-required
wake-after-handled-retry=no

Temporary demo root: /tmp/fm-linear-evidence.fs8eMy
- Outcome: ⚠️ 1 warning across 1 run (3m19s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed ✅
  • 🚨 bin/fm-linear-reconcile.sh:1 - Intent requires: “Remove bin/fm-linear-reconcile.sh, its tests, inventory and verification entries, and every claim that Firstmate ships or owns authoritative Linear polling.” The final commit re-adds this repository-owned workspace polling producer, its executable tests, and documentation/inventory claims. Remove the script, tests, and associated docs entries so hourly reconciliation remains external operator-owned.

🔧 Fix: Remove repository-owned Linear reconciliation producer
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 warning
  • ⚠️ tests/fm-turnend-guard.test.sh:22 - The broader turn-end guard regression fails on a pre-existing copy-contract mismatch: the test expects “Stop-owned automatic recovery,” while both base and target implementation say “stop-hook park.” The target diff changes neither message nor assertion. The narrowed external-event foundation passes its focused validation, but complete-head validation remains held by this unrelated existing failure.
  • tests/fm-procevent-external-event.test.sh
  • tests/fm-procevent.test.sh
  • tests/fm-turnend-guard.test.sh (stopped on unrelated pre-existing message mismatch)
  • Manual operator-style ingest-linear check: accepted uppercase UUID, canonicalized delivery identity, deduplicated lowercase retry, verified 0600 private capture and payload-free wake, replayed after simulated restart, acknowledged handled state, and confirmed retry emitted no wake
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c68f2c4982

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

die 'cannot durably capture event'
}
seq=$(fm_procevent_result_sequence "$result")
if ! publish_locked "$id" "$seq"; then status=1; fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wake supervision when an idle event arrives

Captain, when this ingress runs after an idle home's one-shot watcher has exited, publish_locked only appends to .wake-queue, leaving no process to observe the event. FM_SUP_RESULTS can retain or re-arm supervision only during a later hook or command, so the documented webhook path will not wake Firstmate promptly—or overnight—until unrelated activity occurs; notify or arm the existing watcher after durable publication.

AGENTS.md reference: AGENTS.md:L110-L111

Useful? React with 👍 / 👎.

@greptile-apps

greptile-apps Bot commented Aug 22, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

The PR appears safe to merge within its explicitly narrowed typed-ingress scope.

No blocking failure remains.

Reviews (8): Last reviewed commit: "no-mistakes(document): Clarify active-ho..." | Re-trigger Greptile

@hsuperman hsuperman changed the title feat(bin): add durable typed external event intake feat(bin): add durable typed external event ingress Aug 22, 2026
Comment thread docs/configuration.md Outdated
@hsuperman hsuperman changed the title feat(bin): add durable typed external event ingress feat(bin): add durable typed external-event ingress Aug 22, 2026
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

VISION: aligns. Durable typed ingress treats event bytes as untrusted hints, never captain text or authorization; restart replay and handled-ack keep obligations in records. No HTTP listener; callers authenticate outside Firstmate and invoke a local or operator-SSH command. Wake-time handling must query Linear authoritatively before creating or routing work.

Class: opt-in. New local surface bin/fm-procevent-external-event.sh (ingest / ingest-linear). Not default-on ingress: nothing polls, no daemon, no public listener. Active homes observe via the existing watcher/away supervisor; an idle home retains the event.

Security: none after review. Payload captured private 0600, wake is payload-free, source/delivery charset-bounded, max bytes capped. Untrusted hint boundary is explicit. No workflow-file change.

Overlap: bin/fm-watch.sh with #2749 and #2631 (supervision of unhandled results). Does not own Linear polling or secondmate routing.

CI: HEAD efea26efa4a103ecd9b81ed0e5762418c623cef3. CI run 32586565980 in_progress (approved this pass, not green). Body-compliance 32586565934 and 32586589874 SUCCESS. Structured no-mistakes-pipeline-attestation:v1 head_sha matches THIS HEAD. Greptile SUCCESS — not a gate.

Workflows approved: yes (CI 32586565980, NM 32586565934, NM 32586589874). Not land-eligible while CI is unfinished. Waiting on CI, not the captain. Captain-flag only if later fully green AND this became default-on (it is not: local command).

@hsuperman

Copy link
Copy Markdown
Author

Closing per captain decision: Firstmate-repo development is paused, focus is on RecruitMagic only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants