Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/publish-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Publish image

on:
push:
branches: ["main"]
workflow_dispatch:

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
15 changes: 15 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM python:3.11-slim

WORKDIR /app

COPY pyproject.toml ./
COPY src ./src

RUN pip install --no-cache-dir .

RUN useradd --create-home --shell /usr/sbin/nologin appuser
USER appuser

EXPOSE 8000

CMD ["uvicorn", "sync_engine.app:create_app", "--factory", "--host", "0.0.0.0", "--port", "8000"]
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,12 @@ sync-engine/
│ └── adr/ # Architecture Decision Records
│ └── ADR-001-hybrid-rest-webhook.md
├── src/
│ └── sync_engine/ # main package (upcoming)
│ ├── webhook/ # webhook reception and verification
│ ├── reconciliation/ # REST polling loop
│ ├── processor/ # idempotent event processing
│ └── store/ # sync state persistence
│ └── sync_engine/
│ ├── app.py # FastAPI app: /health, /webhooks
│ ├── webhook/ # webhook reception and verification
│ ├── reconciliation/ # REST polling loop (upcoming)
│ ├── processor/ # idempotent event processing (upcoming)
│ └── store/ # sync state persistence
├── tests/
├── CLAUDE.md
└── README.md
Expand All @@ -59,6 +60,8 @@ sync-engine/
| ADR | Decision | Status |
|-----|----------|--------|
| [ADR-001](docs/adr/ADR-001-hybrid-rest-webhook.md) | Hybrid REST/webhook architecture | Accepted |
| [ADR-002](docs/adr/ADR-002-pluggable-store-interface.md) | Pluggable store interface via abstract base class | Accepted |
| [ADR-003](docs/adr/ADR-003-webhook-hmac-sha256-signature-verification.md) | HMAC-SHA256 signature verification for webhook reception | Accepted |

---

Expand All @@ -80,7 +83,7 @@ Services from `fullstack-pilot` — details added to `src/` as implementations l
## Development

```bash
# setup (upcoming)
# setup
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

Expand Down
1 change: 1 addition & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Tests mirror the source tree under `tests/`:
```
tests/
├── test_exceptions.py
├── test_app.py
├── store/
│ └── test_memory_store.py
├── webhook/
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ requires-python = ">=3.11"
dependencies = [
"fastapi",
"httpx",
"uvicorn[standard]",
]

[project.optional-dependencies]
Expand Down
45 changes: 45 additions & 0 deletions src/sync_engine/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import os

from fastapi import FastAPI, HTTPException, Request

from sync_engine.exceptions import DuplicateEventError, WebhookSignatureError
from sync_engine.store.base import SyncStore
from sync_engine.store.memory import InMemoryStore
from sync_engine.webhook.handler import WebhookHandler


def create_app(store: SyncStore | None = None, secret: str | None = None) -> FastAPI:
store = store or InMemoryStore()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require a persistent store in the deployment factory

When create_app() is invoked as the no-argument deployment factory, every process silently constructs the test-only InMemoryStore. A successful webhook response then acknowledges data that is lost on any restart and is not shared across multiple workers, defeating the documented persistent-queue and at-least-once guarantees; production startup should inject a persistent backend or fail rather than defaulting to this store.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require a persistent store in the production factory

When the new Dockerfile runs uvicorn ... --factory, it invokes create_app() without arguments, so every deployed container takes this branch and uses InMemoryStore, which is explicitly test-only. The API therefore reports webhooks as accepted even though the queue is lost on every restart and cannot be shared across replicas, breaking the engine's at-least-once delivery guarantee; the production factory needs to construct a configured persistent SyncStore instead.

Useful? React with 👍 / 👎.

secret = secret if secret is not None else os.environ["WEBHOOK_SECRET"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject empty webhook secrets at startup

If WEBHOOK_SECRET exists but is empty, or a caller explicitly supplies secret="", the application starts and authenticates HMACs using the publicly guessable empty key. In that misconfiguration, anyone can forge accepted webhooks; startup should reject empty secrets instead of checking only for None or a missing environment key.

Useful? React with 👍 / 👎.

handler = WebhookHandler(store, secret)

app = FastAPI(title="sync-engine")

@app.get("/health")
def health() -> dict:
return {"status": "ok", "last_sync_at": store.get_last_sync_at()}

@app.post("/webhooks")
async def receive_webhook(request: Request) -> dict:
event_id = request.headers.get("X-Event-Id")
if not event_id:
raise HTTPException(status_code=400, detail="Missing X-Event-Id header")

raw_bytes = await request.body()
signature = request.headers.get("X-Hub-Signature-256", "")

try:
payload = await request.json()
except ValueError as exc:
raise HTTPException(status_code=400, detail="Invalid JSON body") from exc
Comment on lines +31 to +34

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-object JSON webhook payloads

A validly signed JSON array, scalar, or null is accepted here and passed to WebhookHandler, even though both the handler and SyncStore.enqueue_webhook require a dict. This places values outside the store contract into the queue and can break downstream processing; validate that the decoded payload is an object and return a 400 otherwise.

Useful? React with 👍 / 👎.

Comment on lines +31 to +34

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-object JSON payloads before enqueueing

For a correctly signed body such as [], null, or a JSON string, request.json() succeeds and the endpoint returns accepted, even though WebhookHandler.handle() and SyncStore.enqueue_webhook() require a dictionary payload. This places values outside the store contract into the queue and can make downstream processing fail when it treats them as mappings; validate that the decoded value is an object before calling the handler.

Useful? React with 👍 / 👎.


try:
handler.handle(event_id, payload, raw_bytes, signature)
except WebhookSignatureError as exc:
raise HTTPException(status_code=401, detail=str(exc)) from exc
except DuplicateEventError:
return {"status": "duplicate"}

return {"status": "accepted"}

return app
84 changes: 84 additions & 0 deletions tests/test_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import hashlib
import hmac

import pytest
from fastapi.testclient import TestClient

from sync_engine.app import create_app
from sync_engine.store.memory import InMemoryStore

SECRET = "test-secret"


def _sig(payload: bytes, secret: str = SECRET) -> str:
return hmac.new(secret.encode(), payload, hashlib.sha256).hexdigest()


@pytest.fixture
def store() -> InMemoryStore:
return InMemoryStore()


@pytest.fixture
def client(store: InMemoryStore) -> TestClient:
return TestClient(create_app(store=store, secret=SECRET))


def test_health_before_any_sync_reports_no_last_sync_at(client: TestClient) -> None:
response = client.get("/health")
assert response.status_code == 200
assert response.json() == {"status": "ok", "last_sync_at": None}


def test_webhook_valid_signature_is_enqueued(
client: TestClient, store: InMemoryStore
) -> None:
body = b'{"type": "created"}'
response = client.post(
"/webhooks",
content=body,
headers={"X-Event-Id": "evt-1", "X-Hub-Signature-256": _sig(body)},
)
assert response.status_code == 200
assert response.json() == {"status": "accepted"}
assert ("evt-1", {"type": "created"}) in store.dequeue_unacknowledged()


def test_webhook_invalid_signature_returns_401_and_not_enqueued(
client: TestClient, store: InMemoryStore
) -> None:
body = b'{"type": "created"}'
response = client.post(
"/webhooks",
content=body,
headers={
"X-Event-Id": "evt-1",
"X-Hub-Signature-256": _sig(body, secret="wrong-secret"),
},
)
assert response.status_code == 401
assert store.dequeue_unacknowledged() == []


def test_webhook_missing_event_id_header_returns_400(client: TestClient) -> None:
body = b'{"type": "created"}'
response = client.post(
"/webhooks",
content=body,
headers={"X-Hub-Signature-256": _sig(body)},
)
assert response.status_code == 400


def test_webhook_already_processed_event_returns_duplicate_status(
client: TestClient, store: InMemoryStore
) -> None:
body = b'{"type": "created"}'
headers = {"X-Event-Id": "evt-1", "X-Hub-Signature-256": _sig(body)}

client.post("/webhooks", content=body, headers=headers)
store.mark_event_processed("evt-1")

response = client.post("/webhooks", content=body, headers=headers)
assert response.status_code == 200
assert response.json() == {"status": "duplicate"}
Loading