Skip to content

feat: add agentic-server package — standalone Express LLM service#1214

Merged
pyramation merged 1 commit into
mainfrom
feat/agentic-server
May 23, 2026
Merged

feat: add agentic-server package — standalone Express LLM service#1214
pyramation merged 1 commit into
mainfrom
feat/agentic-server

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

Summary

Adds agentic-server — the Express-only equivalent of graphile-llm. This is a standalone Express router that provides agent threads, chat streaming, billing metering, and inference logging without needing PostGraphile or any Graphile dependencies.

Uses @constructive-io/express-context for tenant-scoped database access (the whole reason we built express-context).

Endpoints:

Method Path Description
POST /v1/threads Create a new conversation thread
POST /v1/threads/:thread_id/messages Send messages + get AI response (streaming SSE)
POST /v1/orgs/:entity_id/threads Create thread (entity-scoped)
POST /v1/orgs/:entity_id/threads/:thread_id/messages Send message (entity-scoped)
POST /v1/embed Generate embeddings

Architecture:

Cloud Function → POST /v1/threads/:id/messages → agentic-server
                                                      │
                                  ┌───────────────────┼───────────────────┐
                                  │                   │                   │
                         express-context        OllamaAdapter        Billing
                        (tenant DB context)    (LLM provider)    (quota + usage)

Dependencies (minimal):

  • @constructive-io/express-context — tenant context (domain resolution, JWT, pgSettings, withPgClient)
  • @agentic-kit/ollama — LLM provider calls (chat + embeddings)
  • express (peer dep)
  • pg, pg-cache, @pgpmjs/logger — database connectivity and logging

No graphile-anything. The few SQL queries for agent/billing discovery live inside the package with a simple TTL cache (~45 lines).

Review & Testing Checklist for Human

  • Verify the router mounts cleanly alongside the existing PostGraphile server (mount createAgenticRouter() at the top of server.ts to confirm no conflicts)
  • Test streaming SSE output format matches what the frontend/cloud functions expect (OpenAI-compatible format)
  • Verify billing check_quota / record_usage SQL matches what the billing module actually exposes (function signatures)

Notes

  • Build passes (pnpm build), 8 unit tests pass (pnpm test)
  • Lint is skipped (pre-existing monorepo-wide issue: no eslint flat config for packages/*)
  • This extracts the logic from graphql/server/src/middleware/llm-api.ts into a standalone package, but does NOT remove the original llm-api.ts yet — that can happen once this package is verified in production
  • The package publishes as bare agentic-server (no scope) per user request

Link to Devin session: https://app.devin.ai/sessions/2b5a29d83d3f478e8d3d972653b4879c
Requested by: @pyramation

Express-only equivalent of graphile-llm: agent threads, chat streaming,
billing metering, and inference logging. Uses @constructive-io/express-context
for tenant-scoped database access.

Endpoints:
- POST /v1/threads — create conversation thread
- POST /v1/threads/:id/messages — send message + stream response (SSE)
- POST /v1/orgs/:entity_id/threads — entity-scoped thread creation
- POST /v1/orgs/:entity_id/threads/:id/messages — entity-scoped messaging
- POST /v1/embed — generate embeddings

Features:
- OllamaAdapter for chat (streaming + batch) and embeddings
- Automatic billing quota check + usage recording
- Inference logging (token counts, latency, model)
- TTL-based discovery caching (no graphile-cache dependency)
- OpenAI-compatible SSE streaming format
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@pyramation pyramation merged commit 27eb672 into main May 23, 2026
37 checks passed
@pyramation pyramation deleted the feat/agentic-server branch May 23, 2026 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant