Skip to content

feat: replace request_type with service + operation in inference log inserts#1207

Merged
pyramation merged 1 commit into
mainfrom
feat/usage-service-operation
May 21, 2026
Merged

feat: replace request_type with service + operation in inference log inserts#1207
pyramation merged 1 commit into
mainfrom
feat/usage-service-operation

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

Summary

Coordinates with constructive-db PR #1281 (merged) which added service and operation fields to inference_log_module replacing the old request_type field.

Changes in graphile-llm/src/metering.ts:

  • InferenceLogEntry interface: requestTypeservice + operation fields
  • Added cacheReadTokens, cacheWriteTokens (for Anthropic/OpenAI cache billing)
  • Added rawUsage (jsonb for raw provider response audit)
  • logInferenceUsage() SQL updated to INSERT all new columns (20 params)
  • Embedding calls: service: 'embedding', operation: 'create'
  • Chat calls: service: 'llm', operation: 'chat'

Changes in graphql/server/src/middleware/llm-api.ts:

  • logInference() function: requestTypeservice + operation
  • SQL INSERT updated to 11 params
  • Both callers (streaming + non-streaming) updated

Review & Testing Checklist for Human

  • Verify the INSERT column order matches the inference_log_module field order from constructive-db PR #1281
  • After merging, do an e2e test: send a chat message → verify usage_log_inferences has service='llm', operation='chat'
  • Verify embedding requests produce service='embedding', operation='create'

Notes

The service field uses a fixed vocabulary of ~8 values: llm, embedding, tts, stt, ocr, image_gen, search, compute. The operation field is higher cardinality and service-specific.

cacheReadTokens, cacheWriteTokens, and rawUsage are passed as null for now — they'll be populated when Anthropic/OpenAI provider support is added.

Link to Devin session: https://app.devin.ai/sessions/2b5a29d83d3f478e8d3d972653b4879c
Requested by: @pyramation

…inserts

- InferenceLogEntry: requestType → service + operation fields
- Add cacheReadTokens, cacheWriteTokens, rawUsage fields
- Update logInferenceUsage SQL to match new inference_log schema
- Update llm-api.ts logInference function + both callers
- Embedding calls: service='embedding', operation='create'
- Chat calls: service='llm', operation='chat'

Coordinates with constructive-db PR #1281 (merged).
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@pyramation pyramation merged commit 25c5eab into main May 21, 2026
37 checks passed
@pyramation pyramation deleted the feat/usage-service-operation branch May 21, 2026 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant