Skip to content

[bot] Cohere Embed Jobs API (batch embeddings) has zero instrumentation #2426

Description

@braintrust-bot

Summary

Cohere's official TypeScript SDK (cohere-ai on npm) exposes an embedJobs resource — the Embed Jobs API — for submitting large-scale, asynchronous batch embedding jobs (client.embedJobs.create({ model, datasetId, inputType, ... }), plus embedJobs.get, embedJobs.list, and embedJobs.cancel). Cohere's own docs recommend this API instead of the synchronous Embed endpoint for "upwards of 100,000 embeddings." This repo's Cohere instrumentation covers the synchronous embed call but has no channel, plugin logic, or wrapper coverage at all for embedJobs, so submitting or completing a batch embedding job produces zero trace — while a synchronous client.embed() call on the exact same client is fully traced (input, output minimized to embedding length, token metrics).

This is the batch/async counterpart to embeddings execution — squarely the same category of gap already tracked for other providers' batch inference APIs (see #2374), except #2374 does not list Cohere at all, so this specific provider+API combination has no existing tracking issue.

What instrumentation is missing

In js/src/instrumentation/plugins/cohere-channels.ts, defineChannels("cohere-ai", {...}) defines exactly four channels: chat, chatStream, embed, rerank. There is no embedJobs (or embedJobsCreate/embedJobsGet) channel.

In js/src/instrumentation/plugins/cohere-plugin.ts and js/src/wrappers/cohere.ts, the wrapped/instrumented surface only proxies chat, chatStream, embed, and rerank methods — embedJobs.create, embedJobs.get, embedJobs.list, and embedJobs.cancel are not intercepted, so calls pass through completely untraced (no task span for the job, no span capturing the job's model/input dataset/status, no completion span with token/embedding-count metrics once the job finishes).

Braintrust docs status

not_found. The Cohere provider docs page does not exist at a dedicated braintrust.dev/docs/providers/cohere path distinct from the generic integrations list, and no Braintrust documentation mentions "Embed Jobs," embedJobs, or batch/async embeddings for Cohere anywhere. Cohere is only referenced generically as one of several supported chat/embeddings providers in Braintrust's tracing guide (https://www.braintrust.dev/docs/guides/tracing), with no endpoint-level detail.

Upstream sources

Braintrust docs sources checked

Local repo files inspected

  • js/src/instrumentation/plugins/cohere-channels.ts — full channel list: chat, chatStream, embed, rerank (no embedJobs)
  • js/src/instrumentation/plugins/cohere-plugin.ts — provider plugin subscription/extraction logic, no embedJobs handling
  • js/src/wrappers/cohere.ts — manual wrapper proxy, no embedJobs method interception
  • js/src/instrumentation/plugins/cohere-plugin.test.ts, js/src/wrappers/cohere.test.ts — existing test coverage exercises only chat/chatStream/embed/rerank

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions