Add billing event logging for embedding and reranking token usage#2473
Add billing event logging for embedding and reranking token usage#2473Hazel-Datastax wants to merge 24 commits into
Conversation
📈 Unit Test Coverage Delta vs Main Branch
|
Unit Test Coverage Report
|
📉 Integration Test Coverage Delta vs Main Branch (dse69-it)
|
Integration Test Coverage Report (dse69-it)
|
📉 Integration Test Coverage Delta vs Main Branch (hcd-it)
|
Integration Test Coverage Report (hcd-it)
|
tatu-at-datastax
left a comment
There was a problem hiding this comment.
Looks good, some thoughts added.
amorton
left a comment
There was a problem hiding this comment.
some tweaks as discussed, heading in the right direction
| 'billing.events': | ||
| level: INFO |
There was a problem hiding this comment.
for info, maybe make a change. Below is an example of setting a logger category to use a different appender / handler
PLAIN_CONSOLE - defines a new appender / handler that will write using a different format. It is a handler of type console , named PLAIN_CONSOLE, with a different formatt than the default handler called "console"
NOTE: we will want to check with billing peeps what format they want.
this %c %m$n from your code would log
billing.event {"the": json"}
take away %c to just log the json on a line by itself.
the category: io.stargate.sgv2.jsonapi.testbench.reporting.TestBenchConsoleWriter says for this named logger, its level is INFO and it would use the PLAIN_CONSOLE appender / handler, AND importantly DO NOT inherit from above - so do not also use the console appender
log:
console:
format: "%-5p [%t] %d{yyyy-MM-dd HH:mm:ss,SSS} %F:%L - %m%n"
handler:
console:
PLAIN_CONSOLE:
format: "%m%n"
category:
# production log level for this category is DEBUG, way too noisy for tests
'io.stargate':
# Fine for JVM mode; if building native images, use index-dependency instead
'io.quarkus.deployment.steps.ReflectiveHierarchyStep':
level: ERROR
'io.stargate.sgv2.jsonapi.testbench.reporting.TestBenchConsoleWriter':
level: INFO
handlers:
- PLAIN_CONSOLE
use-parent-handlers: false
amorton
left a comment
There was a problem hiding this comment.
hit wrong button, tweaks as discussed
|
I think issue for this is #2137 fwtw. |
What this PR does:
Add billing event logging for embedding and reranking token usage.
Event definition: https://github.com/riptano/cloud-billing/blob/vectorize-usage-design/docs/usage-event-design/vectorize-usage-events.md
Sample logs:
Which issue(s) this PR fixes:
Fixes #2137
Checklist