Document AI Observability data retention policy#17129
Merged
Merged
Conversation
Added a section on data retention policies for AI Observability events, detailing the 30-day retention policy and what properties are removed after this period.
Contributor
Deploy preview
|
edwinyjlim
approved these changes
May 29, 2026
Member
edwinyjlim
left a comment
edwinyjlim
left a comment
There was a problem hiding this comment.
I'd consider adding this as a new Data retention page under Resources. It feels a little tacked on here under the basics page.
Move the AI Observability data retention content out of basics.mdx into a dedicated Resources page, and correct the mechanism: at ingestion each $ai_ event is split into a full copy in the ai_events table (deleted after 30 days) and a trimmed copy in the events table (never includes the large properties). - Remove the Data retention section from basics.mdx - Add contents/docs/ai-observability/data-retention.mdx - Add the page to the AI Observability > Resources nav Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
slshults
changed the title
Radu-Raicea
reviewed
May 29, 2026
| ## What this means for you | ||
|
|
||
| - To read the large properties, query the `ai_events` table – even for traces less than 30 days old. The `events` table never contains them. | ||
| - After 30 days, the large properties are gone. A trace older than 30 days still appears, but its generations show no input or output content. |
Member
There was a problem hiding this comment.
They're gone because we drop the full row in ai_events. That's how we efficiently get rid of the large properties.
slshults
commented
May 29, 2026
Member
Author
slshults
left a comment
slshults
left a comment
There was a problem hiding this comment.
applying change based on Radu's comment
Applying change based on Radu's comment
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents the 30-day data retention policy for AI Observability
$ai_events as a standalone Data retention page under Resources (moved out of the Basics page per review feedback).At ingestion, each
$ai_event is split into two copies:ai_eventstable (deleted after 30 days).eventstable (kept like any other Product Analytics event).To read the large properties (
$ai_input,$ai_output,$ai_output_choices,$ai_input_state,$ai_output_state,$ai_tools), query theai_eventstable — even for traces under 30 days old.Changes
basics.mdxcontents/docs/ai-observability/data-retention.mdx🤖 Generated with Claude Code