feat(trace-utils)!: change buffer implementation by yannham · Pull Request #2055 · DataDog/libdatadog

yannham · 2026-05-28T17:35:10Z

What does this PR do?

This PR backports the change buffer implementation required for dd-trace-js adoption of native spans. Follow-up of #2046.

The change buffer is a contiguous region of memory where span operations are encoded from the JS runtime, similar to a byte code. The implementation decode all those span events and reconstruct the corresponding span objects.

Motivation

Required to land native spans dd-trace-js.

How to test the change?

tests included.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f874ddf2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-28T17:38:21Z

+        let num: u32 = self.change_buffer.read_unchecked(index);
+        self.string_table
+            .get_unchecked(num as usize)
+            .clone()
+            .unwrap_unchecked()


Keep FFI buffer decoding checked

When a cached non-create operation such as SetName or SetServiceName is decoded from the shared FFI buffer, the caller reaches this helper without validating either that the string id is still present in string_table or that the argument bytes are in bounds. An out-of-sync/evicted string id that used to produce ChangeBufferError::StringNotFound now hits unwrap_unchecked(), which is undefined behavior across the FFI-facing change-buffer path rather than a recoverable error.

Useful? React with 👍 / 👎.

github-actions · 2026-05-28T17:38:46Z

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

Base Branch: origin/main
PR Branch: origin/yannham/change-buffer-state

Summary by Rule

Rule	Base Branch	PR Branch	Change
unwrap_used	0	1	⚠️ +1 (N/A)
Total	0	1	⚠️ +1 (N/A)

Annotation Counts by File

File	Base Branch	PR Branch	Change
`libdd-trace-utils/src/change_buffer/mod.rs`	0	1	⚠️ +1 (N/A)

Annotation Stats by Crate

Crate	Base Branch	PR Branch	Change
`clippy-annotation-reporter`	5	5	No change (0%)
`datadog-ffe-ffi`	1	1	No change (0%)
`datadog-ipc`	21	21	No change (0%)
`datadog-live-debugger`	4	4	No change (0%)
`datadog-live-debugger-ffi`	10	10	No change (0%)
`datadog-profiling-replayer`	4	4	No change (0%)
`datadog-sidecar`	46	46	No change (0%)
`libdd-common`	13	13	No change (0%)
`libdd-common-ffi`	12	12	No change (0%)
`libdd-data-pipeline`	5	5	No change (0%)
`libdd-ddsketch`	2	2	No change (0%)
`libdd-dogstatsd-client`	1	1	No change (0%)
`libdd-profiling`	13	13	No change (0%)
`libdd-remote-config`	3	4	⚠️ +1 (+33.3%)
`libdd-telemetry`	20	20	No change (0%)
`libdd-tinybytes`	4	4	No change (0%)
`libdd-trace-normalization`	2	2	No change (0%)
`libdd-trace-obfuscation`	3	3	No change (0%)
`libdd-trace-stats`	1	1	No change (0%)
`libdd-trace-utils`	11	12	⚠️ +1 (+9.1%)
Total	181	183	⚠️ +2 (+1.1%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

codecov-commenter · 2026-05-28T17:51:25Z

Codecov Report

❌ Patch coverage is 0% with 578 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.47%. Comparing base (8431f28) to head (9f874dd).

Additional details and impacted files

@@                         Coverage Diff                          @@
##           yannham/change-buffer-foundation    #2055      +/-   ##
====================================================================
- Coverage                             73.03%   72.47%   -0.56%     
====================================================================
  Files                                   465      465              
  Lines                                 76881    77436     +555     
====================================================================
- Hits                                  56147    56122      -25     
- Misses                                20734    21314     +580

Components	Coverage Δ
libdd-crashtracker	`65.45% <ø> (+0.01%)`	⬆️
libdd-crashtracker-ffi	`38.78% <ø> (ø)`
libdd-alloc	`98.77% <ø> (ø)`
libdd-data-pipeline	`85.52% <ø> (ø)`
libdd-data-pipeline-ffi	`75.28% <ø> (ø)`
libdd-common	`79.89% <ø> (ø)`
libdd-common-ffi	`74.41% <ø> (ø)`
libdd-telemetry	`73.37% <ø> (-0.03%)`	⬇️
libdd-telemetry-ffi	`31.36% <ø> (ø)`
libdd-dogstatsd-client	`82.64% <ø> (ø)`
datadog-ipc	`74.75% <ø> (-1.47%)`	⬇️
libdd-profiling	`81.69% <ø> (-0.02%)`	⬇️
libdd-profiling-ffi	`64.79% <ø> (ø)`
libdd-sampling	`97.46% <ø> (ø)`
datadog-sidecar	`30.22% <ø> (ø)`
datdog-sidecar-ffi	`15.32% <ø> (ø)`
spawn-worker	`48.86% <ø> (ø)`
libdd-tinybytes	`93.80% <ø> (ø)`
libdd-trace-normalization	`81.71% <ø> (ø)`
libdd-trace-obfuscation	`87.30% <ø> (ø)`
libdd-trace-protobuf	`68.25% <ø> (ø)`
libdd-trace-utils	`82.89% <0.00%> (-5.41%)`	⬇️
libdd-tracer-flare	`86.88% <ø> (ø)`
libdd-log	`74.83% <ø> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

datadog-datadog-prod-us1-2 · 2026-05-28T17:52:10Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 4.03%
• Overall Coverage: 73.16% (-0.20%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: acc2888 | Docs | Datadog PR Page | Give us feedback!}

dd-octo-sts · 2026-05-28T18:12:58Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	7.70 MB	7.70 MB	0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	83.68 MB	83.68 MB	+0% (+5.29 KB) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	94.79 MB	94.78 MB	-0% (-3.00 KB) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.34 MB	10.34 MB	-0% (-96 B) 👌

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	24.82 MB	24.83 MB	+.06% (+16.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	86.89 KB	86.89 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	180.80 MB	180.84 MB	+.02% (+40.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	925.69 MB	925.68 MB	-0% (-12.17 KB) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.09 MB	8.09 MB	-0% (-512 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	86.89 KB	86.89 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	23.93 MB	23.93 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	47.78 MB	47.78 MB	-0% (-1.09 KB) 👌

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	21.52 MB	21.52 MB	+0% (+512 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	88.26 KB	88.26 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	184.92 MB	184.91 MB	-0% (-16.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	918.57 MB	918.35 MB	--.02% (-219.24 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.24 MB	6.24 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	88.26 KB	88.26 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	25.66 MB	25.66 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	45.41 MB	45.41 MB	+0% (+946 B) 👌

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	74.60 MB	74.60 MB	-0% (-2.61 KB) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.58 MB	8.58 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	90.02 MB	90.02 MB	+0% (+1.80 KB) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.44 MB	10.44 MB	+.04% (+4.36 KB) 🔍

yannham · 2026-06-05T12:53:18Z

I'm converting back to draft in order to take a closer look at unsafe code/unbound reads and see if we can avoid them.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f4e473edf5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Add the main ChangeBufferState<T> struct and its full implementation: span creation/pooling, change buffer interpretation (opcodes), deferred tag resolution, span header materialization, flush logic, and string table management. This is the core state machine that processes batched span mutations from JS/WASM via a shared memory buffer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

rochdev

There are a few issues with this implementation across the board which would be a bit difficult to explain so I just opened a PR on top of this PR instead: #2105

Feel free to reimplement based on the summary if that's easier, I thought it would be a smaller change when I started on it 😅

yannham · 2026-06-11T12:33:44Z

@rochdev would that be ok to just merge this one first (after addressing reviews) and then proceed with your PR? I don't expect to do a lot of modifications and the code isn't used anywhere yet anyway. That'll just make for an easier review experience. If yes, please lift the request for changes.

yannham requested review from a team as code owners May 28, 2026 17:35

github-actions Bot added the mini-agent label May 28, 2026

chatgpt-codex-connector Bot reviewed May 28, 2026

View reviewed changes

yannham requested a review from a team as a code owner June 2, 2026 08:48

yannham mentioned this pull request Jun 2, 2026

feat(native-spans)!: change buffer foundation #2046

Merged

yannham marked this pull request as draft June 5, 2026 12:52

paullegranddc force-pushed the yannham/change-buffer-foundation branch from 1c21a90 to 33e085c Compare June 8, 2026 11:40

paullegranddc force-pushed the yannham/change-buffer-state branch from 51de208 to bf5f3db Compare June 8, 2026 11:56

yannham force-pushed the yannham/change-buffer-foundation branch from 42a7a66 to 24c7734 Compare June 10, 2026 13:45

yannham force-pushed the yannham/change-buffer-state branch from bf5f3db to f4e473e Compare June 10, 2026 13:52

yannham marked this pull request as ready for review June 10, 2026 13:52

chatgpt-codex-connector Bot reviewed Jun 10, 2026

View reviewed changes

Comment thread libdd-trace-utils/src/change_buffer/mod.rs Outdated

Comment thread libdd-trace-utils/src/change_buffer/mod.rs Outdated

yannham force-pushed the yannham/change-buffer-state branch from 173ccbc to e0053c8 Compare June 10, 2026 13:57

Base automatically changed from yannham/change-buffer-foundation to main June 10, 2026 15:20

danielsn reviewed Jun 10, 2026

View reviewed changes

yannham and others added 8 commits June 10, 2026 17:53

feat: add more methods to VecSpan

8c8d1b0

refactor: Vec -> VecMap in change buffer (plus fixes)

e420e15

chore: various doc and small code improvements

b15439d

chore: remove unused rustc-hash dep

74620b7

doc: add safety comments and missing unsafe annotation

1aa9a8f

chore: remove dead code allow (impl complete)

71ea154

style: format

a4c5953

yannham force-pushed the yannham/change-buffer-state branch from 7658fe2 to a4c5953 Compare June 10, 2026 15:53

paullegranddc reviewed Jun 10, 2026

View reviewed changes

style: implement review suggestions

d111e21

rochdev requested changes Jun 11, 2026

View reviewed changes

rochdev approved these changes Jun 11, 2026

View reviewed changes

yannham added 6 commits June 11, 2026 16:09

refactor: remove slots

27d2770

refactor: specify buffer ptr non null, better split ownership of state

9fde79b

wip

c458071

refactor: get rid of deferred and other small caches

d802030

fix: do not increment counter unconditionally

0208be8

refactor: use a unique trace for one chunk

acc2888

paullegranddc approved these changes Jun 12, 2026

View reviewed changes

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: queued labels Jun 12, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit a21b946 into main Jun 12, 2026
89 checks passed

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Jun 12, 2026

gh-worker-dd-mergequeue-cf854d Bot deleted the yannham/change-buffer-state branch June 12, 2026 14:00

Conversation

yannham commented May 28, 2026

What does this PR do?

Motivation

How to test the change?

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Clippy Allow Annotation Report

Summary by Rule

Annotation Counts by File

Annotation Stats by Crate

About This Report

Uh oh!

codecov-commenter commented May 28, 2026

Codecov Report

Uh oh!

datadog-datadog-prod-us1-2 Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

yannham commented Jun 5, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rochdev left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yannham commented Jun 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions Bot commented May 28, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented May 28, 2026 •

edited

Loading

dd-octo-sts Bot commented May 28, 2026 •

edited

Loading

rochdev left a comment •

edited

Loading