Skip to content

Optimize unlimited broadcast performance#623

Merged
copybara-service[bot] merged 1 commit into
masterfrom
cl/938056969
Jul 3, 2026
Merged

Optimize unlimited broadcast performance#623
copybara-service[bot] merged 1 commit into
masterfrom
cl/938056969

Conversation

@copybara-service

Copy link
Copy Markdown

Optimize unlimited broadcast performance

SaveBroadcastMessage (called when a client is about to receive a broadcast message) tends to fail due to context cancelation (timeout) in large broadcasts. That's because broadcast messages to clients are processed sequentially using a mutex to prevent parallel writes to the task's broadcast allocation, which would lead to various issues depending on the db implementation.

Context: An allocation is a temporary, local quota of the total broadcast limit assigned to a specific task. On startup, tasks create an allocation of an active broadcast (usually 20% of the remaining messages), which allows it to send broadcast messages to clients without syncing with other tasks on how many messages are still remaining.

For broadcasts with no limit (db.BroadcastUnlimited), tasks generally don't need to care how many messages others have sent, and we can therefore skip the allocation mechanism and process messages in parallel.

With this change:

  • Unlimited broadcasts (Limit == db.BroadcastUnlimited) skip database allocation creation.
  • Senders call SaveBroadcastMessage with an empty AllocationID for unlimited broadcasts.
  • Datastores (SQLite, MySQL, Spanner) skip updating broadcast_allocations if the AllocationID is empty.
  • The Manager skips acquiring the in-memory i.lock for unlimited broadcasts, allowing concurrent processing.
  • The sent count is tracked in-memory and flushed to broadcasts.sent when the manager is closed or the broadcast is cleaned up.

@copybara-service copybara-service Bot added the exported Exported change from internal source label Jul 3, 2026
@copybara-service copybara-service Bot force-pushed the cl/938056969 branch 4 times, most recently from 5847a64 to 946c13a Compare July 3, 2026 16:46
SaveBroadcastMessage (called when a client is about to receive a broadcast message) tends to fail due to context cancelation (timeout) in large broadcasts. That's because broadcast messages to clients are processed sequentially using a mutex to prevent parallel writes to the task's broadcast allocation, which would lead to various issues depending on the db implementation.

*Context: An allocation is a temporary, local quota of the total broadcast limit assigned to a specific task. On startup, tasks create an allocation of an active broadcast (usually 20% of the remaining messages), which allows it to send broadcast messages to clients without syncing with other tasks on how many messages are still remaining.*

For broadcasts with no limit (`db.BroadcastUnlimited`), tasks generally don't need to care how many messages others have sent, and we can therefore skip the allocation mechanism and process messages in parallel.

With this change:
- Unlimited broadcasts (`Limit == db.BroadcastUnlimited`) skip database allocation creation.
- Senders call `SaveBroadcastMessage` with an empty `AllocationID` for unlimited broadcasts.
- Datastores (SQLite, MySQL, Spanner) skip updating `broadcast_allocations` if the `AllocationID` is empty.
- The `Manager` skips acquiring the in-memory `i.lock` for unlimited broadcasts, allowing concurrent processing.
- The `sent` count is tracked in-memory and flushed to `broadcasts.sent` when the manager is closed or the broadcast is cleaned up.

PiperOrigin-RevId: 942179136
@copybara-service copybara-service Bot merged commit b33bdde into master Jul 3, 2026
@copybara-service copybara-service Bot deleted the cl/938056969 branch July 3, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

exported Exported change from internal source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants