Skip to content

Feat/usage stats: First iteration of usage stats#303

Merged
St4NNi merged 8 commits into
mainfrom
feat/usage-stats
Jul 3, 2026
Merged

Feat/usage stats: First iteration of usage stats#303
St4NNi merged 8 commits into
mainfrom
feat/usage-stats

Conversation

@St4NNi

@St4NNi St4NNi commented Jul 3, 2026

Copy link
Copy Markdown
Member

Adds maintained usage accounting for buckets, objects, stored blobs, stored bytes, and logical bytes. Counters are updated atomically in the S3 write transactions that change storage state, exposed through node and group usage endpoints, and rebuilt at startup for stores that predate the new counter keyspace.

Changes

  • Adds a usage_stats keyspace with serialized UsageCounters and UsageDelta structs.
  • Stores per-group usage counters and sharded global counters to avoid a single global hot key.
  • Adds counter overflow and underflow checks so invalid deltas fail without partially mutating counters.
  • Adds an embeddable UsageCounterUpdate operation that read-modify-writes counters inside the host transaction.
  • Updates S3 bucket creation and deletion paths to maintain bucket counters.
  • Updates object put, multipart completion, and object deletion paths to maintain object and logical-byte counters.
  • Tracks global physical storage usage through stored blob and stored byte counters while keeping per-group quota basis on logical bytes.
  • Adds RebuildUsageStatsOperation to recompute counters from bucket, blob-location, object-head, and object-version keyspaces.
  • Removes stale usage counter keys during rebuild, including the legacy unsharded global counter.
  • Runs a startup rebuild when the usage counter keyspace is missing, which backfills existing stores once before serving requests.
  • Adds GET /api/v1/info/usage for node aggregate usage.
  • Adds GET /api/v1/groups/{id}/usage for authenticated group members to read group usage.
  • Adds OpenAPI schemas for the usage responses and endpoints.
  • Keeps global usage reads compatible with the legacy global counter if no sharded counters exist.

GitHub Issues

  • Partially addresses [feat] Usage accounting and quota configuration #249 by adding the usage keyspace, transactional counter updates for bucket/object write paths, startup backfill, and usage reads. This PR does not add quota configuration, realm-replicated quota policy, admin quota endpoints, or sync-layer distribution.
  • Partially addresses [feat] Quota enforcement and reporting #250 by adding the current usage reporting endpoint and recount/rebuild foundation. This PR does not add quota enforcement, warning notifications, history snapshots, portal UI, or periodic drift telemetry.

Notes

  • Replication ingest is intentionally not wired to usage counter updates so accepted writes are counted only once on the node that owns the write transaction.
  • stored_blobs and stored_bytes describe physical content-addressed blobs and are meaningful for global node usage.
  • logical_bytes sums materialized version sizes and is the per-group quota accounting basis.
  • Startup rebuild only runs when the first global shard is missing; future repair or periodic recount scheduling remains follow-up work.

@das-Abroxas das-Abroxas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to have persistent usage accounting that makes storage consumption visible at both the node and group level. It will definitely be needed sooner rather than later, for example, when it comes to enforcing quotas. Approved.

@St4NNi St4NNi merged commit bbde808 into main Jul 3, 2026
6 checks passed
@St4NNi St4NNi deleted the feat/usage-stats branch July 3, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants