fix(admin): paginate admin_list_tokens to avoid 512 KB truncation - #36
Open
aelttil wants to merge 1 commit into
Open
fix(admin): paginate admin_list_tokens to avoid 512 KB truncation#36aelttil wants to merge 1 commit into
aelttil wants to merge 1 commit into
Conversation
Without pagination, admin_list_tokens loads the entire token registry
from S3 and returns it in a single JSON response. With large deployments
(hundreds of tokens, each with long space_ids lists and full SHA-256
hashes), this easily exceeds the 512 KB ResponseLimitMiddleware threshold,
causing the response to be replaced by {"_truncated": true, ...}.
Add `limit` (default 100, max 1000) and `offset` (default 0) parameters
to both admin_list_tokens (tool) and TokenService.list_tokens (core).
The response now always includes `total` (count after filters, before
pagination), `limit`, `offset`, and `has_more` so callers can iterate
through large registries page by page.
Rétrocompat : default limit=100 covers all existing small-scale usages
without any call-site changes. The `total` field semantics are unchanged
(still reflects the filtered count), only the returned `tokens` slice is
now bounded.
Fixes #XX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Without pagination, admin_list_tokens loads the entire token registry from S3 and returns it in a single JSON response. With large deployments (hundreds of tokens, each with long space_ids lists and full SHA-256 hashes), this easily exceeds the 512 KB ResponseLimitMiddleware threshold, causing the response to be replaced by {"_truncated": true, ...}.
Add
limit(default 100, max 1000) andoffset(default 0) parameters to both admin_list_tokens (tool) and TokenService.list_tokens (core).The response now always includes
total(count after filters, before pagination),limit,offset, andhas_moreso callers can iterate through large registries page by page.Rétrocompat : default limit=100 covers all existing small-scale usages without any call-site changes. The
totalfield semantics are unchanged (still reflects the filtered count), only the returnedtokensslice is now bounded.Fixes #XX