0.3.70: adopt ciris-server v0.5.204 — the substrate fix for the scans we measured - #80
Merged
Merged
Conversation
… we measured persist v42.0.0 → v42.1.0, edge v21.0.0 → v21.1.0. Two of the changes in it are the upstream answers to what this repo has been measuring for a week: **#572 — dimension reads are index-served.** V137 indexes V106's generated `dimension` column, so a prefix filter compiles to a RANGE on it instead of `json_extract(attestation_envelope, '$.dimension') LIKE ?` evaluated per row. That is the full-table scan behind the roster rebuild (moved to `list_scores` in 0.3.61 to dodge it) and behind every config read. **#570 — the config snapshot cache is one slot per engine, invalidated per engine.** That is CIRISServer#557: `get_config` called `live_config_rows` per key with no cache, so `Config::resolve` cost ~50 scans of everything this node had ever authored, and got worse with every observation emitted. Our `Snapshot` batching (0.3.68) stays. It is no longer load-bearing against an uncached substrate, but one fetch per resolve is still one fetch, it is correct either way, and it keeps the consumer honest if the cache is ever invalidated more eagerly than expected. Belt and braces, not a workaround left behind. What this predicts, so it can be checked rather than assumed: steady-state `pread64` should fall well below the 6,700/s measured on 0.3.68 at 20 minutes uptime (down from 13,400/s on 0.3.66), and the twice-a-minute CPU bursts should shrink again. What it does NOT predict is any change to `fordblks` — 561MB of free list against 47.7MB live survived the config churn being removed, so whatever built it is not these scans. 116 tests, fmt, clippy -D warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
persist v42.0.0 → v42.1.0, edge v21.0.0 → v21.1.0. Two changes in it are the upstream answers to what this repo has spent a week measuring.
#572 — dimension reads are index-served. V137 indexes V106's generated
dimensioncolumn, so a prefix filter compiles to a range on it instead ofjson_extract(attestation_envelope, '$.dimension') LIKE ?evaluated per row. That's the full-table scan behind the roster rebuild (which 0.3.61 dodged by moving tolist_scores) and behind every config read.#570 — per-engine config snapshot cache. That's CIRISServer#557:
get_configcalledlive_config_rowsper key with no cache, soConfig::resolvecost ~50 scans of everything this node had ever authored — and degraded with every observation emitted, since those rows share the attester key the scan seeks on.Our batching stays
Snapshot(0.3.68) is no longer load-bearing against an uncached substrate, but one fetch per resolve is still one fetch, it's correct either way, and it keeps this consumer honest if the cache is ever invalidated more eagerly than expected. Belt and braces rather than a workaround left lying around.What this predicts
Stated so it can be checked rather than assumed:
pread64should fall well below the 6,700/s measured on 0.3.68 at 20 minutes uptime (itself down from 13,400/s on 0.3.66)And what it does not predict: any change to
fordblks. 561 MB of free list against 47.7 MB live survived the config churn being removed, so whatever built it is not these scans.116 tests, fmt, clippy
-D warnings.🤖 Generated with Claude Code