Fix: dynamic memory guard to prevent OOM on large queries#56
Merged
Conversation
- Always apply LIMIT to SELECT/SHOW/DESCRIBE/EXPLAIN (was conditional) - Hard cap at 50,000 rows regardless of user settings - Fix rows_truncated detection to compare against capped_limit - Lower frontend virtualization threshold from 500 to 100 rows - Add truncation warning banner when results are capped Closes #25 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
- Remove hardcoded 50k row cap - Add sysinfo crate for process memory monitoring - MemoryGuard checks RSS every 1000 rows during fetch - Triggers when process hits 75% of system RAM - Gracefully stops fetch and returns accumulated rows with warning - Add OutOfMemory error variant to CoreError - Update frontend to show backend warnings - Truncation banner no longer mentions arbitrary 50k number
EVWorth
changed the title
- Collapse nested if into single condition (clippy::collapsible_if) - Use is_multiple_of(1000) instead of % 1000 == 0 (clippy::manual_is_multiple_of) - Run cargo fmt for consistent formatting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Loading large tables causes out-of-memory crashes. Query results flow through 3 memory copies: Rust Vec -> JSON IPC -> Zustand store -> useMemo object conversion. No streaming or pagination exists.
Solution
Changes
Rust (mas-core/query/executor.rs)
sysinfocrate for process memory monitoringMemoryGuardstruct: tracks RSS, triggers at 75% system RAM thresholdfetch_manystreamwarningsandrows_truncated: trueRust (mas-core/error.rs)
OutOfMemory(String)error variantFrontend (ResultsGrid.tsx)
warnings[]array from backend resultsTesting
cargo check -p mas-core- compiles cleanFuture Work
Closes #25