Skip to content

Finalize V8 Memory Benchmarking script #30

Description

@SHAURYASANYAL3

🛑 The Problem

We claim our streaming architecture uses constant memory (O(1) space complexity) regardless of repository size. However, per Directive 002 (Reality Wins), we must prove this with hard evidence. We need an automated script that monitors the Node.js V8 engine's heap usage during a massive ingestion.

💡 The Solution

Write a Node.js profiling script that runs a test ingestion of a massive repo (like torvalds/linux or facebook/react) and logs the memory heap statistics every 500ms.

🛠️ Implementation Details

  1. Create scripts/benchmark-memory.js.
  2. Use Node's built-in v8 module: v8.getHeapStatistics().
  3. Trigger the Octokit streaming ingestion locally.
  4. Log the used_heap_size to a CSV file or console table.
  5. Assert that the used_heap_size never exceeds 300MB at any point during the hour-long sync.

✅ Acceptance Criteria

  • The script accurately measures V8 memory during network I/O.
  • The output proves the streaming architecture prevents Heap Out of Memory crashes.
  • Results are committed to docs/PRODUCT_TRUTH.md.

Ready to tackle this? Comment .take below to get automatically assigned!

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions