feat:(reproduce-drawer): add reproduce drawer frontend and benchmark_environment table with ingest#346
Open
rafaykhan-source wants to merge 4 commits into
Open
Conversation
Adds a 3-tab drawer (Command / Config JSON / Environment) that opens from scatter pinned tooltip, GPU graph tooltip, or inference table row, showing exactly how a benchmark was produced. - Pure `buildLaunchCommand(framework, config)` library with per-framework generators for vllm, sglang, trt (with the trtllm alias). Disagg configs emit two stitched commands (prefill / decode workers). Compound stacks (Dynamo, ATOM, MoRI) intentionally show a clear fallback that points at the Config JSON tab. - ReproduceDrawer reuses the existing right-side dialog pattern, with per- tab Copy buttons, a Server log link, and Esc + outside-click close that doesn't perturb chart zoom or URL state. - Drawer state lives in InferenceContext so it is reachable from scatter, GPU graph, and the table without prop-drilling. - Analytics: reproduce_drawer_opened, reproduce_copy (with framework), reproduce_drawer_open_clicked, reproduce_server_log_clicked, inference_table_reproduce_clicked. - Tests: 21 unit tests for buildLaunchCommand cover every framework, disagg, compound fallbacks, alias resolution, missing-field placeholders; 4 new tooltipUtils tests assert the Reproduce button only appears when pinned; new E2E spec exercises the table-row entry point and the unofficial-run overlay path. Closes SemiAnalysisAI#270 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: functionstackx <functionstackx@users.noreply.github.com>
|
@rafaykhan-source is attempting to deploy a commit to the SemiAnalysisAI Team on Vercel. A member of the Team first needs to authorize it. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reproduce Drawer
Issue Requirements
A drawer that opens from any benchmark point or row and shows exactly how to reproduce that number: the framework launch command, the full config JSON, and the environment it ran in (framework SHA, container tag, driver, CUDA, GPU SKU).
Requirements
Exists as a button in the tooltip, and rows of the data-table are now clickable, linking to the reproduce drawer.
There are three tabs with a shared copy button that will copy the contents of the active tab.
Reproduce Tooltip:
Launch Command:
Launch Command Fallback:
The pure function is entitled buildLaunchCommand.
Exists alongside the shared copy button for each tab:
Analytics were added and track open and copy events with framework as a property.
Additional Screenshots:
Config JSON Tab:
Framework Tab:
Additional Notes
The schema for benchmark_environments is as follows:
There are two methods for sourcing the information. One
env_jsonis reliant on a possible upstream change to CI to populate this information alongside server logs if possible, and the fallbacklog_parsetries to extract the environment info from server logs.If making the CI change is not something on the roadmap or worth doing, I can totally drop related fields and code predicated on it, and stick with making a new table that only extract what info can be extracted from server_logs.