MILAB-6496: Implement the Feature Integration block#2
Draft
PaulNewling wants to merge 75 commits into
Draft
Conversation
Workflow, model, UI, and per-cell-metrics Python for the feature-barcode -> per-cell feature pipeline (mitool parse -> refine-tags -> tag-stat -u -> per-cell metrics; A-0010 contract p-columns exported for VDJ Multiomic Integration). See changeset. Integration test (test/src/wf.test.ts): samples-and-data FASTQ chain + uploaded tag->feature CSV. The empty-inputs case runs; the end-to-end case is skipped pending mitool #84 -- the per-sample mitool exec runs from a dev-only local override the prebuilt local backend cannot stage into the exec workdir. Two fixes surfaced by the live run: build the processColumn 'extra' map conditionally (an undefined 'control' value created a per-sample body input that never resolved and stalled the body); add prerun.tpl + wf.setPreRun to import/export the CSV during staging, driving its upload before production needs it.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements the feature-barcode workflow, integrating a mitool pipeline (parse -> refine-tags -> tag-stat -u) with Python-based per-cell metrics to process single-cell feature-barcode FASTQs. It introduces configurable read geometry and supports user-uploaded tag-to-feature CSVs, updating the model, UI, and workflow orchestration accordingly. The review feedback highlights several critical Tengo-specific runtime issues that must be addressed: array comparisons using == check for reference identity rather than structural equality, and direct integer-to-string conversions via string(int) yield Unicode code points instead of decimal representations, which will corrupt the generated tag patterns, memory limits, and dominance thresholds. Additionally, task-level resource variables need to be explicitly passed to the pipeline template inputs, and read geometry inputs should be defensively clamped to a minimum of 1 in the model.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Comment on lines
+73
to
+78
| extraInputs := { | ||
| pattern: pattern, | ||
| tagsCsv: csvFile, | ||
| dominanceThreshold: dominanceThreshold, | ||
| fileExtension: fileExtension | ||
| } |
There was a problem hiding this comment.
The mitoolMemGB and mitoolCPUs variables are currently defined in metaExtra but are not passed to the fb-pipeline.tpl.tengo template's inputs map. In @platforma-sdk/workflow-tengo, metaExtra only configures task-level resources and does not automatically inject these values into the template's inputs. This will cause inputs.mitoolMemGB and inputs.mitoolCPUs to be undefined inside fb-pipeline.tpl.tengo, leading to runtime failures.
To fix this, pass them explicitly in extraInputs.
extraInputs := {
pattern: pattern,
tagsCsv: csvFile,
dominanceThreshold: dominanceThreshold,
fileExtension: fileExtension,
mitoolMemGB: is_undefined(args.perProcessMemGB) ? defaultMitoolMemGB : args.perProcessMemGB,
mitoolCPUs: is_undefined(args.perProcessCPUs) ? defaultMitoolCPUs : args.perProcessCPUs
}
| inputArg := "" | ||
| if meta.keyLength == 1 { | ||
| ll.assert( | ||
| aggregationAxesNames == ["pl7.app/sequencing/readIndex"], |
There was a problem hiding this comment.
In Tengo, the == operator on arrays checks for reference identity rather than structural/value equality. Comparing aggregationAxesNames == ["pl7.app/sequencing/readIndex"] will always evaluate to false even if the array contains the correct element, causing this assertion to always fail at runtime.
Instead, compare the array length and elements individually.
len(aggregationAxesNames) == 1 && aggregationAxesNames[0] == "pl7.app/sequencing/readIndex",
| inputArg = "input_{{R}}." + fileExtension | ||
| } else if meta.keyLength == 2 { | ||
| ll.assert( | ||
| aggregationAxesNames == ["pl7.app/sequencing/lane", "pl7.app/sequencing/readIndex"], |
There was a problem hiding this comment.
Similar to the previous assertion, array comparison using == in Tengo checks for reference identity. This assertion will always fail at runtime.
Compare the array length and elements individually instead.
len(aggregationAxesNames) == 2 && aggregationAxesNames[0] == "pl7.app/sequencing/lane" && aggregationAxesNames[1] == "pl7.app/sequencing/readIndex",
Comment on lines
+26
to
+28
| cell := "(" + CELL_TAG + ":N{" + string(geom.cellLen) + "})" | ||
| umi := "(" + UMI_TAG + ":N{" + string(geom.umiLen) + "})" | ||
| feat := "(" + FEATURE_TAG + ":N{" + string(geom.featureLen) + "})" |
There was a problem hiding this comment.
In Tengo, calling string(int) converts the integer to a single-character string of the corresponding Unicode code point (e.g., string(16) returns a control character, not "16"). This will result in a corrupted tag pattern being passed to mitool.
To safely convert these integers to their decimal string representations, cast them to floats first using string(float(v)).
cell := "(" + CELL_TAG + ":N{" + string(float(geom.cellLen)) + "})"
umi := "(" + UMI_TAG + ":N{" + string(float(geom.umiLen)) + "})"
feat := "(" + FEATURE_TAG + ":N{" + string(float(geom.featureLen)) + "})"
Comment on lines
+37
to
+39
| cellLen: data.cellLen ?? 16, | ||
| umiLen: data.umiLen ?? 10, | ||
| featureLen: data.featureLen ?? 15, |
There was a problem hiding this comment.
To prevent invalid read geometry values (such as 0 or negative numbers) from being passed to the workflow and causing mitool to crash, defensively clamp these values to a minimum of 1.
Suggested change
| cellLen: data.cellLen ?? 16, | |
| umiLen: data.umiLen ?? 10, | |
| featureLen: data.featureLen ?? 15, | |
| cellLen: Math.max(1, data.cellLen ?? 16), | |
| umiLen: Math.max(1, data.umiLen ?? 10), | |
| featureLen: Math.max(1, data.featureLen ?? 15), |
…ed mitool Switch the feature-barcode pipeline from the CELLFB cell-tag workaround to mitool's first-class FEATURE tag type (published mitool 2.3.1-129-main, PR milaboratory/mitool#86): - tag-pattern: FEATURE_TAG "CELLFB" -> "FEATURE" - refine-tags reordered to -t CELL -t FEATURE -t UMI, so FEATURE is corrected within each cell and UMI is deduped within (cell, feature) - pin software-mitool to published 2.3.1-129-main; drop the dev-only override Fold in the block-tools structure-refresh SDK upgrade (block-tools 2.11.6, model/ui 1.79.20, test 1.79.23, tengo-builder 4.0.11): - block restructured to block/src/ (index/AGENTS/agents-extra/block-extra) - add typescript ~5.9.3 catalog entry (refresh emitted a dangling catalog: ref) - update e2e import to FeatureIntegrationBlockPointer; un-skip the e2e - remove stray workflow vitest test script
…integration Populate the negative-control dropdown from a new staging emit-features step (controlOptions was an empty stub), plus observability and robustness improvements reviewed against recent blocks: - tag-stat -u runs with --use-local-temp; mitool parse/refine memory sized from the input read blob size via memFormula (clamped, metaExtra floor fallback) - QC page: per-step mitool/Python logs (PlLogView) + raw tag-stat QC table; isRunning spinner signal - results table now retentive + withStatus (no flicker); export column/axis specs moved to column-specs.lib.tengo with standard abundance annotations - client-side feature-count preview on the CSV input; dynamic subtitle - per-cell-metrics: join diagnostics + header-only specificity CSV when no control is set - test/.oxfmtrc.json ignores gitignored test runtime scratch (work, .test_auth.json) build:dev green 10/10. Live e2e still pending the local backend blob-staging blocker.
- mitool parse/refine now use mixcr semantics: an explicit per-process memory override (metaExtra) is a hard fixed request; otherwise the read-blob-size memFormula. main.tpl passes the base floor always and adds the override key only when set, so an overridden run doesn't dedup with the formula-sized one. - TODO on parse_report/refine_report: document why they're saveFile'd but unconsumed, and that they should be dropped if no reports UI is built. build:dev green 10/10.
Add an optional, chemistry-selected cell-barcode whitelist knob to Feature Integration. refine-tags corrects the cell barcode de-novo by default (unchanged); when a 10x whitelist is selected it snaps CELL to that mitool built-in (#builtin:<name>), so emitted pl7.app/sc/cellId strings match the VDJ producer (mixcr-clonotyping) by construction — making the downstream VDJ Multiomic Integration join deterministic instead of ~99% probabilistic. Default "" = de-novo, so existing behaviour and non-10x/synthetic inputs are unchanged; downstream blocks and the FEATURE panel correction are untouched. - model: cellWhitelist arg (default "") + optional BlockData field - workflow: thread as always-defined string; conditional -t CELL#builtin:<name> on refine-tags only (tag-stat / --cell-col untouched) - ui: "Cell barcode whitelist (10x)" dropdown in Advanced Settings; whitelist names/chemistries sourced from 10x via mitool built-ins (commented)
…deadlock The D4 column-mapping dropdowns (csvColumnOptions/controlOptions) are populated by the prerun reading the uploaded CSV and are required by args(), but the CSV upload was driven only from the main render (getImportProgress on ctx.outputs), which is unreachable until args() passes. Result: nothing uploaded the CSV pre-Run, emit-columns never ran, dropdowns stayed empty, args() kept throwing, Run stayed disabled — a circular dependency. Expose the CSV import handle from the prerun and add a ctx.prerun getImportProgress driver (isActive) mirroring samples-and-data, so the CSV uploads during staging. Dropdowns then populate, args() passes, and Run enables.
The QC summary was an Xsv processColumn output with axes: [] (a per-sample scalar), emitted in the same processColumn as the A-0010 contract columns. xsv import cannot produce a 0-within-axis column, so the empty-axes spec tripped xsv.importFile's partitionKeyLength < len(axes) assertion and crashed the whole shared render, taking perCellTable/tagstatQc down with it. Collect the per-sample QC CSV as a [sampleId] file map (type: Resource) instead of importing inline (also fixes a wrong body path: qcSummary -> qc), then concatenate the per-sample one-row CSVs (injecting the real sampleId from the map key) and import once, keyed [sampleId], in a new qc-summary.tpl child template. Mirrors mixcr-clonotyping's QC report assembly. The 8 metric columns keep their types/labels/formats.
createPlDataTableV3 cannot render these self-contained, non-batch processColumn frames: the object (scoped-sources) form returns undefined for every anchor/maxHops config, and the array-columns form runs discoverLabelColumnVariants over the whole result pool and hangs on the upstream Samples & Data FASTQ File-dataset. createPlDataTableV2(ctx, pCols, state) takes the columns directly and renders the mixed-granularity join (umiCount/fraction per [sampleId,cellId,featureId] + consensusFeature broadcast per [sampleId,cellId]), the pattern peptide-extraction uses. No spec/data change. qcSummaryTable stays on V3.
… banner, Logs slide-over, QC table render) - Cell-barcode whitelist help text -> #tooltip slot - 'Feature-barcode FASTQ' -> 'Select dataset' - remove the 'N features detected' hint - no-negative-control banner is now dismissable (closable), persisted via a controlInfoDismissed UI-only data field - pipeline logs moved off the QC page into a Logs slide-over (button by Settings) - QC page: wrap each stacked PlAgDataTableV2 in a bounded-height container so both render (the component is height:100% and collapsed without a bounded parent)
Each QC table is now its own full-height single-table page (the standard one-table-per-page pattern), replacing the single QC page that stacked both tables in bounded-height wrappers.
…ank) _refine_assigned_fraction was a stub returning None, so the QC summary's panelAssignedFraction column was always empty. Read the FEATURE refine step's outputCount/inputCount (fraction of reads kept after correcting the feature barcode against the panel whitelist). Falls back to blank when no refine report / no FEATURE step / zero input reads. Adds behavioral tests (parametrized fraction cases + no-FEATURE-step blank). Also syncs uv.lock to the pyproject scipy 1.15.3 pin (the lock was stale and would fail CI's uv sync --locked).
Subtitle now reads data.defaultBlockLabel (static fallback otherwise), mirrored by a UI watchEffect from a new suggestedBlockLabel model output: a dynamic '<dataset> · <barcode> → <feature>' string from the selected FASTQ dataset (resolved from the result pool by ref), the barcode-sequence column, and the feature-name column. The derivation lives in the output because the subtitle render context has no result pool (touching it renders 'Invalid subtitle'). Replaces the control-feature-only subtitle.
An empty (cell, feature) join (all reads off-panel, or a header-only tag-stat) crashed the per-sample run: consensus/specificity built from empty row-lists gave a schema-less frame whose .sort() raised ColumnNotFoundError, and a header-only tag-stat made polars infer String columns, breaking the fraction division. Build the frames with explicit schemas and coerce the UMI-count column to numeric on read, so empty input writes header-only CSVs. Adds a parametrized regression test over both empty-input modes.
Read the prerun feature/column lists with getDataAsJsonOrUndefined instead of getDataAsJson (the latter throws while staging is still computing), and reject in args() when the barcode-sequence and feature-name columns map to the same CSV column (previously only caught by the Python after the full mitool chain ran).
…earable no-control banner Remove the cell-barcode whitelist selector from the UI (not spec-required; de-novo already yields the ~99% cross-block join; only 5' v2 was verified and the other options carried footguns). The #builtin plumbing stays dormant as a documented seam (see docs/cell-whitelist-correction-plan.md). Also fix the no-negative-control info banner, which was never clearable: PlAlert's prop is 'closeable' (not 'closable') and it emits update:modelValue (not 'close'), so neither the close button nor the dismiss handler worked.
- init job runs-on: hz-ubuntu-dind - add gha-runner-label: hz-ubuntu-dind to node-simple-pnpm - add HZ_CI_TURBO_S3_* and HZ_CI_CACHE_S3_* to secrets.env
…Formula) The two input-sized steps used a fixed 8 GiB request that couldn't grow with the data, so a large sample could OOM them regardless of node capacity. Size both from input blob (tag-stat by refined.mic, per-cell-metrics by the tag-stat TSV it reads into polars), base 8 GiB + input x multiplier, clamped to 128 GiB, mirroring parse/refine. Falls back to the floor where formulas can't be evaluated; the Advanced per-process override still scopes to parse/refine only.
Replace the per-cell (consensus) and per-(cell,feature) (specificity, via iter_rows) Python loops with vectorized polars/numpy column ops. Removes the Python-object row copy that dominated memory on large samples and speeds it up (1.2M rows in ~0.9s / ~0.6 GiB peak). Output is byte-identical, guarded by the golden consensus test plus new oracle tests that cross-check both vectorized paths against the pure consensus_category/specificity_score rules. Empty input stays header-only via polars schema-preservation.
Replace the Main page's static "run the block" placeholder with a live
per-sample progress list while a run is in progress. Each sample appears as
soon as the roster is enumerated ("Queued"), shows live mitool parse progress
(stage + % + ETA) in a PlProgressCell, falls back to an indeterminate
"Processing..." while the downstream steps run, then flips to "Done".
- workflow: surface the per-sample parse step's stdout as a flat parseLogStream
Log column; use a shared [==PROGRESS==] MI_PROGRESS_PREFIX sentinel matched by
the model (the previous "parse" value was unused).
- model: new outputs started, sampleProgress (roster + live progress line per
sample, gated on getInputsLocked), completedSamples (from qcJson), and
sampleLabels; plus ProgressPrefix/ProgressPattern.
- ui: parseProgress.ts + results.ts derive the per-sample rows; MainPage renders
a stack of PlProgressCell (no new dependencies). Modeled on peptide-extraction.
Docs: docs/progress-loading-screen-plan.md. Changeset appended to the v1 entry.
'Graph' read as an unnamed/placeholder tab (it's graph-maker's default). Name the tab and the plot's default title after what the violin shows — the per-cell feature fraction distribution — matching this codebase's '<metric> Distribution' tab idiom.
…ting Loading screen (extends the per-sample progress list): - Show each sample's current mitool step by name with live percent + ETA (Parsing reads -> Refining barcodes -> Counting UMIs) instead of a flat "Processing...". The workflow emits a per-sample x per-step stepLogs Log map (parse / refine-tags / tag-stat, each with the shared [==PROGRESS==] sentinel); the model exposes stepProgress; results.ts picks each sample's latest active step. - Gate the progress list on isRunning: perCellTable is a withStatus output, so its envelope is truthy during the run and would otherwise win the v-if and show the generic table overlay. The results table shows once the run finishes. - Drop the non-idiomatic "Starting run..." info alert from the running state. Feature breakdown column: - Rename "Features" -> "Feature breakdown". - Reformat the per-feature list to "feature (fraction%, umiCount UMI)", bullet-separated with non-breaking padding, sorted dominant-first, with "<1%" for a nonzero feature that rounds below 1%. Display-only; the exported per-feature matrix (A-0010 contract) is unchanged.
Checkpoint of the test-data reorg: split into fixtures/ (committed automated-test data) and manual/ (recipe-only manual beds); remove the previous generator/README/data layout; update conftest.py and .gitignore for the new structure.
Checkpoint before the V3 results-table migration.
- Replace the PlProgressCell stack with an in-memory AgGridVue table (row-number,
Sample, Progress columns; PlAgOverlayLoading covers the pre-roster window) —
the blocks/peptide-extraction pattern, no custom CSS. Adds ag-grid-vue3 /
ag-grid-enterprise (catalog + lockfile).
- results.ts emits the structured progress cell {status, percent, text, suffix};
a finished-but-not-done step shows a full bar + "Done" suffix with the step name.
refine-tags runs several internal passes (CELL / FEATURE / UMI / writing), each counting 0→100%, so its live percent jumped up and down in the progress grid. Show that step as an indeterminate (animated) bar with a blank right-hand note instead — the progress cell otherwise defaults that note to "0%". Parse and tag-stat keep their live percent, and each step still flips to a full bar + "Done" when it finishes.
… page Match MiXCR Clonotyping's progress-only view — the block no longer surfaces result data in its own UI. - Main page: always show the per-sample progress grid; when the run finishes, every row settles into its "Done" state instead of swapping to the results table. Overlay uses not-ready/running like MiXCR. - sections(): comment out the "Raw tag-stat" and "Feature Fraction Distribution" (Graph) tabs; keep "Main" + "Per-sample QC". - All hidden code left in place (results-table branch + imports, both tab pages, their model outputs, and the app.ts routes) so a tab is re-enabled by uncommenting one line. Display/UI only — the A-0010 export contract to VDJ Multiomic Integration is unaffected.
tag-stat -u makes several non-monotonic passes over the on-disk sort, so its live percent jumps up and down. Add "3-tagstat" to INDETERMINATE_STEPS so the "Counting UMIs" step renders an animated bar with no percentage, matching the existing "Refining barcodes" (refine-tags) handling.
Drive the per-sample progress cell from mitool's structured stage labels instead of a single jumpy percent: - Refining barcodes: indeterminate bar, but the text names the tag being corrected and its position (CELL -> FEATURE -> UMI), e.g. "Refining feature barcodes / 2 of 3". Per-tag percent is non-monotonic (recursive correction passes), so no percentage is shown. - Counting UMIs: split into "sorting" (indeterminate, the data-dependent on-disk sort) and "writing N%" (a real monotonic bar for the final write pass). - Parsing reads: unchanged monotonic percent. Pure UI logic — mitool already emits these stage labels; no workflow change. The refine tag list is pinned in results.ts in sync with fb-pipeline.tpl.tengo.
Keep the refine step's left label anchored on "Refining barcodes" and vary only the colon-suffix so the text doesn't jump between sub-steps: Refining barcodes (Initialization, no suffix) Refining barcodes: Cell barcodes (1 of 3) Refining barcodes: Feature barcodes (2 of 3) Refining barcodes: UMIs (3 of 3) Refining barcodes: Finalizing (Filtering / Final sorting / Writing, no suffix) The bare lead-in is detected via mitool's "Initialization" stage; all other non-tag stages collapse to ": Finalizing".
Mirror MiXCR / peptide-extraction: two per-sample QC columns that fill in as each sample finishes (blank while running). - Quality: an OK/WARN/ALERT status tag (PlAgCellStatusTag), derived in the UI from the per-sample QC metrics — ALERT if 0 cells detected or panel-assigned <25%; WARN if panel-assigned <50% or matched <80%; else OK. Cutoffs are in one place in results.ts for easy tuning. - Read recovery: a compact stacked bar (PlAgChartStackedBarCell) splitting each sample's reads into Usable (matched + panel-assigned) / Off-panel / No pattern match. Falls back to Usable + No-match when no refine report is available. Model exposes per-sample QC as a new `sampleQc` output (keyed by sampleId, same qcJson source as completedSamples); QcRow is now exported. Pure derivation in the UI, no workflow change.
…ator A `degraded` scenario for MANUAL QC-visualization: four samples tuned to different matched / panel-assigned levels so the block's Quality tag and Read recovery bar show a full spread (OK / WARN / WARN / ALERT; green / orange / red). Unlike the other scenarios (behavioral assertions), this one exists to exercise the QC columns, not to hold counts fixed. Two new perturbations — convert_offpanel (rewrite a fraction of feature barcodes off-panel → lowers panel-assigned) and add_malformed (append unparseable reads → lowers matched) — plus generate_degraded() and a `degraded` --scenario choice.
- Per-cell results: the per-cell table returns as its own tab after Per-sample QC (it was the old Main page). Main stays the per-sample progress grid. - Remove the Graph (violin) and Raw tag-stat views end to end: UI pages + routes, model outputs (pf / pfPcols / tagstatQcTable) + their UI-state fields, and the workflow outputs (graphPf, the tag-stat QC frame + its column/axis specs + the fb-pipeline tagstat body output). The block now surfaces only the per-cell results table and per-sample QC. - Hide the Per-sample QC and Per-cell results tabs until the block has run (sections gate on ctx.outputs); a fresh block shows only Main. The A-0010 export (perCellFeatures) is unchanged. No changeset change — the model/ui/workflow packages are already major in the consolidated v1 changeset.
The Graph (violin) view was removed, so @milaboratories/graph-maker is no longer imported anywhere. Remove it from the ui and model package.json files and the pnpm-workspace catalog, and refresh the lockfile.
Real 10x BEAM Core Kit ships 15 antigens + 1 control; published panels are smaller (LIBRA-seq 9, flagship 2-5). Default the realistic profile to 15+1 and keep 64 only as an explicitly-labelled capacity stress test.
… + document sample model Generators now write bare `donorNN.tsv` / `donorNN.csv` (was `donorNN_airr_sc` / `donorNN_counts`) so Samples & Data extracts the same donor stem from all three arms and mints ONE shared sampleId — the multiomics cohort now imports coherently without a manual sampleId collapse. Validator updated in lockstep; regenerated + validated ALL PASS (412/412), 14632 clonotypes, 0 antigen mismatches. Adds a 'Sample model & assumptions' section to the README documenting the two synthetic-data assumptions to revisit against real BEAM: (1) filenames may carry library suffixes in real deliveries (which would re-fork sampleIds), (2) pooled-donor captures + demux are a future input mode, not assumed 1:1. Also corrects stale 64-antigen panel references to the 15+1 default committed earlier.
….create sub-templates Each mitool step (parse / refine-tags / tag-stat) captures rate-dependent progress via saveStdoutStream(), and the SDK exec.tpl is hash_override-pinned, so any getFile() off a streaming exec has a stable resource identity but a drifting content hash. Feeding such a file into a downstream exec's addFile and flattening that exec's per-sample output into a p-frame put a stable-id / drifting-hash node on the dedup path, which conflicted on re-render. Split the pipeline so every exec stage runs inside its own render.create sub-template (fb-parse / fb-refine / fb-tagstat / fb-downstream); fb-pipeline is now a thin orchestrator returning only render outputs, never an inline exec.getFile(). A render.create resource has a content-derived identity, so a consumer inside a boundary absorbs the drift and its per-sample outputs flatten cleanly (mirrors the peptide-extraction block). Progress grid and the A-0010 export contract are unchanged; unused saveStdoutStream on panel/qc/metrics and the unconsumed parse/refine .txt reports were dropped.
…sync CI) The root package.json software:reqs script runs 'turbo run software:reqs', but turbo.json did not declare the task, so turbo failed with 'Could not find task software:reqs in project' and the requirements-sync CI check errored before it could compile requirements. Add the task (matches the definition on main). Verified: pnpm software:reqs regenerates src/requirements.txt identically (numpy 2.2.6 / polars-lts-cpu 1.33.1 / scipy 1.15.3) -> no diff.
The 'emits per-cell umiCount' test hangs on the CI / run-platforma FS-storage backend: the tag->feature CSV is a direct upload consumed by file.importFile, and raw file.importFile of a local handle never finalizes there, so the prerun's csvColumns never resolves and awaitStableState aborts (field_not_resolved:csvColumns). This is a test-backend limitation, not a block bug — the block runs correctly live. No block e2e-tests this direct-upload path (immune-assay-data, blast, makeblastdb, antibody-sequence-liabilities all ship without block tests; the Samples & Data upstream chain, the only CI-working file-input pattern, cannot supply a direct CSV upload). The 'empty inputs' test still runs. Re-enable if the backend gains a working local file.importFile or the CSV moves to a pool column.
…t import orientation import-sc-rnaseq-data's detect_orientation() infers matrix orientation from axis lengths and transposes whenever cells outnumber genes. The reduced marker+filler GEX panel (~1041 genes) had fewer genes than the 2000 cells/donor, so the importer silently flipped it — CellTypist then saw cell barcodes as gene names and failed with "No features overlap with the model". Scale the filler so the gene count exceeds the largest per-donor cell count (now 2241 genes > 2000 cells), add a hard guard, and correct the docstring. More genes than cells is also the realistic scRNA-seq regime.
Collaborator
Author
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # Also emit the row as JSON so the model can read per-sample QC (getDataAsJson) to build the block's | ||
| # live "Analysis logs" — the per-sample completed count (heartbeat) and the run-level summary. | ||
| with open("result_qc.json", "w") as jf: | ||
| json.dump(row, jf) |
There was a problem hiding this comment.
panelAssignedFraction writes "" into a column declared as valueType: "Double"
qc_report.py emits an empty string for panelAssignedFraction when no refine report is available. The downstream qcSummaryColumnsSpec (column-specs.lib.tengo) declares that column as valueType: "Double". After qc-summary.tpl.tengo concatenates the per-sample one-row CSVs with inferSchema: false, the combined CSV contains "" for any sample that ran without a refine report. Whether the SDK's xsv.importFile coerces "" to null for a Double column or raises a parse error is implementation-dependent, and there is no test covering a mixed-presence refine-report scenario end-to-end through the import. A None sentinel (empty cell, but the CSV writer should already handle that) or a sentinel like NaN/-1 would make the intent explicit; alternately, panelAssignedFraction could be declared valueType: "String" and formatted in the UI layer.
Prompt To Fix With AI
This is a comment left during a code review.
Path: software/per-cell-metrics/src/qc_report.py
Line: 104
Comment:
**`panelAssignedFraction` writes `""` into a column declared as `valueType: "Double"`**
`qc_report.py` emits an empty string for `panelAssignedFraction` when no refine report is available. The downstream `qcSummaryColumnsSpec` (column-specs.lib.tengo) declares that column as `valueType: "Double"`. After `qc-summary.tpl.tengo` concatenates the per-sample one-row CSVs with `inferSchema: false`, the combined CSV contains `""` for any sample that ran without a refine report. Whether the SDK's `xsv.importFile` coerces `""` to null for a Double column or raises a parse error is implementation-dependent, and there is no test covering a mixed-presence refine-report scenario end-to-end through the import. A `None` sentinel (empty cell, but the CSV writer should already handle that) or a sentinel like `NaN`/`-1` would make the intent explicit; alternately, `panelAssignedFraction` could be declared `valueType: "String"` and formatted in the UI layer.
How can I resolve this? If you propose a fix, please make it concise.
Comment on lines
+311
to
+318
| numCol("medianUmisPerCell", "medianUmisPerCell", "Median UMIs / cell", 84000, ".1f"), | ||
| numCol("panelAssignedFraction", "panelAssignedFraction", "Panel-assigned fraction", 83000, ".2p", "Fraction of feature-barcode reads kept after correcting their barcode against the panel; reads too far from any panel entry are dropped. A low value flags a panel or read-geometry mismatch.") | ||
| ], | ||
| storageFormat: "Parquet", | ||
| partitionKeyLength: 0 | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
valueType: "Double"
readsTotal, readsMatched, cellsDetected, featuresDetected, and totalUniqueUmis are integer counts emitted by qc_report.py but imported with valueType: "Double". While functional (the CSV values parse cleanly as floating-point), this means downstream consumers see these metrics as floats rather than integers, which can affect formatting (e.g., 900.0 instead of 900) and type-level assertions in future code that reads these columns. Consider valueType: "Int" or valueType: "Long" for the count columns.
Prompt To Fix With AI
This is a comment left during a code review.
Path: workflow/src/column-specs.lib.tengo
Line: 311-318
Comment:
**Integer count columns declared as `valueType: "Double"`**
`readsTotal`, `readsMatched`, `cellsDetected`, `featuresDetected`, and `totalUniqueUmis` are integer counts emitted by `qc_report.py` but imported with `valueType: "Double"`. While functional (the CSV values parse cleanly as floating-point), this means downstream consumers see these metrics as floats rather than integers, which can affect formatting (e.g., `900.0` instead of `900`) and type-level assertions in future code that reads these columns. Consider `valueType: "Int"` or `valueType: "Long"` for the count columns.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Comment on lines
+83
to
+84
| ctrl = per_cell.filter(pl.col("feature") == control).select(["cellId", pl.col("umiCount").alias("_controlUmi")]) | ||
| per_cell = per_cell.join(ctrl, on="cellId", how="left").with_columns(pl.col("_controlUmi").fill_null(0)) |
There was a problem hiding this comment.
sampleId so the function is correct for any caller, not just the current single-sample invocations. Joining on ["sampleId", "cellId"] avoids cross-sample cellId collisions.
Suggested change
| ctrl = per_cell.filter(pl.col("feature") == control).select(["cellId", pl.col("umiCount").alias("_controlUmi")]) | |
| per_cell = per_cell.join(ctrl, on="cellId", how="left").with_columns(pl.col("_controlUmi").fill_null(0)) | |
| ctrl = per_cell.filter(pl.col("feature") == control).select(["sampleId", "cellId", pl.col("umiCount").alias("_controlUmi")]) | |
| per_cell = per_cell.join(ctrl, on=["sampleId", "cellId"], how="left").with_columns(pl.col("_controlUmi").fill_null(0)) |
Prompt To Fix With AI
This is a comment left during a code review.
Path: software/per-cell-metrics/src/per_cell_metrics.py
Line: 83-84
Comment:
The control join should include `sampleId` so the function is correct for any caller, not just the current single-sample invocations. Joining on `["sampleId", "cellId"]` avoids cross-sample cellId collisions.
```suggestion
ctrl = per_cell.filter(pl.col("feature") == control).select(["sampleId", "cellId", pl.col("umiCount").alias("_controlUmi")])
per_cell = per_cell.join(ctrl, on=["sampleId", "cellId"], how="left").with_columns(pl.col("_controlUmi").fill_null(0))
```
How can I resolve this? If you propose a fix, please make it concise.
Comment on lines
+357
to
+392
| lines.push( | ||
| `Reads parsed: ${nf(readsTotal)} total` + | ||
| (medMatched !== undefined | ||
| ? ` · ${pct(medMatched)} matched the read pattern (median)` | ||
| : "") + | ||
| ".", | ||
| ); | ||
| if (medAssigned !== undefined && assigned.length > 0) { | ||
| lines.push( | ||
| `Panel-assigned: ${pct(medAssigned)} of reads (median; range ${pct(Math.min(...assigned))}–${pct(Math.max(...assigned))}).`, | ||
| ); | ||
| } | ||
| lines.push(`Cells detected: ${nf(cellsTotal)} · ${features} features.`); | ||
| lines.push(""); | ||
| if (flagged.length > 0) { | ||
| const names = flagged.map((e) => labels?.[String(e.key[0])] ?? String(e.key[0])); | ||
| lines.push( | ||
| `${flagged.length} sample${flagged.length === 1 ? "" : "s"} flagged — panel-assigned fraction below ${pct(PANEL_ASSIGNED_FLOOR)} (or zero cells): ${names.join(", ")}.`, | ||
| ); | ||
| lines.push(" See the QC page for per-sample detail."); | ||
| } else { | ||
| lines.push("No samples flagged."); | ||
| } | ||
| lines.push("", "Analysis complete. Full per-sample statistics are on the QC page."); | ||
| return lines; | ||
| }) | ||
| // DECISION (2026-07-02, operator): the Main table is now ONE ROW PER CELL [sampleId, cellId]. | ||
| // Supersedes the 2026-07-01 "single unified matrix table" decision — the per-(cell x feature) rows | ||
| // moved OUT of this table into the collapsed workflow frame (consensus + the per-cell summary | ||
| // columns: Max Feature UMI count, Max Feature Fraction, Max Specificity score, and a "Feature | ||
| // breakdown" string listing every feature as "feature (fraction%, umi)" sorted by descending fraction). The | ||
| // per-feature matrix is not lost: it is still exported to the result pool (perCellFeatures, the | ||
| // A-0010 contract) for VDJ Multiomic Integration. This output resolves the workflow's collapsed | ||
| // perCellTable PFrame; undefined until the workflow emits it (guarded by the UI). | ||
| // | ||
| // Uses createPlDataTableV2 (columns passed directly via getPColumns), NOT V3. This frame is our OWN |
There was a problem hiding this comment.
analysisLog duplicates the sampleLabels lookup verbatim
The ~30-line label-lookup block (find the pl7.app/label column, decode the JSON-keyed entries) appears identically in both sampleLabels and analysisLog. The code comment acknowledges the constraint ("outputs cannot read one another"), but a plain helper function extracted above the .create() call would let both outputs call it without any SDK constraint — outputs compose over ctx, which is available in both closures. The duplication is a maintenance hazard: a future change to the axis-domain matching logic needs to be applied twice.
Prompt To Fix With AI
This is a comment left during a code review.
Path: model/src/index.ts
Line: 357-392
Comment:
**`analysisLog` duplicates the `sampleLabels` lookup verbatim**
The ~30-line label-lookup block (find the `pl7.app/label` column, decode the JSON-keyed entries) appears identically in both `sampleLabels` and `analysisLog`. The code comment acknowledges the constraint ("outputs cannot read one another"), but a plain helper function extracted above the `.create()` call would let both outputs call it without any SDK constraint — outputs compose over `ctx`, which is available in both closures. The duplication is a maintenance hazard: a future change to the axis-domain matching logic needs to be applied twice.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…dup) Correctness: - qc_report: cast the UMI column to Int64 before summing, matching per_cell_metrics._load, so a header-only tag-stat (all-off-panel sample) reports zeros instead of crashing the whole per-sample render. Add a header-only regression test. - model args(): clear a stale controlFeature (UI watchEffect) and reject a barcode/feature CSV column that collides with a reserved tag-stat column, so a no-longer-valid control can't silently score specificity against a zero control and a colliding column fails up front instead of at pipeline end. - wf.test.ts: update the (skipped) e2e golden values to the collapsed one-row-per-cell perCellTable contract. Fragility / cleanup: - Single PROGRESS_PREFIX source in tag-pattern.lib.tengo, referenced by the three exec templates; model comment points at it. - Name the mitool prose-matched stage substrings in results.ts. - per_cell_metrics: compute the within-cell fraction and betaCDF specificity once (with_fraction / with_specificity) and thread them into per_cell_summary; add a test that the summary maxima equal the exported columns. - model: extract resolveSampleLabels / parseQcRows helpers, removing the duplicated label resolver and qcJson parse across outputs.
…ictError on re-render) saveStdoutStream writes rate-dependent stdout as a file into the AID-pinned mitool exec's content-hashed files map, so the exec output's CID drifts across re-executions of the same AID. A same-version stale-upstream re-render re-executes the exec and the drifted CID throws CIDConflictError. The earlier render.create split only relocated the conflict. Drop saveStdoutStream (and the now-dead per-step progress plumbing) from parse/refine-tags/tag-stat so exec outputs are deterministic and dedup cleanly. Trade-off: loses the live per-step progress bars; the Main grid now shows Processing -> Done from qcJson. A-0010 export contract unchanged. Verified live in loud mode (PL_CID_CONFLICT_MODE=error), project BEAM6: virgin glossary, clean 2-sample baseline, then a 4-sample stale re-render with no rebuild finishes Done with perCellTable/qcSummaryTable/sampleQc all ok:true (the streaming version threw CIDConflictError under the same test).
The render.create changeset section claimed the progress grid still reads parseLogStream/stepLogs — false after the saveStdoutStream strip. Corrected it to note that split only relocated the conflict and point to the actual fix. Also removed the now-unused PROGRESS_PREFIX constant + export from tag-pattern.lib.tengo (no consumers after the strip).
…the column dropdowns
The tag→feature CSV column/feature dropdowns showed no feedback while staging
parsed the CSV, and staging ran two execs (a second one gated behind the
feature-column pick), so the perceived wait was two round-trips with a silent
empty state.
B1 — one staging exec:
- Replace emit_columns.py + emit_features.py with emit_csv_meta.py, which emits
{columns, valuesByColumn} in one exec: headers for the column dropdowns and
every column's distinct values for the negative-control dropdown.
- prerun.tpl runs that single exec on upload; drop featureNameColumn from
prerunArgs so picking the feature column no longer triggers a staging rerun —
the control dropdown re-indexes the already-emitted map (a model recompute).
- csvColumnOptions / controlOptions read csvMeta via a shared readCsvMeta(ctx).
- package.json: replace the emit-features + emit-columns entrypoints with
emit-csv-meta; swap the test to test_emit_csv_meta.py.
A1 — loading state:
- New non-retentive model output csvColumnsLoading (handle set, csvMeta not yet
resolved) — reports the live loading state on first upload and on CSV swap.
- MainPage shows a "Reading columns…" note and disables the CSV-derived
dropdowns while loading, so the empty window reads as loading, not "no
columns found".
Replace the per-arm manual test-data layout (antigen/, multiomics/, panel-swap/) with a single orchestrator (manual/generate.py + lib/) that builds a full, colocated multiomic run under runs/<preset>/ — the antigen, VDJ and GEX arms plus the shared block uploads and truth all in one folder, instead of scattered across three directories. - One entry point: presets tiny/realistic/whitelist737k, per-arm builds (--arm), antigen-only scenarios (--scenario, including the folded panel-swap and multisample beds), scale overrides, and --validate-only. The arm generators and the offline validator now live as importable modules in lib/. - VDJ/GEX read the antigen ground truth from within the one run dir, removing the old cross-arm ../antigen/<profile> coupling. - Drop the toy calibration path (always realistic now); hoist the shared load_clear_antigens into panel.py. - Move assets (737K list, gene annotations, harvested whitelist) to manual/assets/; update .gitignore to track only the recipe (generators, docs, whitelist_cells.txt) and ignore all generated output under runs/. - Merge the four arm READMEs into manual/README.md, carrying the FI + VDJM block-and-settings guide (verified against BEAM7, including the Sequence Space block and the VDJM per-antigen settings). Verified: realistic 412/412 and tiny 38/38 offline-validator PASS; all scenarios generate; arg overrides, whitelist737k, per-arm rebuild and --validate-only PASS; per-cell-metrics pytest 42/42 unaffected. fixtures/per-cell-metrics/ untouched.
noqa the gex PROGRAMS marker-gene data literals (clearer on one line) and the post-sys.path lib imports in generate.py. test-data is excluded from the package ruff gate, so these only surface when ruff is pointed at the files directly; keeping them clean avoids noise on explicit runs.
…ut watcher The negative-control selection was cleared by a watchEffect on the controlOptions output writing back to data.controlFeature — the spec-facts-resync hairpin (model.md / hairpin.md). Replace it with clearing on the user gesture that invalidates the control (changing the CSV or the feature-name column), a data->data write with no multi-client interleave and no hydration-timing risk.
…alls The control feature flowed into the antigen-level outputs: it could be picked as a cell's consensusFeature, and it got a self-score in specificity that could drive maxSpecificityScore. Treat it as the reference it is (spec A-0014): drop it from the consensus winner candidates (keeping its UMIs in the denominator, so a control-dominated cell reads as ambiguous rather than being renormalised into a false antigen call) and null its self-score so it is neither exported as a scored feature nor counted in the per-cell max. Abundance/fractions export unchanged. Adds control-handling unit + CLI oracle tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature Integration assigns antigens to single cells from feature-barcode (antigen-capture) reads — the antigen arm of the BEAM single-cell antibody-discovery workflow. It runs a per-sample mitool pipeline, collapses the result to per-cell metrics, and exports a p-frame that VDJ Multiomic Integration joins onto clonotypes.
Status: draft / work in progress — see caveats at the bottom.
Pipeline (per sample, fanned out by
processColumn)CELL(16) +UMI(10) from R1 and the feature barcodeFEATURE(15) from R2 (10x 5′ v2 geometry; lengths configurable).FEATUREsnaps to the panel whitelist (the tag column of the user's CSV); off-panel reads drop.CELLis de-novo corrected. The tag order scopes UMI dedup to(cell, feature).(cell, feature).Downstream contract (exports)
The block exports a
perCellFeaturesp-frame to the result pool, keyed[pl7.app/sampleId, pl7.app/sc/cellId, pl7.app/feature/featureId]:umiCount,fraction,consensusFeature, andspecificityScore(when a negative control is set). VDJ Multiomic Integration discoversumiCountthrough the cell linker under its clonotype anchor. This is the A-0010 contract and is stable.UI
[sampleId, cellId]: consensus feature, per-cell max metrics, and a feature-breakdown string.Both result tabs appear only after a run.
Caveats
#builtin:) plumbing is kept dormant behind a documented seam; cross-arm cell-barcode alignment is a chain-level concern to verify once the downstream join is checked end to end.Greptile Summary
This PR implements the Feature Integration block — the antigen-capture arm of the BEAM single-cell workflow. It builds a full per-sample mitool pipeline (
parse → refine-tags → tag-stat -u) plus a Python (per_cell_metrics.py) layer that collapses the result to the A-0010 p-frame contract keyed[sampleId, cellId, featureId], and exports it to the result pool for downstream VDJ Multiomic Integration.render.createsub-template (fb-parse/fb-refine/fb-tagstat/fb-downstream) so streaming-exec file outputs never cross a processColumn flatten boundary directly.PlRefto a direct-uploadImportFileHandle, with a prerun staging template that drives the upload, emits CSV column headers for the barcode/feature dropdowns (D4), and emits feature names for the negative-control dropdown before the user clicks Run.[sampleId, cellId]using the newperCellSummarycollapse).Confidence Score: 3/5
Draft PR with one unverified runtime path (panelAssignedFraction empty-string → Double xsv import) and the CID fix itself acknowledged as not yet live-verified; should not be merged until a fresh-project run confirms both.
The panelAssignedFraction field is written as an empty string by qc_report.py when no refine report is present, and the xsv import schema declares it Double. If the SDK's CSV importer does not silently coerce "" to null for numeric columns, every sample without a refine report will fail the QC-summary import at runtime — a path not covered by any existing test. The CID fix is confirmed by the author as not yet live-verified; it builds correctly but the resource-graph conflict is a runtime error the build cannot catch.
software/per-cell-metrics/src/qc_report.py (panelAssignedFraction empty-string), workflow/src/column-specs.lib.tengo (integer-count columns typed as Double), model/src/index.ts (duplicated label-lookup logic)
Important Files Changed
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant User participant UI as MainPage (Vue) participant Model as model/src/index.ts participant Prerun as prerun.tpl.tengo participant Main as main.tpl.tengo participant FbPipeline as fb-pipeline (per sample) participant FbParse as fb-parse sub-tpl participant FbRefine as fb-refine sub-tpl participant FbTagstat as fb-tagstat sub-tpl participant FbDownstream as fb-downstream sub-tpl participant Python as per_cell_metrics.py participant QcReport as qc_report.py participant QcSummary as qc-summary.tpl.tengo User->>UI: Upload tags.csv + select FASTQ UI->>Prerun: prerunArgs (tagFeatureCsvHandle, featureNameColumn) Prerun->>Prerun: emit-columns → csvColumns Prerun->>Prerun: emit-features → featureNames Prerun-->>Model: csvColumnOptions, controlOptions (via prerun resolve) Model-->>UI: populates barcode/feature dropdowns + control dropdown User->>UI: Configure settings, click Run UI->>Main: args (fbFastqRef, tagFeatureCsvHandle, barcodeSeqColumn, featureNameColumn, ...) Main->>Main: file.importFile(tagFeatureCsvHandle) Main->>Main: tagPattern.build(geom) → pattern loop per sample (processColumn fan-out) Main->>FbPipeline: extra inputs (pattern, tagsCsv, control, ...) FbPipeline->>FbParse: render.create → parsedMic, parseReport, parseLogStream FbParse->>FbParse: mitool parse FbPipeline->>FbRefine: render.create → refinedMic, refineReport, refineLogStream FbRefine->>FbRefine: emit-panel (panel.txt) + mitool refine-tags FbPipeline->>FbTagstat: render.create → tagstatTsv, tagstatLogStream FbTagstat->>FbTagstat: mitool tag-stat -u FbPipeline->>FbDownstream: render.create → abundance/fractions/consensus/specificity/perCellSummary/qc/qcJson FbDownstream->>QcReport: qc_report.py → result_qc.csv/.json FbDownstream->>Python: "per_cell_metrics.py → result_*.csv" end Main->>QcSummary: "render.create(qcData=[sampleId]→qcCSV)" QcSummary->>QcSummary: concat per-sample QC CSVs + xsv.importFile Main-->>Model: exports perCellFeatures (A-0010), outputs perCellTable/qcSummaryTable/qcJson/stepLogs Model-->>UI: completedSamples, sampleProgress, sampleQc, analysisLog%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant User participant UI as MainPage (Vue) participant Model as model/src/index.ts participant Prerun as prerun.tpl.tengo participant Main as main.tpl.tengo participant FbPipeline as fb-pipeline (per sample) participant FbParse as fb-parse sub-tpl participant FbRefine as fb-refine sub-tpl participant FbTagstat as fb-tagstat sub-tpl participant FbDownstream as fb-downstream sub-tpl participant Python as per_cell_metrics.py participant QcReport as qc_report.py participant QcSummary as qc-summary.tpl.tengo User->>UI: Upload tags.csv + select FASTQ UI->>Prerun: prerunArgs (tagFeatureCsvHandle, featureNameColumn) Prerun->>Prerun: emit-columns → csvColumns Prerun->>Prerun: emit-features → featureNames Prerun-->>Model: csvColumnOptions, controlOptions (via prerun resolve) Model-->>UI: populates barcode/feature dropdowns + control dropdown User->>UI: Configure settings, click Run UI->>Main: args (fbFastqRef, tagFeatureCsvHandle, barcodeSeqColumn, featureNameColumn, ...) Main->>Main: file.importFile(tagFeatureCsvHandle) Main->>Main: tagPattern.build(geom) → pattern loop per sample (processColumn fan-out) Main->>FbPipeline: extra inputs (pattern, tagsCsv, control, ...) FbPipeline->>FbParse: render.create → parsedMic, parseReport, parseLogStream FbParse->>FbParse: mitool parse FbPipeline->>FbRefine: render.create → refinedMic, refineReport, refineLogStream FbRefine->>FbRefine: emit-panel (panel.txt) + mitool refine-tags FbPipeline->>FbTagstat: render.create → tagstatTsv, tagstatLogStream FbTagstat->>FbTagstat: mitool tag-stat -u FbPipeline->>FbDownstream: render.create → abundance/fractions/consensus/specificity/perCellSummary/qc/qcJson FbDownstream->>QcReport: qc_report.py → result_qc.csv/.json FbDownstream->>Python: "per_cell_metrics.py → result_*.csv" end Main->>QcSummary: "render.create(qcData=[sampleId]→qcCSV)" QcSummary->>QcSummary: concat per-sample QC CSVs + xsv.importFile Main-->>Model: exports perCellFeatures (A-0010), outputs perCellTable/qcSummaryTable/qcJson/stepLogs Model-->>UI: completedSamples, sampleProgress, sampleQc, analysisLogPrompt To Fix All With AI
Reviews (1): Last reviewed commit: "MILAB-6496: skip the CI-unrunnable end-t..." | Re-trigger Greptile
Context used: