You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Replace the three separate PipelineIterator constructors — new_synthetic, new_from_file, new_from_file_streaming (qdp-core/src/pipeline_runner.rs) — with a single new that takes a Source enum, so the source becomes an argument rather than a function name.
Why
The three constructors share the same normalize → build → spawn-producer-thread logic, differing only in which source they read from. Keeping three copies means every pipeline-setup change is made three times and they drift apart. A Source enum makes the source an argument rather than a function name, so the setup path exists once.
new runs the shared config.normalize() / build / spawn logic and matches on source only where behavior genuinely differs (note the current file constructors also take a batch_limit — fold it into the File/FileStreaming variants or the config, and be explicit about where it lives).
Either keep the old new_* methods as thin deprecated wrappers that delegate to new, or update the PyO3 call sites (qdp-python/src/loader.rs) in the same PR — pick one and be consistent.
No producer logic change in this issue (that is E2); this is purely constructor unification.
What
Replace the three separate
PipelineIteratorconstructors —new_synthetic,new_from_file,new_from_file_streaming(qdp-core/src/pipeline_runner.rs) — with a singlenewthat takes aSourceenum, so the source becomes an argument rather than a function name.Why
The three constructors share the same normalize → build → spawn-producer-thread logic, differing only in which source they read from. Keeping three copies means every pipeline-setup change is made three times and they drift apart. A
Sourceenum makes the source an argument rather than a function name, so the setup path exists once.How
Sourceenum inpipeline_runner.rs:newruns the sharedconfig.normalize()/ build / spawn logic and matches onsourceonly where behavior genuinely differs (note the current file constructors also take abatch_limit— fold it into theFile/FileStreamingvariants or theconfig, and be explicit about where it lives).new_*methods as thin deprecated wrappers that delegate tonew, or update the PyO3 call sites (qdp-python/src/loader.rs) in the same PR — pick one and be consistent.Out of scope: Producer unification (E2), VecDeque buffer change (E3), any behavior change to batch production.
Acceptance criteria:
Sourcevariant constructed viaPipelineIterator::newnew_*paths still work (as wrappers) or all call sites updated in-PR