Currently stream::rebatch uses column_t::take that has a complexity linear to the number of elements and thus rebatch may have up to quadratic complexity when large batch is rebatched to batch size 1. A solution would be to store the batch in std::deque instead of std::vector.
See https://github.com/iterait/hipipe/blob/dev/include/hipipe/core/stream/column_t.hpp#L171.
Currently
stream::rebatchusescolumn_t::takethat has a complexity linear to the number of elements and thusrebatchmay have up to quadratic complexity when large batch is rebatched to batch size 1. A solution would be to store the batch instd::dequeinstead ofstd::vector.See https://github.com/iterait/hipipe/blob/dev/include/hipipe/core/stream/column_t.hpp#L171.