Skip to content

Fix labelled data counts not covering all selected train IDs - #751

Open
philsmt wants to merge 1 commit into
masterfrom
fix/incomplete-labelled-counts
Open

Fix labelled data counts not covering all selected train IDs#751
philsmt wants to merge 1 commit into
masterfrom
fix/incomplete-labelled-counts

Conversation

@philsmt

@philsmt philsmt commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

When building tests for exdf-tools, I noticed a difference in behaviour between labelled and unlabelled KeyData.data_counts(). Whereas the latter always covers the selected train IDs, the former only contains only those described by data chunks for this key.

This situation can arise if a source is missing entirely in one or more sequences as opposed to being present with no data counts. In practice, this can happen e.g. when:

  • A device goes down during a run, causing the aggregator to close the current sequence and start a new one
  • Combining two runs with a source only present in one of them
  • A source being dropped from sequence due to train selection in exdf-reduce (what triggered this observation)

While this is somewhat recoverable given the returned series is labelled, I would argue our APIs generally cover all selected train IDs and hence I would fix this for consistency.

@philsmt
philsmt requested a review from takluyver July 2, 2026 06:39
@philsmt
philsmt force-pushed the fix/incomplete-labelled-counts branch from a42ce8a to 86303ce Compare July 2, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant