Skip to content

Populate accession_id for HuBMAP collections #102

Description

@conradbzura

Populate accession_id for HuBMAP collections

Follow-up to #36, which added the cross-DCC accession_id field, and to #37/#38, which populated it for 4DN and ENCODE. HuBMAP was deliberately deferred; this issue records why and what it would take.

Why HuBMAP was deferred

There is no per-file accession. HuBMAP files are not individually accessioned. Per the entity-matching contract in src/cfdb/services/hubmap.py, a file is located by collection doi_url → dataset, then filename — there is no HBM…-style identifier on a file to put in accession_id. This is a genuine difference from 4DN (accession in persistent_id) and ENCODE (accession is local_id), not a gap in the ingest. File-level accession_id should stay null for HuBMAP.

The dataset-level identifier exists but is not ingested. HuBMAP datasets do carry a hubmap_id (e.g. HBM123.ABCD.456), which is the natural collections[].accession_id. The Search API enrichment currently pulls dataset_type, analyte_class, pipeline, processing, and friends, and matches collections on doi_url — it never reads hubmap_id. So populating the field is blocked on extending the ingest, not just on a mapping decision.

There is no HuBMAP data to validate against. The dev environment currently holds zero HuBMAP files (4DN 53,697 + ENCODE 242,000 = 295,697 total), so any implementation would ship unverified end-to-end.

Expected outcome

  • _enrich_hubmap_collections_and_subjects in src/cfdb/services/sync.py reads hubmap_id from the Search API dataset payload and sets collections[].accession_id from it, folded through cfdb.accessions.normalize_accession so it matches what the GraphQL layer folds filter values to.
  • The mapping is recorded in the field-mapping docstring at the top of src/cfdb/services/hubmap.py, alongside the existing entries.
  • A HuBMAP dataset with no hubmap_id leaves accession_id null and is logged, rather than failing the sync — matching how the 4DN passes handle an unparseable accession.
  • File-level accession_id remains null for HuBMAP, and that is documented as intended rather than as a gap.
  • Requires HuBMAP data present in the target environment to verify; confirm a sync has actually loaded HuBMAP files before treating this as done.

Notes

accession_id is written during sync/materialization, so this only takes effect on the next HuBMAP sync — existing documents stay null until then.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions