Populate accession_id for HuBMAP collections
Follow-up to #36, which added the cross-DCC accession_id field, and to #37/#38, which populated it for 4DN and ENCODE. HuBMAP was deliberately deferred; this issue records why and what it would take.
Why HuBMAP was deferred
There is no per-file accession. HuBMAP files are not individually accessioned. Per the entity-matching contract in src/cfdb/services/hubmap.py, a file is located by collection doi_url → dataset, then filename — there is no HBM…-style identifier on a file to put in accession_id. This is a genuine difference from 4DN (accession in persistent_id) and ENCODE (accession is local_id), not a gap in the ingest. File-level accession_id should stay null for HuBMAP.
The dataset-level identifier exists but is not ingested. HuBMAP datasets do carry a hubmap_id (e.g. HBM123.ABCD.456), which is the natural collections[].accession_id. The Search API enrichment currently pulls dataset_type, analyte_class, pipeline, processing, and friends, and matches collections on doi_url — it never reads hubmap_id. So populating the field is blocked on extending the ingest, not just on a mapping decision.
There is no HuBMAP data to validate against. The dev environment currently holds zero HuBMAP files (4DN 53,697 + ENCODE 242,000 = 295,697 total), so any implementation would ship unverified end-to-end.
Expected outcome
_enrich_hubmap_collections_and_subjects in src/cfdb/services/sync.py reads hubmap_id from the Search API dataset payload and sets collections[].accession_id from it, folded through cfdb.accessions.normalize_accession so it matches what the GraphQL layer folds filter values to.
- The mapping is recorded in the field-mapping docstring at the top of
src/cfdb/services/hubmap.py, alongside the existing entries.
- A HuBMAP dataset with no
hubmap_id leaves accession_id null and is logged, rather than failing the sync — matching how the 4DN passes handle an unparseable accession.
- File-level
accession_id remains null for HuBMAP, and that is documented as intended rather than as a gap.
- Requires HuBMAP data present in the target environment to verify; confirm a sync has actually loaded HuBMAP files before treating this as done.
Notes
accession_id is written during sync/materialization, so this only takes effect on the next HuBMAP sync — existing documents stay null until then.
Populate
accession_idfor HuBMAP collectionsFollow-up to #36, which added the cross-DCC
accession_idfield, and to #37/#38, which populated it for 4DN and ENCODE. HuBMAP was deliberately deferred; this issue records why and what it would take.Why HuBMAP was deferred
There is no per-file accession. HuBMAP files are not individually accessioned. Per the entity-matching contract in
src/cfdb/services/hubmap.py, a file is located by collectiondoi_url→ dataset, then filename — there is noHBM…-style identifier on a file to put inaccession_id. This is a genuine difference from 4DN (accession inpersistent_id) and ENCODE (accession islocal_id), not a gap in the ingest. File-levelaccession_idshould stay null for HuBMAP.The dataset-level identifier exists but is not ingested. HuBMAP datasets do carry a
hubmap_id(e.g.HBM123.ABCD.456), which is the naturalcollections[].accession_id. The Search API enrichment currently pullsdataset_type,analyte_class,pipeline,processing, and friends, and matches collections ondoi_url— it never readshubmap_id. So populating the field is blocked on extending the ingest, not just on a mapping decision.There is no HuBMAP data to validate against. The dev environment currently holds zero HuBMAP files (4DN 53,697 + ENCODE 242,000 = 295,697 total), so any implementation would ship unverified end-to-end.
Expected outcome
_enrich_hubmap_collections_and_subjectsinsrc/cfdb/services/sync.pyreadshubmap_idfrom the Search API dataset payload and setscollections[].accession_idfrom it, folded throughcfdb.accessions.normalize_accessionso it matches what the GraphQL layer folds filter values to.src/cfdb/services/hubmap.py, alongside the existing entries.hubmap_idleavesaccession_idnull and is logged, rather than failing the sync — matching how the 4DN passes handle an unparseable accession.accession_idremains null for HuBMAP, and that is documented as intended rather than as a gap.Notes
accession_idis written during sync/materialization, so this only takes effect on the next HuBMAP sync — existing documents stay null until then.