Data Updates wamsi-csiem(v1.7.0)

[santi727G]

This thread will record all updates and changes to the v1.7.0 dataset.
Please add a new comment below whenever data files are modified, replaced, or reprocessed.

[santi727G]

Data sync csiem-data v1.6.0 ->

code

Updated:

• \import\wamsi_theme3\CTD\import_theme3ctd_data.m
• \import\wamsi_theme3\SEDDEPO\IMPORTSEDDEPO.m

Added:

• \import_py\WAMSI\importWAMSIWWMSP4_chla.py

data-governance

Updated: site_key and variable_key

data-lake updates

• \CSIRO\Dalseno was added

• WAMSI\WWMSP3\WWMSP3.1_SEDDEPO → WWMSP3.1.3_SEDDEPO: updated to latest available data.

• WAMSI\WWMSP3\WWMSP3.1_CTD → WWMSP3.1.4_CTD: updated to latest available data.

• WAMSI\WWMSP4\WWMSP4_zoop\Chl_a_Westport_process.xlsx: added.

data-warehouse

Updated: csv\wamsi\wwmsp3\ctd, csv\wamsi\wwmsp3\seddepo
Added: csv\wamsi\wwmsp4\chla, csv\csiro\dalseno

Aggregations

• csiem_WAMSI_public.mat was updated with the new data

[santi727G]

Sentinel Data

Code

Created: \code\import_py\ESA\importESASENTINEL2.py

New Sentinel‑2 importer. Recognizes site names (keeps CS_CENTER, Point_6, etc.) and writes matching DATA/HEADER files. No path or date‑format changes.

Updated: \code\import_py\ESA\importESASENTINEL3.py

Previose Sentinel import was change to Santinel 3. Previoes export now land under the SEN3/OLCI warehouse path. No date‑format changes.

data-governance

Updated: site_key and variable_key

• site_key: validated/added coordinates for SEN2 C2RCC‑NN points (id, lat, lon).

• variable_key: confirmed mapping from chl_value to Chlorophyll‑a (IDs and conversion).

data-warehouse

Sentinel‑2 (C2RCC‑NN)

• Exported named sites: 16 DATA files.

• Exported polygons: 11 DATA files.

• Exported numeric points: 3 DATA files (Point_6, Point_8, Point_9).

• Skipped points with no valid values (Point_1–5, 7, 10–13).

Sentinel‑3 (OLCI)

• Exported points: 32 DATA files (one per Point_*).

• Polygon aggregates also present in the OLCI folder.

notes

• All exported DATA use the existing format: Date,Depth,Data,QC with Date as yyyy‑mm‑dd HH:MM:SS (kept consistent with prior loads)

• All SMD were updated

[santi727G]

BOM IDO Data

Code

Created: \code\import_py\BOM\importBOMIDO.py, a BOM IDO importer that consumes the refreshed site/variable keys and writes DATA/HEADER pairs without altering legacy folders or date formats.

data-governance

• site_key: added/validated the BOM IDO station IDs plus lat/lon so each point resolves during the import.

• variable_key: mapped BOM IDO measurements (e.g., temperature, rainfall, wind) to their Chlorophyll‑a-style counterparts with confirmed IDs/conversions.

data-warehouse | BOM IDO

Generated DATA/HEADER outputs for every 1 station was included in the refreshed site_key

[santi727G]

CSIEM mat files update

data-warehouse

csiem_ESA_public and csiem_WAMSI_public have been updated including lasted data

[santi727G]

WCMA data update

Code:

• Updated: \code\import\WCWA\calc_dates.m

Import was updated in order to solve a date misalignment of oxygen data

• Updated:\code\import\WCWA\import_PhyWQ_1334_09.m

New functions were included to import the dissolved oxigen concentration into data-warehouse

data-warehouse

• Dissolved oxigen concentration has been add to the current available data

• csiem_WCWA_public has been updated to solve a date misalignment of WC dataset and the inclusion of dissolved oxigen concentration

[santi727G]

DWER CSMOORING update

Code:

Created:

• \code\import_py\DWER\ImportDWER_CSMooring_WQB.py : Campaign B import (Cockburn Mooring EXO dataset)

• \code\import_py\DWER\ importDWER_CSMooring_MS9D.py: Campaign D import (CS86 Dual MS9 Kd dataset).

Updated:

• \code\import_py\DWER\ importDWER_CSMooring_WQC.py : TAG was updated to specify the campaing name

Data Governance:

• site_key: Updated to include and align stations required by the new DWER mooring imports.

• variable_key: Updated to support mappings required by B/C/D imports.

Data Warehouse:

A/C/D outputs are now ingested into DWER CSMOORING warehouse paths with campaign-specific processing.

[santi727G]

Update Data Summary

Changes

• Added new summary images including the latest imported dataset
• Updated raw data plot styling