DeMeTrA: Detection and Tracking of Medicanes

Overview

DeMeTrA is a self-supervised vision-transformer framework for detecting and tracking Mediterranean tropical-like cyclones (Medicanes) from SEVIRI/MSG AirmassRGB satellite imagery. It provides workflows for self-supervised pretraining, supervised cyclone detection, and cyclone-centre tracking.

Preprint: Detection and Tracking of Medicanes Through DeMeTrA Self-Supervised Vision Transformer — Daniele D’Armiento, Stefano Sebastianelli, Leo Pio D’Adderio, Paolo Sanò, Daniele Casella, and Giulia Panegrossi (2026). DOI: 10.20944/preprints202605.1494.v1

The repo is developed to:

• Pretrain the model for a specialization phase using unlabeled satellite video sequences.
• Fine-tune the pretrained model for cyclone detection and center tracking.

---

Terminology

Throughout this repository, Mediterranean cyclone reference tracks (or cyclone-centre reference tracks when discussing spatial labels) refers to the composite reference-track datasets introduced by Flaounas et al.:

Flaounas, E., et al. (2023). A composite approach to produce reference datasets for extratropical cyclone tracks: application to Mediterranean cyclones. Weather and Climate Dynamics, 4, 639–661. https://doi.org/10.5194/wcd-4-639-2023

Some legacy filenames, function names, notebook names, and command-line options contain manos. These identifiers are retained only for compatibility and should not be interpreted as the name or source of the dataset.

---

Repository Structure

├── specialization.py            # Additional unsupervised pretraining ("specialization")
├── classification.py            # Fine-tuning entry point (classification)
├── tracking.py                  # Cyclone-centre regression training
├── inference_classification.py  # Classification inference
├── predict_and_track_from_folder.py # End-to-end operational inference
├── engine_for_pretraining.py    # Training loop for pretraining
├── engine_for_finetuning.py     # Training loop for fine-tuning
├── engine_for_tracking.py       # Training loop for tracking
├── dataset/                     # Dataset building, loading, and augmentation
├── models/                      # VideoMAE-based model definitions
├── medicane_utils/              # Satellite and geospatial utilities
├── make_dataset_from_rgb.py     # Entry points for dataset construction
├── docs/                        # Project and workflow documentation
└── arguments.py                 # Centralised training arguments

The project guide explains how these components fit together across the complete data, training, inference, and evaluation pipeline.

---

Documentation

The following index collects the project documentation, notebooks, and operational notes.

Project overview and setup

• Extended project guide
• Installation
• Supplementary material: Ianos reference-track–image consistency

Training workflows

• Self-supervised pretraining
• Specialization workflow
• Classification training
• Cyclone-centre tracking
• Distributed multi-GPU and multi-node training
• Training call tree
• Learning-rate mechanics

Dataset construction and preparation

• Building the VideoMAE dataset
• Dataset experiments and relabelling
• Building datasets from AirmassRGB imagery
• Cloud-index analysis
• Geographic constants and utilities
• Pixel-to-kilometre coordinate conversion

Inference and operational workflows

• Classification inference
• Inference from a folder
• Detection and tracking from a folder
• Tracking from a folder
• General prediction workflow

Analysis, evaluation, and visualisation

• Mediterranean cyclone reference-track analysis
• Performance metrics
• Model statistics
• Training-loss visualisation
• Metrics comparison plots
• Mediterranean tracking predictions
• Mediterranean validation predictions
• Test-tile visualisation
• Tracking-tile visualisation
• Patch verification
• Cyclone-video generation

Project notes

• Completed work and metrics
• Open tasks

---

Datasets

• Unsupervised pretraining: uses unlabeled video sequences (AirmassRGB), with loss for patch reconstruction
• Supervised fine-tuning: uses Medicanes tracks file for
  • classification training for detection,
  • Regression training for center tracking.

---

How to Run

Installation

Please follow the instructions in INSTALL.md.

Pretraining

python specialization.py [OPTIONS...]

Fine-tuning: Medicane Detection

python classification.py [OPTIONS...]

Fine-tuning: Center Tracking

python tracking.py [OPTIONS...]

Inference From Folder

See docs/inference_from_folder.md.

---

Download and Processing of AirmassRGB

To download and process EUMETSAT satellite images into AirmassRGB composites, use the script:

python medicane_utils/download_airmassRGB.py --start "2020-09-01 00:00" --end "2020-09-15 23:59"

Citation

If you use DeMeTrA, please cite the preprint:

D’Armiento, D., Sebastianelli, S., D’Adderio, L. P., Sanò, P., Casella, D., & Panegrossi, G. (2026). Detection and Tracking of Medicanes Through DeMeTrA Self-Supervised Vision Transformer. Preprints.org. https://doi.org/10.20944/preprints202605.1494.v1

GitHub also exposes this citation through CITATION.cff.

Upstream work

DeMeTrA builds on VideoMAE v2. Please also cite the upstream work where appropriate:

• VideoMAE v1 (NeurIPS 2022)
• VideoMAE v2 (CVPR 2023)