Define leakage-resistant delay-model contract

.gitignore

@@ -11,3 +11,7 @@ build/
 *.egg-info/
 .DS_Store
 .vercel
+data/raw/
+data/processed/
+artifacts/models/
+artifacts/reports/

README.md

@@ -153,6 +153,8 @@ GitHub Actions installs the project in a clean Python 3.12 runner and executes t
 - [ ] AWS deployment with Terraform and a cost estimate
 - [ ] Load-test report, SLO, and incident write-up
 
+The trained-model milestone is specified in [Issue #2](https://github.com/mitulpatel123/flightops-ai/issues/2) and begins with a leakage-resistant [data contract](docs/ml/data-contract.md).
+
 ## Project integrity
 
 This is original work. External datasets and libraries will be documented with their sources and licenses. Future capabilities are listed as roadmap items instead of being presented as finished work.

docs/ml/data-contract.md

@@ -0,0 +1,125 @@
+# Flight-delay model data contract
+
+Status: proposed for [Issue #2](https://github.com/mitulpatel123/flightops-ai/issues/2)
+
+## Decision this contract protects
+
+The model answers one operational question:
+
+> At 24 hours before scheduled departure, what is the probability that an operated, non-diverted flight will arrive at least 15 minutes late?
+
+This prediction time is fixed before feature engineering. A field is eligible only if it would be known at that time or can be calculated exclusively from earlier flights.
+
+The model does **not** predict cancellation, diversion, causal delay attribution, or passenger impact. Those require separate targets and evaluation.
+
+## Source and unit of observation
+
+- Source: U.S. Bureau of Transportation Statistics, Reporting Carrier On-Time Performance
+- Official dataset page: https://www.transtats.bts.gov/TableInfo.asp?QO_fu146_anzr=b0-gvzr&gnoyr_VQ=FGJ
+- Official field reference: https://www.transtats.bts.gov/Fields.asp?gnoyr_VQ=FGJ
+- Unit: one scheduled nonstop domestic flight
+- Candidate study window: January 2024 through June 2025
+- Planned chronological split:
+  - train: January–December 2024
+  - validation: January–March 2025
+  - test: April–June 2025
+
+The exact downloaded files, release identifiers, byte sizes, and checksums must be recorded by the ingestion command before training. Large raw files must not be committed to Git.
+
+## Cohort and target
+
+Include rows where:
+
+- `Cancelled == 0`
+- `Diverted == 0`
+- `ArrDel15` is present
+
+Define:
+
+```text
+target_arrival_delay_15 = 1 when ArrDel15 == 1, otherwise 0
+```
+
+BTS defines an arrival as delayed when it is at least 15 minutes after the published arrival time. Cancelled and diverted flights are excluded from this target, but their exclusion rate must be reported for every split.
+
+## Features available at T-24h
+
+Direct schedule fields:
+
+- `Year`, `Quarter`, `Month`, `DayofMonth`, `DayOfWeek`, `FlightDate`
+- `Reporting_Airline` or stable DOT carrier ID
+- `OriginAirportID`, `DestAirportID`
+- `CRSDepTime`, `CRSArrTime`, `DepTimeBlk`, `ArrTimeBlk`
+- `CRSElapsedTime`
+- `Distance`, `DistanceGroup`
+
+Derived schedule fields:
+
+- departure hour and cyclical hour encoding
+- weekend indicator
+- route identifier
+- route distance band
+
+Historical aggregates are permitted only when calculated from rows strictly earlier than the prediction row:
+
+- carrier prior-delay rate
+- origin and destination prior-delay rates
+- route prior-delay rate and volume
+- calendar-month prior-delay rate
+
+All low-volume aggregate features must use smoothing and an explicit fallback to the training-set prior.
+
+## Prohibited leakage fields
+
+The following fields reveal events at or after departure and must never enter the T-24h feature matrix:
+
+- `DepTime`, `DepDelay`, `DepDelayMinutes`, `DepDel15`, `DepartureDelayGroups`
+- `TaxiOut`, `WheelsOff`, `WheelsOn`, `TaxiIn`
+- `ArrTime`, `ArrDelay`, `ArrDelayMinutes`, `ArrDel15`, `ArrivalDelayGroups`
+- `ActualElapsedTime`, `AirTime`
+- `Cancelled`, `CancellationCode`, `Diverted`, and diverted-flight details
+- `CarrierDelay`, `WeatherDelay`, `NASDelay`, `SecurityDelay`, `LateAircraftDelay`
+- gate-return and additional-ground-time fields
+
+`ArrDel15` is allowed only while constructing the label. After that step it must be removed from the feature frame.
+
+`Tail_Number` is excluded from v1 because aircraft assignment may change and availability at T-24h is not guaranteed.
+
+## Missing values and categories
+
+- Reject rows missing the target, scheduled times, carrier, origin, or destination.
+- Fit imputers and encoders on the training split only.
+- Preserve an explicit unknown category for carriers, airports, and routes not seen during training.
+- Record missingness rates by split before imputation.
+- Fail preprocessing when required columns are absent or their types change unexpectedly.
+
+## Evaluation contract
+
+Every report must include:
+
+- row count, positive prevalence, cancellation exclusions, and diversion exclusions by split
+- majority-class baseline
+- logistic-regression baseline
+- one tree-based model with a fixed, documented search budget
+- ROC-AUC and PR-AUC
+- precision, recall, and F1 at the selected operating threshold
+- Brier score and a calibration curve
+- confusion matrix and expected operational cost at the selected threshold
+- error slices by carrier, origin, destination, route-volume band, departure-time block, and calendar month
+
+The operating threshold must be chosen on validation data from a visible cost assumption, then evaluated once on the untouched test period.
+
+## Reproducibility and artifact safety
+
+- Pin training dependencies separately from API runtime dependencies.
+- Store configuration, feature names, metrics, code commit, and data checksums with every model version.
+- Prefer portable, inspectable model artifacts. Do not load untrusted pickle or joblib files.
+- Keep `baseline-rules-v1` available as an explainable fallback until the trained model passes all acceptance criteria.
+- Do not claim production accuracy from a retrospective public dataset.
+
+## Known limitations
+
+- BTS coverage is limited to reporting U.S. carriers and reportable domestic operations.
+- Scheduled information alone cannot represent real-time weather, maintenance, crew, or network disruptions.
+- Historical rates can encode structural differences between airports, routes, and carriers; error slices must be reviewed before use.
+- Distribution shifts across seasons and operational regimes can reduce performance after the evaluation window.