Milestone: evaluated flight-delay prediction model

[mitulpatel123]

Objective

Replace the rules-only delay score with a reproducible trained-model milestone while retaining baseline-rules-v1 as the benchmark and fallback.

Data source

Use the official U.S. Bureau of Transportation Statistics Airline On-Time Performance data:

• https://www.bts.gov/browse-statistical-products-and-data/bts-publications/airline-service-quality-performance-234-time
• https://transtats.bts.gov/DatabaseInfo.asp?DB_URL=&QO_VQ=EFD

Record the exact months, download date, selected fields, and transformation logic. Do not commit a large raw dataset to Git.

Acceptance criteria

• Define the prediction time and target: arrival delay of at least 15 minutes
• Create a documented data contract and reproducible preprocessing command
• Prevent target leakage by excluding actual arrival/departure outcomes unavailable at prediction time
• Use a chronological train/validation/test split
• Compare a naive majority baseline, logistic regression, and one tree-based model
• Report ROC-AUC, PR-AUC, precision, recall, F1, Brier score, and calibration
• Include error slices by carrier, origin airport, route volume, and time period
• Choose the operating threshold from a documented operational cost assumption
• Save versioned model metadata without committing unsafe or opaque serialized artifacts
• Publish a model card with data limitations, intended use, non-goals, and failure modes
• Add unit tests for preprocessing and an integration test for model-backed prediction
• Keep baseline-rules-v1 available as a transparent fallback

Recruiter evidence

The README must show a compact comparison table, one calibration visualization, one error-slice finding, the exact reproduction command, and an honest explanation of what improved and what did not.

Out of scope

• claiming real-time airline production readiness
• claiming causality
• using post-event fields that leak the outcome
• adding an LLM before the predictive model and evaluation are credible

[mitulpatel123]

The first milestone slice is now in draft PR #3: a fixed T-24h prediction contract, explicit leakage exclusions, chronological evaluation split, calibration/error-slice requirements, and artifact-safety rules. Both CI runs pass.

[mitulpatel123]

Completed in PR #4. The milestone now includes official BTS data lineage, a 90,000-row chronological evaluation, logistic and fixed-budget tree baselines, validation-only threshold selection, calibration, error slices, a portable JSON model, a model card, API integration, and passing CI.