CogNitAI is a machine learning–powered web application that estimates an adult’s risk of scoring in the lowest decile on standardized cognitive tests, using 16 self-reported health and lifestyle answers. It combines a logistic regression model trained on NHANES survey data with a questionnaire-based interface, served through Flask.
It is a demonstration of an end-to-end ML pipeline, not a diagnostic instrument. See Limitations.
Early detection of cognitive decline is critical for timely intervention. CogNitAI explores a narrower question: how much signal do self-reported questionnaire answers actually carry, and what does it take to measure that honestly?
The project demonstrates an end-to-end machine learning pipeline, from data preprocessing and exploratory analysis to model training and deployment in a web environment.
- Source: NHANES 2011–2012 and 2013–2014 public questionnaire files
- Cohort: 2,934 participants aged 60–80 who completed all four cognitive assessments
- Label: composite of four z-scored test scores — CERAD trial 1, CERAD delayed recall, Digit Symbol Substitution, and Animal Fluency. Participants in the bottom 10% of that composite are the positive class.
- Features: 16 questionnaire items covering age, sex, race/ethnicity, household income, sleep, blood pressure, cholesterol, diabetes, sedentary time, chest pain, mood, weight history, cancer history and self-rated diet.
The label is a proxy for low measured cognitive performance, not a clinical diagnosis of decline.
Evaluated on a held-out set of 587 participants, split so that no participant appears in both training and test data.
| Metric | Logistic Regression | KNN (k=2) |
|---|---|---|
| ROC AUC | 0.837 | 0.595 |
ROC AUC is the comparison metric here because it does not depend on where the decision threshold is placed. Accuracy is actively misleading on this problem: the positive class is only ~10% of the data, so a model that predicts "no decline" for everyone scores 89.9%.
Logistic regression is the deployed model. At the app's decision threshold of 0.125:
| Cases caught | 44 of 59 (recall 0.75) |
| Users flagged | 149 of 587 (25%) |
| Precision | 0.30 |
| Flagged per true case | ~3.4 |
| Accuracy | 0.796 |
That accuracy is below the 89.9% always-negative baseline, and the model is nonetheless far more useful — it finds 44 cases where guessing "no" finds none. This is the clearest illustration of why accuracy was the wrong metric to optimise.
The threshold is deliberately well below 0.5. Only ~10% of the population carries the label, so a 0.5 cut-off flagged just 1.5% of users and caught 6 of 59 cases. DECISION_THRESHOLD in app.py controls this trade-off in one line.
Calibration. The predicted probabilities are reliable, which is what makes the displayed percentage meaningful:
- Brier score 0.078 (vs 0.090 for always predicting the base rate)
- Mean predicted risk 0.103 against an observed rate of 0.101
Two findings shaped the final pipeline and are worth recording:
Participant-level leakage. NHANES RXQ_RX stores one row per prescription, so a naive left merge duplicated each participant once per medication — 12,004 rows for 2,934 people. Splitting rows rather than people put 91.9% of test rows in the training set as well, and KNN with n_neighbors=2 simply retrieved each person's own duplicate. That produced an apparent AUC of 0.978, which fell to 0.508 (chance) once participants were confined to one side of the split. The pipeline now collapses long-format sources before merging and asserts that no participant crosses the split.
Missing-value codes. NHANES encodes "Refused" and "Don't know" as 7/9, 77/99 or 7777/9999 rather than as blanks. Treated as ordinary categories, four of the model's top ten coefficients were these codes — it was partly learning who declined to answer. They are now mapped to NaN before imputation.
Continuous features (RIDAGEYR, PAD680) are scaled rather than one-hot encoded, so age contributes one ordered coefficient (+0.78 per SD) instead of 21 independent dummies.
- Interactive web-based questionnaire for user input
- Calibrated probability output, not just a binary verdict
- Server-side validation of the supported age range
- Clean Flask-based backend and templated frontend
- Integrated data preprocessing and feature engineering pipeline
- Not a diagnostic tool. The label is low performance on cognitive tests, not a clinical diagnosis, and the model has no access to medical records or examination data.
- Ages 60–80 only. NHANES administers these tests to adults 60+, and codes age 80 as "80 or older". The app refuses younger ages rather than extrapolating.
- Precision is 0.30. Roughly two thirds of flagged users are false alarms. The tool is a prompt to seek professional assessment, not a verdict.
- Cross-sectional data. Participants were measured once, so the model captures association, not progression over time.
- Age dominates. Much of the model's discrimination comes from age alone; the remaining questionnaire items add comparatively little.
All exploratory data analysis, preprocessing, and model training were conducted in model.ipynb.
The notebook includes:
- Confusion matrices for both candidate models
- Logistic regression coefficient rankings
- Permutation importance for feature contribution
- Calibration curve with Brier score and predicted-risk distribution
These insights informed feature selection and model optimization prior to deployment.
- Python
- Flask
- Scikit-learn
- Pandas / NumPy
- HTML / CSS (Jinja templates)
- Jupyter Notebook
CogNitAI/
│
├── app.py # Flask backend
├── model.ipynb # Model training + EDA
├── models/ # Saved ML pipelines (.sav)
├── data/ # Dataset files (raw/ and processed/)
├── Static/ # CSS / JS assets
├── Templates/ # HTML pages (index.html, output.html)
└── requirements.txt
Requires Python 3.9–3.12. The pinned dependency versions have no prebuilt wheels for Python 3.13+, so newer interpreters will try (and fail) to compile NumPy from source.
On macOS, xgboost also needs the OpenMP runtime, which pip does not install:
brew install libompThis is only needed to run model.ipynb — the Flask app itself does not use XGBoost.
# Clone repository
git clone https://github.com/BlueWhaleGirl/CogNitAI.git
# Enter directory
cd CogNitAI
# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# Run application
python app.pyThe app starts on http://127.0.0.1:5000.