A production-grade machine learning platform that transforms raw e-commerce transactions into actionable customer intelligence. It combines RFM feature engineering, unsupervised K-Means segmentation, XGBoost churn prediction with Optuna hyperparameter tuning, SHAP explainability, MLflow experiment tracking, a FastAPI inference service, and an interactive Streamlit analytics dashboard.
- RFM Feature Engineering — Derives Recency, Frequency, Monetary, Average Order Value, Purchase Frequency Rate, and Customer Age from raw transaction data.
- Customer Segmentation — K-Means clustering with automatic elbow/silhouette analysis, PCA and t-SNE visualizations, and business-friendly segment labels (Champions, Promising, At Risk, Hibernating).
- Churn Prediction — XGBoost classifier with Optuna Bayesian hyperparameter tuning (50 trials by default), stratified train/test split, and class imbalance handling via
scale_pos_weight. - Explainability — SHAP feature-importance bar plots and beeswarm summaries generated automatically after training.
- Experiment Tracking — MLflow logs all hyperparameters, metrics, model artifacts, and diagnostic plots for every training run.
- Inference API — FastAPI service with
/predict,/segments, and/healthendpoints, CORS-enabled, and Pydantic-validated request/response schemas. - Analytics Dashboard — Four-page Streamlit app with Overview KPIs, Segmentation Explorer, Churn Diagnostics, and a Live Prediction form that calls the API in real time.
- Synthetic Data Fallback — If the UCI dataset download fails (no internet), a deterministic synthetic retail dataset is generated automatically so the pipeline can still run.
- Reproducibility — Global random seeding, deterministic Optuna studies, and consistent logging via Loguru.
- Docker Support — Multi-service
docker-composesetup for the API, dashboard, and MLflow UI.
| Layer | Technology |
|---|---|
| Language | Python 3.10+ |
| ML / Modeling | scikit-learn, XGBoost, Optuna, SHAP |
| Data Processing | pandas, NumPy, openpyxl, SciPy |
| Visualization | Plotly, Matplotlib, Seaborn |
| Experiment Tracking | MLflow |
| API | FastAPI, Uvicorn, Pydantic |
| Dashboard | Streamlit |
| HTTP Client | httpx |
| Serialization | joblib |
| Logging | Loguru |
| Testing | pytest |
| Configuration | python-dotenv |
| Containerization | Docker, Docker Compose |
+------------------------------+
| UCI Online Retail Dataset |
+--------------+---------------+
|
v
+------------------------------+
| data/download_data.py |
| Raw ingestion / synthetic |
| fallback generation |
+--------------+---------------+
|
v
+------------------------------+
| src/data_processing.py |
| Cleaning + RFM feature |
| engineering + scaling |
+--------------+---------------+
|
+-----------------+------------------+
| |
v v
+------------------------------+ +------------------------------+
| src/segmentation.py | | src/churn_model.py |
| K-Means + PCA + t-SNE | | XGBoost + Optuna + SHAP |
| Elbow / Silhouette analysis | | MLflow experiment tracking |
+--------------+---------------+ +--------------+---------------+
| |
+-----------------+-------------------+
|
v
+------------------------------+
| models/ + data/processed/ |
| Serialized artifacts (pkl, |
| json, csv) |
+-----------+------------------+
|
+--------------+---------------+
| |
v v
+------------------------------+ +------------------------------+
| api/main.py | | dashboard/app.py |
| FastAPI inference service | | Streamlit decision layer |
| /predict /segments /health | | Overview · Segments · Churn |
+------------------------------+ | · Live Prediction |
+------------------------------+
The end-to-end workflow is orchestrated by src/pipeline.py, which chains data download → cleaning → feature engineering → segmentation → churn modeling in a single make train command.
customer-segmentation/
├── data/
│ ├── download_data.py # Dataset acquisition + synthetic fallback
│ └── processed/ # Generated CSV outputs (gitignored)
├── notebooks/
│ └── exploration.ipynb # Exploratory data analysis notebook
├── src/
│ ├── __init__.py
│ ├── pipeline.py # End-to-end training orchestration
│ ├── data_processing.py # Cleaning, RFM features, scaling
│ ├── segmentation.py # K-Means clustering, PCA/t-SNE plots
│ ├── churn_model.py # XGBoost training, Optuna, SHAP, MLflow
│ └── utils.py # Paths, logging, seeding, I/O helpers
├── api/
│ ├── __init__.py
│ ├── main.py # FastAPI app with /predict, /segments, /health
│ ├── model_loader.py # Loads serialized model artifacts at startup
│ └── schemas.py # Pydantic request/response models
├── dashboard/
│ └── app.py # Streamlit multi-page analytics dashboard
├── models/ # Serialized model artifacts (.pkl, .json)
├── outputs/
│ └── plots/ # Auto-generated diagnostic PNGs
├── tests/
│ ├── test_api.py # FastAPI endpoint tests with mock registry
│ ├── test_model.py # Model loading & inference helper tests
│ └── test_processing.py # Data cleaning & feature engineering tests
├── mlflow_tracking/ # MLflow experiment store (gitignored)
├── Dockerfile # Multi-stage Python 3.11 build
├── docker-compose.yml # API + Dashboard + MLflow services
├── requirements.txt # Python dependencies
├── .env.example # Template for environment variables
├── Makefile # Developer workflow shortcuts
└── README.md
- Python 3.10+ (3.11 recommended, matches the Docker image)
- pip (or any Python package manager)
- Docker & Docker Compose (optional, for containerized deployment)
- macOS users: If using XGBoost locally, you may need
libompinstalled via Homebrew:brew install libomp export DYLD_LIBRARY_PATH=/opt/homebrew/opt/libomp/lib:$DYLD_LIBRARY_PATH
git clone <your-repo-url>
cd customer-segmentation
# Create and activate a virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install all dependencies
make install
# — or equivalently —
pip install -r requirements.txtdocker-compose up --buildThis starts three services:
| Service | URL |
|---|---|
| FastAPI | http://localhost:8000 |
| Streamlit | http://localhost:8501 |
| MLflow UI | http://localhost:5000 |
Copy the template and modify as needed:
cp .env.example .env| Variable | Default | Description |
|---|---|---|
PYTHONPATH |
. |
Ensures package imports work from the project root |
LOG_LEVEL |
INFO |
Loguru verbosity (DEBUG, INFO, WARNING, …) |
API_URL |
http://localhost:8000 |
Base URL the dashboard uses to reach the API |
MLFLOW_TRACKING_URI |
file:./mlflow_tracking |
Where MLflow stores experiment data |
OPTUNA_TRIALS |
50 |
Number of Optuna hyperparameter search trials |
Downloads data (or generates synthetic data), engineers features, segments customers, tunes and trains the churn model, and saves all artifacts:
make trainmake api
# → Uvicorn starts at http://localhost:8000 with hot reloadmake dashboard
# → Streamlit starts at http://localhost:8501| Command | What it does |
|---|---|
make install |
Install Python dependencies from requirements.txt |
make train |
Run the full training pipeline (src/pipeline.py) |
make api |
Start the FastAPI server with hot reload |
make dashboard |
Start the Streamlit dashboard |
make test |
Run the test suite with pytest |
make docker-build |
Build the Docker image |
make docker-up |
Start all services via Docker Compose |
# Terminal 1 — train models, then serve the API
source .venv/bin/activate
make train
make api
# Terminal 2 — launch the dashboard
source .venv/bin/activate
make dashboard| Page | Purpose |
|---|---|
| Overview | KPI cards (total customers, churn rate, avg CLV, model AUC), donut chart, bubble chart, RFM histograms |
| Segmentation | PCA scatter projection, segment size bar chart, comparison table |
| Churn Analysis | SHAP importance plots, churn rate by segment, interactive ROC curve |
| Predict | Live risk-scoring form → calls /predict → shows segment, risk level, churn gauge, and recommended actions |
Once the API server is running, interactive docs are available at:
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
Returns service health status.
{ "status": "ok" }Returns summary metrics for each customer segment.
[
{
"segment": "Champions",
"size": 420,
"avg_recency": 18.5,
"avg_frequency": 7.2,
"avg_monetary": 1250.30,
"churn_rate": 0.05
}
]Predict churn probability and receive segment assignment with business recommendations.
Request body:
{
"recency": 30,
"frequency": 5,
"monetary": 250.0
}| Field | Type | Constraints | Description |
|---|---|---|---|
recency |
int |
>= 0 |
Days since the customer's last purchase |
frequency |
int |
>= 1 |
Number of distinct orders placed |
monetary |
float |
> 0 |
Total monetary value spent (GBP) |
Response:
{
"segment": "Promising",
"churn_probability": 0.3421,
"risk_level": "Low",
"recommendations": [
"Nudge with personalized cross-sell offer",
"Send product education sequence"
]
}| Field | Type | Description |
|---|---|---|
segment |
string |
Champions, Promising, At Risk, or Hibernating |
churn_probability |
float |
0.0 – 1.0 probability of churning |
risk_level |
string |
Low (< 0.4), Medium (0.4 – 0.7), High (≥ 0.7) |
recommendations |
string[] |
Actionable retention strategies for the segment |
The project uses pytest with three test modules:
make test
# — or —
pytest -q| Test file | Coverage area |
|---|---|
test_processing.py |
Transaction cleaning, RFM feature engineering, scaling |
test_model.py |
Inference frame construction, model bundle serialization roundtrip |
test_api.py |
/health, /segments, /predict endpoints with a mock registry |
| Issue | Fix |
|---|---|
ModuleNotFoundError: No module named 'src' |
Make sure PYTHONPATH=. is set, or run commands from the project root. The .env file handles this automatically if loaded. |
| XGBoost crashes on macOS (libomp) | Install OpenMP: brew install libomp and export DYLD_LIBRARY_PATH=/opt/homebrew/opt/libomp/lib:$DYLD_LIBRARY_PATH. |
| Dashboard shows "Missing processed data file" | Run make train before make dashboard. The dashboard requires data/processed/clustered_customers.csv and model artifacts. |
/predict returns 503 |
The API could not load model artifacts. Run make train to generate them, then restart the API server. |
| Dataset download fails (network error) | The pipeline automatically generates a synthetic dataset as a fallback. No action needed — training will still complete. |
| MLflow UI is empty | Ensure MLFLOW_TRACKING_URI points to the correct path (default: file:./mlflow_tracking). Re-run make train to create a new experiment run. |
| Port already in use | Kill existing processes: lsof -ti :8000 | xargs kill (for the API) or :8501 (for the dashboard). |
- Fork the repository and create a feature branch from
main. - Install dependencies and run the full pipeline to confirm baseline behavior.
- Write tests for any new functionality in the
tests/directory. - Follow existing patterns — Loguru for logging, Pydantic for schemas, dataclasses for data containers.
- Run the test suite before submitting:
make test - Submit a Pull Request with a clear description of what changed and why.
This project is released for educational and portfolio purposes. If you plan to use it commercially, please check the licensing terms of the UCI Online Retail Dataset and each dependency listed in requirements.txt.