An end-to-end machine learning regression project using the Ames Housing dataset to predict residential property sale prices.
The project covers exploratory data analysis, context-aware missing-value treatment, categorical encoding, baseline modelling, regularisation, Gradient Boosting, hyperparameter tuning, feature-importance analysis and deployment through an interactive Streamlit application.
House-price prediction is a regression problem involving a mixture of numerical, nominal and ordinal property characteristics.
The objective of this project was not simply to train a predictive model, but to develop a structured machine learning workflow that addressed:
- exploratory data analysis;
- missing-value interpretation and treatment;
- ordinal and nominal categorical encoding;
- baseline model development;
- model comparison and regularisation;
- hyperparameter tuning with cross-validation;
- held-out model evaluation;
- feature-importance analysis;
- prediction on unseen test data; and
- deployment through Streamlit.
The final tuned Gradient Boosting model achieved an RMSE of $26,532.18 and an R² of 0.9082 on the held-out validation data.
Try the deployed House Price Prediction application:
The application allows users to adjust key property characteristics and generate an estimated sale price using the trained Gradient Boosting model.
This project uses the Ames Housing dataset provided through Kaggle's House Prices: Advanced Regression Techniques competition.
Dataset source: House Prices: Advanced Regression Techniques — Kaggle
The dataset describes residential properties in Ames, Iowa and contains a mixture of numerical and categorical variables covering property quality, size, location, construction, basement, garage and other housing characteristics.
The training dataset contains:
- 1,460 properties
- 81 original columns
- 38 numerical variables
- 43 categorical variables
- Target variable:
SalePrice
The accompanying data_description.txt file provides detailed definitions of the dataset's variables and category codes.
Initial analysis found no duplicate rows or duplicate property IDs.
The average property sale price was approximately $180,921, while the median was $163,000.
The distribution was positively skewed, with most properties concentrated between approximately $100,000 and $250,000 and a smaller number of substantially higher-value properties.
Missing values were not treated with a single blanket imputation strategy.
Instead, missingness was investigated according to the meaning of each variable.
For categorical variables where a missing value represented the absence of a physical feature, such as a garage, basement, fireplace or pool, an explicit No_feature category was introduced.
Examples included:
PoolQCFireplaceQuGarageTypeGarageFinishGarageQualBsmtQualBsmtCondBsmtExposure
Other strategies included:
Electrical— mode imputation;MasVnrArea— zero where no masonry veneer existed;GarageYrBlt— zero used as a sentinel where no garage existed;LotFrontage— imputed using the median frontage for the property's neighbourhood.
The cleaned dataset was validated to ensure that no missing values remained.
Categorical variables were separated according to whether their values represented a meaningful order.
Nineteen ordinal variables were manually encoded.
Where a property feature could genuinely be absent, 0 was reserved for No_feature, with existing categories ranked from lowest to highest quality.
Examples included:
ExterQualKitchenQualBsmtQualBsmtExposureGarageFinishFireplaceQuPoolQC
Nominal variables without a defensible numerical ordering were transformed using one-hot encoding.
MSSubClass was also converted from an integer into a categorical variable because its values represent dwelling classifications rather than measurable quantities.
Following encoding, the dataset expanded from 81 to 245 columns, with 243 predictors used for modelling after removing SalePrice and Id.
Validation confirmed:
- no remaining object columns;
- no missing values; and
- valid one-hot encoded category structures.
A Linear Regression model was used to establish the initial benchmark.
The encoded training data was divided into:
- 80% training data
- 20% held-out validation data
random_state=42
| Metric | Result |
|---|---|
| RMSE | $30,571.89 |
| R² | 0.8781 |
The model explained approximately 87.8% of the observed variation in sale prices within the held-out sample.
Analysis of actual versus predicted prices showed stronger performance across lower and middle price ranges, with larger errors appearing among some high-value properties.
Three additional approaches were evaluated against the same held-out data:
- Ridge Regression
- Lasso Regression
- Gradient Boosting Regression
Ridge and Lasso were standardised before training because their regularisation penalties are sensitive to feature scale.
| Model | RMSE | R² |
|---|---|---|
| Linear Regression | $30,571.89 | 0.8781 |
| Ridge Regression | $29,858.92 | 0.8838 |
| Lasso Regression | $29,298.95 | 0.8881 |
| Initial Gradient Boosting | $27,898.74 | 0.8985 |
| Tuned Gradient Boosting | $26,532.18 | 0.9082 |
Lasso reduced 66 of the 243 coefficients to zero, effectively removing approximately 27.2% of the predictors.
Gradient Boosting produced a larger improvement, indicating that non-linear relationships and interactions between property characteristics provided additional predictive value.
Gradient Boosting was tuned using GridSearchCV with five-fold cross-validation on the training data.
The search evaluated combinations of:
- number of estimators;
- learning rate;
- maximum tree depth; and
- minimum samples per leaf.
A total of 81 parameter combinations were evaluated across five folds.
The selected parameters were:
GradientBoostingRegressor(
n_estimators=300,
learning_rate=0.05,
max_depth=4,
min_samples_leaf=1,
random_state=42
)The tuned model achieved:
RMSE: $26,532.18
R²: 0.9082
Compared with the original Linear Regression baseline, this represented an approximately 13.2% reduction in RMSE.
The tuned Gradient Boosting model was therefore selected as the final model.
Feature importance from the selected Gradient Boosting model was analysed to identify the property characteristics used most heavily by the model.
| Rank | Feature | Importance |
|---|---|---|
| 1 | OverallQual | 0.5147 |
| 2 | GrLivArea | 0.1355 |
| 3 | GarageCars | 0.0400 |
| 4 | BsmtFinSF1 | 0.0327 |
| 5 | TotalBsmtSF | 0.0313 |
| 6 | 1stFlrSF | 0.0269 |
| 7 | 2ndFlrSF | 0.0257 |
| 8 | ExterQual | 0.0213 |
| 9 | LotArea | 0.0178 |
| 10 | KitchenQual | 0.0178 |
OverallQual was the dominant predictor, followed by above-ground living area.
Together, the two strongest features accounted for approximately 65.0% of total model feature importance, while the top ten accounted for approximately 86.4%.
The results indicate that the model relied particularly heavily on two broad property characteristics:
quality and usable space.
Feature importance represents how strongly the fitted model used a predictor. It does not establish causation or the monetary effect of changing an individual characteristic.
After model selection, the tuned Gradient Boosting configuration was retrained using all 1,460 labelled training observations.
The Kaggle test.csv dataset contained 1,459 properties without SalePrice.
The same preprocessing logic was applied to the test data, including:
- structural missing-value treatment;
- training-derived imputations;
- ordinal mappings;
- one-hot encoding; and
- feature alignment.
The test feature matrix was aligned to the exact 243-feature structure expected by the trained model.
The final model generated predictions for all 1,459 unseen properties.
Prediction summary:
| Statistic | Predicted Sale Price |
|---|---|
| Minimum | $27,768.64 |
| Mean | $178,797.64 |
| Maximum | $605,837.73 |
The mean predicted value was close to the training dataset mean of approximately $180,921.
As an additional deployment stage, the final Gradient Boosting model was integrated into an interactive Streamlit application.
The application allows a user to adjust ten of the most influential property characteristics:
- Overall property quality
- Above-ground living area
- Garage capacity
- Finished basement area
- Total basement area
- First-floor area
- Second-floor area
- Exterior quality
- Lot area
- Kitchen quality
The trained model still expects all 243 predictors. Rather than assigning arbitrary zeros to the remaining predictors, the application uses a genuine representative property from the training dataset as its baseline.
The baseline property was selected as the observation whose sale price was closest to the training median of $163,000.
The application then:
- loads the complete 243-feature baseline;
- replaces the ten exposed characteristics with user selections;
- preserves the exact model feature structure;
- generates a prediction using the trained Gradient Boosting model; and
- compares the prediction against the model's baseline estimate.
This approach provides a usable demonstration interface while retaining a valid encoded feature structure.
The application is intended as a machine learning demonstration rather than a professional property valuation system.
Key limitations include:
- The model was trained on the Ames Housing dataset and should not be assumed to generalise to other housing markets without retraining.
- Only ten characteristics are directly controlled through the Streamlit interface.
- Characteristics not exposed in the application retain the values of the representative baseline property.
- The held-out validation RMSE was approximately $26,532, meaning individual predictions can differ materially from observed sale prices.
- Feature importance does not establish causal relationships.
- Manual ordinal encoding imposes ordered numerical representations that may not capture every relationship equally well.
- A production system would benefit from a unified preprocessing and modelling pipeline fitted exclusively within training folds.
House Price Prediction/
│
├── app/
│ └── app.py
│
├── dataset/
│ ├── data_description.txt
│ ├── test.csv
│ ├── train.csv
│ ├── train_cleaned.csv
│ └── train_encoded.csv
│
├── models/
│ ├── baseline_features.pkl
│ ├── house_price_gradient_boosting.pkl
│ └── model_features.pkl
│
├── notebook/
│ ├── 01_exploring-the-data.ipynb
│ ├── 02_missing-values.ipynb
│ ├── 03_categorical-encoding.ipynb
│ ├── 04_baseline-regression.ipynb
│ ├── 05_model-iteration.ipynb
│ ├── 06_feature-importance.ipynb
│ └── 07_final-model-deployment.ipynb
│
├── screenshots/
│ ├── 01_distribution.png
│ ├── 02_lot-property-dist.png
│ ├── 03_regression.png
│ ├── 04_comparing-regression-models.png
│ ├── 05_top-10-features.png
│ └── 06_streamlit_app.png
│
└── README.md
- Python
- pandas
- NumPy
- scikit-learn
- Matplotlib
- Jupyter Notebook
- Streamlit
- Joblib
- Visual Studio Code
- Exploratory Data Analysis
- Missing-value analysis and imputation
- Feature engineering
- Ordinal encoding
- One-hot encoding
- Train/test splitting
- Linear Regression
- Ridge Regression
- Lasso Regression
- Gradient Boosting Regression
- Hyperparameter tuning
- GridSearchCV
- Cross-validation
- RMSE and R² evaluation
- Feature importance
- Model serialisation
- Interactive model deployment
This project demonstrated the importance of treating preprocessing as part of the modelling problem rather than simply applying generic transformations.
Context-aware treatment of missing values preserved information about absent property features, while separating ordinal and nominal variables prevented inappropriate numerical assumptions during categorical encoding.
Model iteration demonstrated progressively stronger performance, with the tuned Gradient Boosting model reducing RMSE by approximately 13.2% compared with the Linear Regression baseline.
The feature-importance analysis also produced an interpretable business conclusion: overall property quality and usable living space were the dominant characteristics used by the model when estimating sale prices.
Finally, deploying the model through Streamlit extended the project beyond notebook-based analysis into an interactive prediction application while highlighting the practical challenges of maintaining preprocessing and feature consistency during deployment.
This project is licensed under the MIT License. See the LICENSE file for details.
The Ames Housing dataset used in this project is sourced separately through the Kaggle House Prices: Advanced Regression Techniques competition and remains subject to the applicable dataset and competition terms.