Urban Vision AI

AI-Based Civic Issue Reporting & Routing System

SmartCity Sentinel is a modern civic engagement platform that empowers citizens to report urban maintenance issues (like potholes, broken streetlights, fallen trees, and garbage piles) simply by taking a photo. Using an integrated Custom Convolutional Neural Network (CNN) trained in TensorFlow, the platform automatically classifies the issue, assigns a priority, routes it to the responsible municipal department, and checks for duplicates to streamline civic operations.

---

Architecture & Workflow

The project consists of three main components:

1. Frontend: React-based Single Page Application (SPA) with interactive maps, citizen dashboards, and analytics.
2. Backend: Node.js & Express REST API managing authentication, notifications, geospatial queries, and duplicate checks.
3. Machine Learning Model: Python-based inference engine executing a custom CNN image classifier.

graph TD
    User([Citizen/Admin]) -->|Uploads Image & Location| Client[React Frontend]
    Client -->|API Request: POST /api/predict| Server[Node.js Express Server]
    Server -->|Saves Upload & Spawns Python Process| Python[predict_image.py]
    Python -->|Loads CNN Model & Runs Inference| Model[urban_issues_cnn_model.h5]
    Model -->|Predicts Class Confidences| Python
    Python -->|Returns JSON Results| Server
    Server -->|Matches Department & Priority| Server
    Server -->|Sends AI Suggestions| Client
    Client -->|Confirm & Submit Complaint| Server
    Server -->|Checks Location Proximity & Image Hash| DB[(MongoDB)]
    Server -->|Saves to DB or Auto-Upvotes| DB
    Server -->|Real-Time Status & Admin Portal| User

Loading

---

Key Features

• AI-Powered Image Classification: Analyzes uploaded photos on-the-fly. The custom CNN model is trained on a variety of urban issue classes, automatically extracting labels, confidence levels, and severity.
• Auto-Routing & Prioritization: Automatically routes complaints to target municipal departments (e.g., Road Department, Sanitation, Electricity) and assigns severity levels (High, Medium, Low) based on classification.
• Proactive Duplicate Detection: Prevents double-reporting by combining two strategies:
  • Perceptual Image Hashing: Generates an 8x8 grayscale dHash of the image to check for similar visual features.
  • Geospatial Proximity: Computes distance to check if a similar complaint exists within 100 meters.
  • If a duplicate is found, the system automatically upvotes the existing complaint on behalf of the user and redirects them to its details page.
• Interactive GIS Map View: Rendered with Leaflet and react-leaflet to display geolocated complaints, color-coded by issue status and priority.
• Citizen Verification Flow: Citizens can upvote existing issues to escalate priority, and verify reported resolutions ("Resolved" or "Still Broken") to build community accountability.
• Analytics & Admin Dashboard: Interactive graphs made with Recharts that summarize municipal workloads, issue distributions, and resolution times.

---

Project Directory Structure

smartcity-sentinel/
├── client/                     # React Frontend Application
│   ├── public/                 # Static assets & HTML entry point
│   └── src/
│       ├── components/         # Reusable UI components (Navbar, ProtectedRoutes, etc.)
│       ├── context/            # AuthContext for login session management
│       ├── hooks/              # API custom hook for Axios endpoints
│       ├── pages/              # Primary views (Dashboard, Submit Form, Global Feed)
│       ├── App.js              # Application routing & layout configurations
│       └── index.css           # Premium Custom styling system
├── server/                     # Express Backend REST API
│   ├── config/                 # Database connection configurations
│   ├── controllers/            # Request handlers (Complaints, Predict, Analytics)
│   ├── middleware/             # Express middlewares (Auth guard, Multer uploads)
│   ├── models/                 # Mongoose schemas (User, Complaint, Notification)
│   ├── routes/                 # API endpoint routers
│   ├── utils/                  # Helper modules (perceptual hashing, keyword classifiers)
│   └── index.js                # Server entry point
├── model/                      # Python TensorFlow CNN Classifier
│   ├── processed_dataset/      # Prepared data directories
│   ├── labels.json             # Classification classes & issue maps
│   ├── predict_image.py        # Inference script invoked by backend child processes
│   └── urban_issues_cnn_model.h5 # Pre-trained Keras model weights
├── render.yaml                 # Infrastructure configuration for Render Cloud
├── render-build.sh             # Custom build automation script for deployment
├── requirements.txt            # Python dependencies (TensorFlow, Pillow, NumPy)
└── package.json                # Root dependency manager & concurrently script orchestrator

---

Local Development & Setup

Prerequisites

• Node.js (v16.x or newer)
• npm (v8.x or newer)
• Python (3.8 - 3.11 recommended for TensorFlow compatibility)
• MongoDB (Local instance or MongoDB Atlas cluster connection string)

1. Repository Setup & Dependencies

Install the dependencies for the root, frontend, backend, and machine learning components with a single command:

npm run install:all

2. Python Virtual Environment

Initialize a local Python virtual environment to install packages required for image classification:

On Windows:

python -m venv venv
.\venv\Scripts\activate
pip install -r requirements.txt

On macOS/Linux:

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

3. Backend Environment Variables

Create a file named .env inside the server/ directory:

PORT=5000
MONGODB_URI=mongodb://localhost:27017/smartcity_sentinel
JWT_SECRET=your_production_secret_key_min_32_characters_long
NODE_ENV=development

# Cloudinary credentials for hosting complaint images
CLOUDINARY_CLOUD_NAME=your_cloudinary_cloud_name
CLOUDINARY_API_KEY=your_cloudinary_api_key
CLOUDINARY_API_SECRET=your_cloudinary_api_secret

4. Frontend Environment Variables

Create a file named .env inside the client/ directory:

REACT_APP_API_URL=http://localhost:5000

5. Running the Application

Run both the frontend and backend servers concurrently using:

npm run dev

• Frontend will open at http://localhost:3000
• Backend API will run at http://localhost:5000

---

AI Model Details

The model is a Convolutional Neural Network (CNN) classifying 9 specific categories mapping to 7 issue types:

Image Class (CNN Class)	Predicted Issue Type	Target Department	Default Priority
Damaged concrete structures	building	Building & Infrastructure	Low
DamagedElectricalPoles	electric	Electricity Department	High
DamagedRoadSigns	road	Road Department	High
DeadAnimalsPollution	garbage	Sanitation Department	Medium
FallenTrees	tree	Forest & Municipal	Low
Garbage	garbage	Sanitation Department	Medium
IllegalParking	road	Road Department	High
Potholes and RoadCracks	pothole	Road Department	High

Note: If the CNN predicts other with a confidence score of $\le$ 30%, the system throws a warning requesting a clearer photo of the issue to filter out spam or irrelevant pictures.