Brief: Eco-Robo-Sort is a prototype autonomous waste-sorting robot that uses a Convolutional Neural Network (DetectNet / SSD-MobileNet-V2) running on an NVIDIA Jetson Nano to detect and classify waste items from a camera feed and then actuates an Arduino-controlled mechanism (motors/servos) to sort detected items.
- Real-time object detection on NVIDIA Jetson Nano using SSD-MobileNet-V2 (DetectNet implementation).
- Hardware control via an Arduino Mega for motors, servos and ultrasonic distance sensing.
- Designed for research, prototyping and demonstration of embedded vision + robotics for waste-sorting.
detectnet_20210723_USB.py— Inference and robot control glue (Python, runs on Jetson Nano).test.ino— Example Arduino sketch for motor/servo control and ultrasonic sensor (C++).- (You will add photos, videos and dataset/model files as needed.)
- NVIDIA Jetson Nano (main compute / inference)
- Arduino Mega (hardware control: motors, servos, sensors)
- Camera module (compatible with Jetson; USB or CSI depending on your build)
- Ultrasonic range sensor (for distance-based triggering)
- DC motors + motor driver (for conveyor / actuator)
- Servos (for sorting gate / flipper)
- Jetson-side (Python): SSD-MobileNet-V2 model via the DetectNet implementation from the
jetson-inferenceproject (https://github.com/dusty-nv/jetson-inference). - Uses
jetson.utils.videoSourcefor camera capture and thejetson.inferencedetection APIs (or equivalent bindings) for inference. - Arduino-side (C++): Sketch for controlling motors, servos and reading ultrasonic sensor.
The CNN/detection pipeline is based on DetectNet from the jetson-inference project by dustynv (NVIDIA). Please see:
Follow the original project's instructions for obtaining trained models, converting weights and using their deployment utilities. This project uses the SSD-MobileNet-V2 architecture as the detection backbone.
These are concise steps — follow the linked detailed Jetson docs if you're not familiar with Jetson Nano setup.
- Flash JetPack (recommended) onto your Jetson Nano SD card and complete initial setup (https://developer.nvidia.com/embedded/jetpack).
- Install the
jetson-inferenceproject and its Python bindings per the repository instructions. That provides the DetectNet/SSD code and model support. - Connect your camera and verify it as a
videoSource(USB or CSI). Test withjetson-vieweror a small test script. - Copy
detectnet_20210723_USB.pyto your Jetson, install Python dependencies (if any), and ensure your model files are accessible. - Upload
test.ino(or your Arduino sketch) to the Arduino Mega. Wire the Arduino to the Jetson (USB) for serial commands.
Example (illustrative) commands you might run on the Jetson Nano:
# clone the jetson-inference repo (follow its build steps)
git clone https://github.com/dusty-nv/jetson-inference.git
# run inference script (adjust args to your device and model path)
sudo python3 detectnet_20210723_USB.py --input /dev/video0 --model <path-to-model> --labels <labels.txt>Note: The exact flags and invocation depend on how detectnet_20210723_USB.py was written; inspect the script for supported CLI options.
- Camera -> Jetson Nano (USB or CSI)
- Arduino Mega -> Jetson Nano (USB serial) — Jetson sends simple commands/packets to trigger motors/servos
- Ultrasonic sensor -> Arduino input (used for distance detection; Arduino reports status over serial)
- Motors/Servos -> Motor driver / servo power supply -> controlled by Arduino PWM/Digital pins
Keep power supplies separate and appropriate for motors/servos to avoid drawing from Jetson/Arduino 5V rails.
- Jetson captures frames from the camera using
jetson.utils.videoSource. - Each frame is passed to the DetectNet / SSD-MobileNet-V2 model for detection and classification.
- When a target object is detected with confidence above a threshold, Jetson sends a command over serial to the Arduino indicating which actuator action to perform (e.g., gate left, gate right, accept, discard).
- Arduino executes the servo/motor sequence, and optionally reports back sensor data or completion status.
- If you need custom classes or better accuracy, create or augment a dataset of labelled waste items and retrain or fine-tune a MobileNet-SSD model. The
jetson-inferencerepo contains training and conversion utilities. - When changing model input sizes or class labels, update the inference script and the labels file expected by the Python code.
Place images in a docs/images/ folder (or media/) and add YouTube links below. Example Markdown placeholder:
Watch the demo: https://youtu.be/yKOSB_V62G0
- Low FPS: Lower model input size, reduce inference frequency, or use TensorRT-optimized models.
- Camera not found: Verify
v4l2-ctl --list-devicesand that the device node (e.g.,/dev/video0) is accessible. - Serial communication issues: Ensure correct USB port, baud rate, and that Arduino isn't resetting frequently (add 100nF cap between RESET and DTR if needed during programming/debugging).
- Power issues/servo jitter: Use a dedicated power supply for motors/servos and common ground between Arduino and Jetson.
- Create a small test harness that feeds known images to the detection script and verifies expected labels and serial commands.
- Record inference timings to validate throughput on the Jetson Nano.
- Add dataset and training notebook for custom waste categories.
- Add an automated calibration routine for servo positions and gate timings.
- Implement logging of detections and sorting outcomes for accuracy measurement.
This repository includes code written by you and uses models/tools from the jetson-inference project. Respect the original licenses (MIT / Apache as indicated in their repo). Add your preferred license file here (e.g., LICENSE) and keep third-party license notices intact.
- DetectNet / jetson-inference by dustynv (NVIDIA) — https://github.com/dusty-nv/jetson-inference
For questions or collaboration, add your name and contact info here.