This repository contains the Deep Learning and behavior-decision subproject for a NUS Deep Learning and robotics summer-school demo. The current directory is the project-structure root, even when it is checked out inside a larger course workspace. All project plans, logs, rules, code, tests, and documentation are relative to this directory.
Work is split into two sequential phases:
Official Baseline (now, due 17 July)
robot stream -> five-breed cat classifier -> visible console/UI -> census
|
v reuse proven infrastructure
DeskMate Advanced (only after Baseline Gate B4)
multi-expert perception -> WorldState -> FSM -> safety gate -> robot adapter
The current P0 is the official Great Cat Census: fine-tune the Ultralytics
yolo26s-cls.pt whole-image classifier for Ragdoll, Singapura, Persian, Sphynx,
and Pallas cat plus an internal not_target rejection output; run it on
quality-gated, operator-aligned multi-scale ROIs from the live robot camera;
calibrate and aggregate correlated probabilities; print only confirmed target
species; and support the remotely piloted 15-minute course. Baseline does not
train an object detector or implement autonomous search. Model selection,
calibration, and untouched robot final testing use separate data. Advanced work
is paused until the Baseline passes three complete rehearsals and its offline
release is frozen.
Both phases target the local NVIDIA RTX 4070 plus CPU and must remain usable
without cloud inference. Advanced reuses the Baseline's Ultralytics/PyTorch
toolchain, bounded-queue frame capture, reconnect, model packaging, generic
ModelRunner lifecycle, UI, telemetry, replay tests, and robot video
configuration. Cat classification
weights, heads, labels, datasets, thresholds, and Results.probs are not reused
as detection assets.
Baseline Gate B0 is NOT PASSED. The software and data-pilot checks are complete. The requested camera profile is now 480 x 480 JPEG quality 85 at 8 FPS, upright and not mirrored, with OpenCV BGR output after decode. Two human inputs remain: one consented frame from the actual robot camera and its delivery protocol/endpoint. The exact collection steps and ownership blanks are exposed in the Phase 0 manual-action dashboard.
Gate B1 human image review is now IN PROGRESS. The latest machine audit still has 2,321 pending candidates and zero accepted images until reviewer decisions are written back. This is a separate dataset-freeze blocker, not a third Gate B0 failure.
- High-level plan index: authority order and document scope.
- Official Baseline plan: five-cat data, model, robot-stream integration, three-day schedule, report evidence, and gates.
- DeskMate Advanced plan: post-Baseline multi-expert perception, semantic fusion, decisions, safety, and schedule.
- Advanced dataset sourcing: post-Baseline desk-object data-source decisions.
- Advanced dataset download plan: post-Baseline acquisition and local storage workflow.
- Formal requirement transcriptions: local evidence for the assignment, evaluation SOP, announcement, and answer book.
- Repository Agent rules: model, data, validation, robot-safety, and commit policy.
- Repository workflow: target plans and factual maintenance logs.
- Phase 0 manual-action dashboard: open robot evidence for B0 and the separate human image-review queue for B1.
From this /project root:
- Run
git status --short --branchand audit existing dirty files. - Create
plan/<YYYY-MM-DD-target-name>/plan.mdfrom the repository template. - Declare owned and read-only paths before editing.
- Implement only the bounded target.
- Run target-specific validation plus
git diff --checkandgit status --short --branch. - Record the factual outcome in
plan/log.md. - Commit or push only when explicitly requested.
This directory is the independent chenzc24/DeskMate repository and is tracked
as the project Git submodule by the containing course repository. The parent
checkout remains outside this project's workflow boundary. Large datasets,
downloaded model weights, private videos, and training outputs are local
artifacts and must not be committed.