RFC: external per-joint target streaming (robot.setJoints) - #169
Draft
cagataycali wants to merge 7 commits into
Draft
RFC: external per-joint target streaming (robot.setJoints)#169cagataycali wants to merge 7 commits into
cagataycali wants to merge 7 commits into
Conversation
Scoping RFC for an external per-joint-write path: a robot.setJoints intent + an External drive mode that uses streamed joint targets in place of the on-device policy, all through the existing duck-control Safety chokepoint. Closes the safety.rs actuator-travel-vs-anatomical-limit gap, adds a per-tick step clamp and an external deadman. Unlocks the strands-robots passthrough driver + running off-robot/mid-iteration RL policies on the real robot. Distinct from the updater model-deploy path. Draft for Pollen review.
Add the RFC's protocol layer: ROBOT_SET_JOINTS method, Call::RobotSetJoints
and SetJointsParams { targets: Vec<f64> (len NUM_JOINTS), gain: Option<u16> }.
Wired through method(), params(), destination() (Robot/Prompt) and parse(),
which rejects targets.len() != JOINT_NAMES.len() by name with INVALID_PARAMS.
A notification like robot.move (no id). Covered by every_call() plus exact-wire
and length-rejection round-trip tests. cargo test -p duck-ipc-proto green (51).
Safety::apply_external — the sink for the RFC's external joint stream, through the same single motor-write chokepoint as apply(), adding the guarantees raw off-robot targets need: - external deadman (SafetyConfig.external_deadman, 150ms; tighter than the 500ms twist deadman): a stalled controller holds the fallback pose and drops to limp - per-joint anatomical limit table ANATOMICAL_LIMITS (conservative placeholders, TODO source from the MJCF) — closes the actuator-travel-vs-anatomical gap the module admitted at safety.rs:42-46; clamps external targets to real bounds - per-tick step clamp (SafetyConfig.external_max_step, 0.2 rad) vs the last external target, so one bad frame cannot snap a joint - NaN/inf refused outright (delegated to apply, holds — never clamped) New Limit::Step; clear_external() drops the step baseline on mode exit. 8 unit tests vs FakeIo (passthrough, NaN, out-of-limit clamp, over-fast step, stale=hold+limp, external<twist deadman, baseline reset, home-in-limits). robotd: limit_name gains the Step arm; SafetyConfig ctor takes external defaults via ..default() (dedicated params are a robotd-params follow-up). cargo test -p duck-control green (68), clippy clean.
…nal) The RFC's 'Runtime' is robotd's Controller (robotd/src/control.rs), which turns a Command into a Step of joint targets. Add the External path as Step::external: in External mode the whole policy stage (observe -> infer -> scatter -> scale -> low-pass) is skipped and the tick IS the off-robot controller's supplied targets, carried through unchanged for the safety chokepoint (Safety::apply_external) to clamp downstream. Labelled 'external', carries the caller's gain, busy=false; holds no IO handle like every producer here. main.rs wires it in Step 4 (hence a temporary #[allow(dead_code)]). Also: adding Call::RobotSetJoints in Step 1 broke updater's exhaustive dispatch match — add it to the 'robot.* is served by robotd, not updaterd' group so the robotd build (which depends on updater) compiles. robotd builds AND tests on macOS: cargo test -p robotd control:: green (4), including the_external_branch_returns_the_supplied_targets. (mediad/btd/padd route tables also match Call exhaustively and are updated in the workspace-green pass; they are not in robotd's dependency graph.)
The External drive plumbing, minus the control-loop branch (Step 4b):
intents.rs: a stamped external-target slot on its OWN clock plus an
external_active flag, so streaming joints never touches the twist deadman and
vice versa (the deadman-critical invariant). set_external (used by setJoints,
engages the mode implicitly), set_external_mode (setMode external / handback),
external_engaged, and Snapshot.external: Option<ExternalDrive> {targets, age,
gain}. request_relax now also leaves External. 8 unit tests: independent clock,
implicit engage, gain carry, clear on leave/relax.
main.rs dispatch:
- robot.setJoints: the External door. Refuses (named reason, touches nothing)
on wrong arity, non-finite target, or a robot not powered+homed ('call
robot.init first'); otherwise stores the frame and enters External. The
safety envelope (limits/step/deadman) stays Safety::apply_external's job.
- robot.setMode 'external' engages the mode (no policy needed, no walk/roller
switch); walk/roller leave External and hand back to the policy.
- robot.mode reports 'external' while engaged.
- robot.enable(on) leaves External (policy takes back over).
setMode refusal message now lists external. New dispatch test gates the
setJoints door end to end; setMode test covers external engage/handback.
Loop consumption is Step 4b (transient #[allow(dead_code)] on Snapshot.external
+ Step::external until then). cargo test -p robotd green (103 + 7 integ),
clippy clean.
…tep 4b) The control loop now consumes Snapshot.external and drives it end to end. robotd/src/main.rs control loop: - Reads snapshot.external; external_drive = engaged AND powered+homed (Ready), not mid limp-fall, not powered off. The policy 'driving' now yields to it (&& external.is_none()), so a controller streaming joints supersedes the on-device policy for the same servos. - New match arm produces the tick straight from the supplied targets via control::Step::external (gain = client's per-frame ask, else the mode's running gain); moving=true so safeToRestart won't say yes mid-stream. - The apply site routes External through Safety::apply_external(targets, hold, age, gain) — anatomical clamp + per-tick step + external deadman — instead of apply(); everything else (policy/homing/limp-fall/hold) still uses apply. - On the falling edge of external_drive the loop calls safety.clear_external() so a later re-entry rate-limits from the current pose, not a stale target. New tokio loop test: powers+homes a robot, streams a head_yaw frame, asserts it reaches the bus through apply_external while un-commanded joints stay home (proving the external frame drives, not a policy). Transient #[allow(dead_code)] on Snapshot.external / Step::external removed — both are now consumed. Transport permission gates (both exhaustive by design, so the new Call variant forced a decision in each): - btd/src/route.rs: robot.setJoints => false. Raw per-joint streaming is the most direct motor control there is; refused over BLE for every reason robot.move is, and then some (20-byte, ~73s-late notification link). - mediad/src/route.rs: robot.setJoints => true. The WebRTC datachannel is the teleop transport External drive is for; the peer has the camera and the safety layer bounds every frame. Refused over BLE, permitted here. cargo test --workspace green; clippy --all-targets clean; fmt clean (folded in a rustfmt pass over the ANATOMICAL_LIMITS table from an earlier step).
…l drive (Step 5) Rollout item 4: the reference External-drive client and the docs for it. The RFC named a duckctl example, but setJoints is (correctly) refused over BLE — the transport duckctl speaks — so the streaming client lives in robotctl, which reaches robotd's dispatch directly over the local Unix socket. robotctl robot external <joint> [--amplitude --hz --seconds --rate --gain]: - resolves the joint by JOINT_NAMES name or index; reads the current pose from one robot.state frame so the sweep oscillates around the live angle and every other joint is held where it stands; - enters External (robot.setMode external), streams robot.setJoints frames (targets[j] = base[j] + amplitude*sin(2π·hz·t)) at --rate, checking each IntentResult so a refusal — e.g. a robot that isn't powered+homed — is surfaced with the fix, not swallowed; - on finish OR Ctrl-C settles the joint back and hands control to the walking policy (setMode walk); a hard kill instead relies on the external deadman (~150 ms hold-then-limp). SIGINT handler shared with the theremin path. It is a downstream driver in miniature: read pose, enter External, stream absolute targets, leave cleanly. docs/design/robotd-design.md: new §3.1 'External drive' subsection (setMode external + setJoints wire examples; drive-source-not-policy-mode; separate clock; the apply_external safety envelope; BLE-refused / WebRTC-permitted); §3.5 reconciled so setJoints reads as an intent gated per transport, distinct from the maintenance-namespace raw joint writes. cargo test -p robotctl green (138; cli_definition_is_valid + bash_completions_cover_the_command_tree pass with the new subcommand); workspace build + clippy --all-targets clean; fmt clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a scoping RFC (design doc only, no implementation yet) — opening it to get your direction on the shape and the safety design before writing code. The doc lands as
docs/rfc/0001-external-per-joint-write.md.Summary
Add one intent to robotd's
robot.*socket —robot.setJoints— that lets an off-robot controller stream joint targets at the control rate, plus a matchingExternaldrive mode in which the loop uses those targets in place of the on-device policy. Everything still flows through the existingduck-controlSafetychokepoint; nothing bypassesSafety::apply, andcontrol::Runtimestill holds no IO handle.Why
robotd is intent-level by design (
robot.move/pose/head/do→ on-devicePolicy→ targets), and that stays the default. But two workflows have no path today:modelcomponent on every change. The teleop / VLA / lerobot / MHS pattern is controller computes joint targets, streams them at rate; there's no verb for it.This is distinct from deploying a finished on-device policy, which the updater's
modelcomponent already serves (update.apply, gated byrobot.modelApi). The RFC does not touch that path.The safety thesis
robot.setJointsmust be no less safe than the policy path, and one gap has to close.duck-control/src/safety.rs(lines 42–46) says the range clamp is the actuator's travel, not a per-joint anatomical limit — "the real joint limits live in the MJCF, which is not vendored here." A policy trained in that MJCF stays inside those limits implicitly; arbitrary external targets do not. So, all inside the existingSafetychokepoint:setJointshonoured only inExternalmode with torque on; refused otherwise (named error), like a refusedrun_policy.|target[j] − previous[j]|to a max joint velocity so one bad frame can't snap a joint.[min,max]from the MJCF and clamp to them, closing the gap above.Shape (details in the doc)
duck-ipc-proto:method::ROBOT_SET_JOINTS = "robot.setJoints"+Call::RobotSetJoints(SetJointsParams { targets: Vec<f64> /* len NUM_JOINTS */, gain: Option<u16> }), a notification likerobot.move, with a serde round-trip + exact-wire test.Externaldrive mode viarobot.setMode; external-target slot + own age clock inintents.rs; loop becomesread → observe → gate → {External ? external_targets : policy} → safety.apply.strands-robotsnative driver from delegate-only to a real passthrough mode.Open questions for you
DEFAULT_POSITION?Externalmode, or gatesetJointson an existing mode?duck-control, or a config file robotd loads?setJoints, or masked like the policy path masks it?Happy to implement along whichever lines you prefer.