Hi, thanks for the clarification in the existing discussion.
We are converting ABC-130K into a LeRobot-style format and ran into several metadata issues that affect geometry-aware use cases such as EEF projection, camera/robot alignment, and visual verification.
From the released MCAP files, the foxglove.CameraCalibration messages appear to contain camera intrinsics only. For example, we can see fields such as timestamp, frame_id, width, height, distortion_model, D, K, and P, but we could not find camera extrinsics for the external or wrist cameras.
Would it be possible to release camera extrinsics as additional metadata? A camera-to-base or base-to-camera transform for each camera would be very helpful, as long as the transform direction and coordinate frame convention are documented.
We also found that camera intrinsics are not always complete for every episode-camera pair. In our scan of 130,703 episodes, 56 task/split reports had partial camera intrinsics, with 3,301 missing observation.images.cam_left_wrist intrinsic entries. cam_high and cam_right_wrist were complete in this scan. No task appears to be missing all intrinsics, but some episodes are missing the left wrist calibration.
Separately, EEF pose fields appear to be partially missing or empty in the raw RobotState/RobotAction messages. In our scan, 69,721 episodes had raw pose present, while 60,982 episodes had empty/invalid pose for the required arm state/action pose fields. We can derive an approximation from joint positions and the MJCF, but it would be very helpful if the dataset release could include the missing EEF pose fields directly, or provide the official derivation procedure and frame convention.
Could you confirm whether the team plans to publish an updated metadata release covering:
1. camera extrinsics for the external and wrist cameras;
2. missing left wrist camera intrinsics;
3. missing or empty EEF pose fields, or an official FK-based derivation script;
4. the expected timeline for such a release, if planned?
We would be happy to test early samples of the updated metadata if useful.
Hi, thanks for the clarification in the existing discussion.
We are converting ABC-130K into a LeRobot-style format and ran into several metadata issues that affect geometry-aware use cases such as EEF projection, camera/robot alignment, and visual verification.
From the released MCAP files, the
foxglove.CameraCalibrationmessages appear to contain camera intrinsics only. For example, we can see fields such astimestamp,frame_id,width,height,distortion_model,D,K, andP, but we could not find camera extrinsics for the external or wrist cameras.Would it be possible to release camera extrinsics as additional metadata? A camera-to-base or base-to-camera transform for each camera would be very helpful, as long as the transform direction and coordinate frame convention are documented.
We also found that camera intrinsics are not always complete for every episode-camera pair. In our scan of 130,703 episodes, 56 task/split reports had partial camera intrinsics, with 3,301 missing
observation.images.cam_left_wristintrinsic entries.cam_highandcam_right_wristwere complete in this scan. No task appears to be missing all intrinsics, but some episodes are missing the left wrist calibration.Separately, EEF pose fields appear to be partially missing or empty in the raw RobotState/RobotAction messages. In our scan, 69,721 episodes had raw pose present, while 60,982 episodes had empty/invalid pose for the required arm state/action pose fields. We can derive an approximation from joint positions and the MJCF, but it would be very helpful if the dataset release could include the missing EEF pose fields directly, or provide the official derivation procedure and frame convention.
Could you confirm whether the team plans to publish an updated metadata release covering:
1. camera extrinsics for the external and wrist cameras;
2. missing left wrist camera intrinsics;
3. missing or empty EEF pose fields, or an official FK-based derivation script;
4. the expected timeline for such a release, if planned?
We would be happy to test early samples of the updated metadata if useful.