Skip to content

Fix/memory2 go2 autorecorder#1925

Open
leshy wants to merge 5 commits intodevfrom
fix/memory2-autorecorder
Open

Fix/memory2 go2 autorecorder#1925
leshy wants to merge 5 commits intodevfrom
fix/memory2-autorecorder

Conversation

@leshy
Copy link
Copy Markdown
Contributor

@leshy leshy commented Apr 28, 2026

  • confirmed auto-recording of go2 when running on real platform
  • created new (temporary) china office recording - better one coming tomorrow
  • ./bin/lfs_push ignores sqlite sidecar files
  • fixed bgr/rgb stuff once and for all, removed legacy hacks fixing this

@leshy leshy changed the title Fix/memory2 autorecorder Fix/memory2 go2 autorecorder Apr 28, 2026
Comment on lines +543 to +544
rgb_array = self.to_rgb().data
jpeg_data = jpeg.encode(rgb_array, quality=quality, pixel_format=TJPF_RGB)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 RGBA images will break JPEG encoding

to_rgb() for ImageFormat.RGBA returns a 4-channel copy (it doesn't strip alpha — see the return self.copy() branch at line 247), so rgb_array will have shape (H, W, 4). Passing a 4-channel array to jpeg.encode(..., pixel_format=TJPF_RGB) (which expects 3-channel RGB) will raise an error at runtime.

The old to_bgr() path correctly dropped the alpha channel via cv2.COLOR_RGBA2BGR. The new path doesn't. If an RGBA-formatted Image ever reaches lcm_jpeg_encode, it will now fail where it previously succeeded.

Comment thread dimos/memory2/codecs/test_codecs.py Outdated
paul-nechifor
paul-nechifor previously approved these changes Apr 28, 2026
@dimensionalOS dimensionalOS deleted a comment from greptile-apps Bot Apr 29, 2026
@leshy leshy enabled auto-merge (squash) April 29, 2026 10:34
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 29, 2026

Greptile Summary

This PR confirms auto-recording on the real Go2 platform, ships a new go2_china_office recording, and performs a clean-up of long-standing BGR/RGB confusion. Key changes: to_rgb() now correctly strips alpha channels from RGBA/BGRA inputs, JPEG encode/decode is canonicalised to TJPF_RGB end-to-end, the Recorder skips setup in replay mode to avoid touching the playback database, and SqliteStore gains a must_exist guard for replay paths.

Confidence Score: 5/5

Safe to merge — all findings are P2 and limited to a test helper.

The only issue found is in _jpeg_eq: the preliminary shape guard compares against the pre-conversion shape rather than to_rgb().data.shape, causing false-negatives for RGBA/BGRA/grayscale inputs. This is confined to a test utility and does not affect production code. All production-path changes (JPEG BGR→RGB, alpha-drop fixes, replay guard, must_exist flag) are correct and consistent.

dimos/memory2/codecs/test_codecs.py — shape check in _jpeg_eq should use original.to_rgb().data.shape.

Important Files Changed

Filename Overview
bin/lfs_push Extends iteration from directories-only to all entries; adds a case filter to skip SQLite WAL/shm/journal sidecar files that would produce useless LFS tarballs.
dimos/msgs/sensor_msgs/Image.py Fixes RGBA/BGRA to_rgb() to strip the alpha channel (was returning a 4-channel copy); switches JPEG encode/decode to explicit TJPF_RGB path, making format canonicalisation deterministic and removing the previous RGBA 4-channel crash.
dimos/memory2/module.py Adds an early-return guard in Recorder.start() when replay mode is active, preventing the recorder from overwriting or conflicting with the playback database.
dimos/memory2/store/sqlite.py Adds must_exist: bool = False flag to SqliteStoreConfig; raises a clear FileNotFoundError at construction time rather than silently creating an empty DB during replay.
dimos/robot/unitree/go2/connection.py Renames dir_namedataset, updates default dataset to go2_china_office, and removes the legacy _autocast_video BGR→RGB relabelling hack — all references updated consistently.
dimos/robot/unitree/go2/blueprints/smart/unitree_go2.py Adds odom: In[PoseStamped] port to Go2Memory so odometry is now captured alongside color image and lidar during live recording.
dimos/memory2/codecs/test_codecs.py Updates _jpeg_eq to check ImageFormat.RGB tag and compare against original.to_rgb(); shape guard still uses pre-conversion shape, causing false-negatives for RGBA/BGRA/grayscale inputs.
dimos/utils/testing/replay.py Passes must_exist=True when opening replay stores so missing datasets fail loudly instead of silently creating empty databases.

Sequence Diagram

sequenceDiagram
    participant RC as ReplayConnection
    participant TSR as TimedSensorReplay
    participant SS as SqliteStore<br/>(must_exist=True)
    participant Rec as Recorder (Go2Memory)
    participant DB as SQLite DB

    RC->>TSR: video_stream() / lidar_stream() / odom_stream()
    TSR->>SS: _get_store(dataset)
    SS->>DB: open (raises FileNotFoundError if missing)
    DB-->>SS: connection
    SS-->>TSR: store
    TSR-->>RC: Observable[Image|PointCloud2|PoseStamped]

    Note over Rec: start() called
    Rec->>Rec: g.replay == True?
    alt replay mode
        Rec-->>Rec: return early — DB left untouched
    else live mode
        Rec->>DB: record color_image / lidar / odom streams
    end

    Note over RC: lcm_jpeg_encode
    RC->>RC: to_rgb() → TJPF_RGB encode
    RC->>RC: lcm_jpeg_decode → TJPF_RGB → ImageFormat.RGB
Loading

Reviews (3): Last reviewed commit: "fix(Image): to_rgb/to_bgr correctly hand..." | Re-trigger Greptile

to_rgb() previously returned a 4-channel RGBA-tagged Image for RGBA input
and an RGBA Image for BGRA input, contradicting its name and breaking
lcm_jpeg_encode (which passes the result to TurboJPEG with TJPF_RGB,
expecting 3 channels). Both branches now strip alpha via cv2 and return a
real RGB Image. to_rgb/to_bgr now also warn when dropping alpha.

Also moves ImageFormat import to top of test_codecs.py per review feedback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants