Conversation
Merge "Fix stream camera rotation" with working branch
…troller Co-authored-by: Cursor <cursoragent@cursor.com>
EIS is owned by asg_client via SysControl.setEisEnable, which disables it during streaming to reduce camera HAL thermal load. Having a second path in StreamPackLite that force-enables EIS on every startRequestSession contradicts that and also bakes a Mentra-Live-specific vendor key (com.pixsmart.eisfeature.eisEnable) plus a CONTROL_SCENE_MODE_SPORTS override into upstream-able library code.
Re-introduces the Mentra-Live-specific Pixsmart EIS configuration (CONTROL_SCENE_MODE_SPORTS + com.pixsmart.eisfeature.eisEnable vendor key) but only fires when CameraController.enablePixsmartEisOnRequest is set to true. Default is false so the fork stays generic for any other consumer. asg_client flips it from StreamCommandHandler around the livestream lifecycle.
Brings in native-capture-crop streaming support: - VideoConfig.captureResolution for camera/SurfaceTexture buffer sizing - FullFrameRect center-crop in the GL pipeline - VideoMediaCodecEncoder wiring for capture vs encoder size - Opt-in CameraController.enablePixsmartEisOnRequest flag (defaults false; asg_client toggles it from StreamCommandHandler)
Throttle the GL path to the requested fps, force a fixed Camera2 range for Mentra Live, and surface rolling encode fps/bitrate for ASG telemetry.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| } catch (e: Exception) { | ||
| Logger.d(TAG, "Could not set encoder parameters: ${e.message}") | ||
| } | ||
| } |
There was a problem hiding this comment.
Parameters set before configure
Medium Severity
codec.setParameters runs before codec.configure, while the codec is still uninitialized. MediaCodec only accepts parameters in the executing state, so this always fails into the catch block. The comment mentions operating rate, but the code sets PARAMETER_KEY_VIDEO_BITRATE on both audio and video encoders.
Reviewed by Cursor Bugbot for commit e7a4517. Configure here.
| get() = sharedPref.getBoolean(resources.getString(R.string.video_enable_key), field) | ||
|
|
||
| var powerSavingMode: Boolean = true | ||
| get() = sharedPref.getBoolean(resources.getString(R.string.video_power_saving_key), field) |
There was a problem hiding this comment.
Power-saving toggle unused
Medium Severity
powerSavingMode is exposed in preferences and read from SharedPreferences, but nothing in the app ever checks it. The new Power saving switch therefore has no effect on streaming behavior.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit e7a4517. Configure here.
| get() = sharedPref.getString(resources.getString(R.string.video_encoder_key), field)!! | ||
|
|
||
| var fps: Int = 30 | ||
| var fps: Int = 15 // Lower framerate to reduce CPU usage |
There was a problem hiding this comment.
Conflicting FPS defaults
Medium Severity
Configuration.Video.fps now defaults to 15, but the preference XML defaults to 24 and FpsEntries only lists 24/25/30/60. Before preferences are persisted, streaming uses an FPS that is not a valid UI option.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit e7a4517. Configure here.
| } | ||
|
|
||
| var bitrate: Int = 2000 | ||
| var bitrate: Int = 250 // Low bitrate (250 kbps) to reduce encoding load |
There was a problem hiding this comment.
Bitrate default below minimum
Medium Severity
Video bitrate now defaults to 250 kbps in Configuration, but the bitrate SeekBarPreference enforces min=500 and defaultValue=500. First-run streams can encode below the UI’s allowed minimum.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit e7a4517. Configure here.
| ) | ||
|
|
||
| var videoBitrateRange: Range<Int> = Range(300, 5000000) | ||
| var videoBitrateRange: Range<Int> = Range(100, 500000) |
There was a problem hiding this comment.
Bitrate range default inflated
Medium Severity
videoBitrateRange defaults to Range(100, 500000), but the getter multiplies both ends by 1000 as kbps. A missing target-bitrate preference therefore becomes a 500 Mbps regulation ceiling, far above the intended ~500 kbps default.
Reviewed by Cursor Bugbot for commit e7a4517. Configure here.
Only invent synthetic [fps,fps] ranges when forceFixedFpsInsideSupportedBand is enabled (Mentra Live). Otherwise stay on advertised containing ranges. Reset measuredCaptureFps sampling when a request session starts or the camera stops.
Min-interval-since-last-accept locked 30fps capture targeting 20fps to every other frame (~15fps). Pace with a next-deadline schedule so average encode rate matches VideoConfig.fps, and drain dropped SurfaceTexture frames so the camera buffer queue does not stall.
…encode-metrics Honor VideoConfig fps and expose measured encode metrics
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 6 total unresolved issues (including 5 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 8d7824f. Configure here.
| if (targets.isEmpty()) { | ||
| Logger.e(TAG, "No valid surfaces available for camera preview") | ||
| return | ||
| } |
There was a problem hiding this comment.
Silent preview start failure
High Severity
When no valid surfaces are available, startPreview logs an error and returns normally instead of throwing. Callers such as BaseCameraStreamer.startPreview and PreviewView then treat the call as success and fire onPreviewStarted, even though the camera never started.
Reviewed by Cursor Bugbot for commit 8d7824f. Configure here.
3 participants
Note
High Risk
Touches the core camera session template, FPS selection, and GL encode path for all users; global behavior changes (preview template, stabilization off, hardcoded tiny preview) can regress quality or compatibility on standard phones despite opt-in device flags.
Overview
Adds a low-power live streaming path across camera capture, GL compositing, and MediaCodec encoding, aimed at constrained devices (e.g. Mentra Live) while exposing runtime telemetry for tuning.
Video pipeline:
VideoConfiggains optionalcaptureResolutionso the camera/SurfaceTexture can run at a larger buffer while the encoder still targetsresolution;FullFrameRectcenter-crops via newsetMVPMatrixViewPortAndCrop. The surface encoder paces incoming frames toVideoConfig.fpswith a deadline-based scheduler (fixes 30→20 fps locking to ~15), drains dropped frames to avoid stalls, and runs the GL thread at minimum priority.MediaCodecEncodertracks rolling measured encode bitrate and fps; streamer settings exposemeasuredFps,measuredBitrateBps, and camera measuredCaptureFps.Camera2:
CameraControllerswitches repeating requests fromTEMPLATE_RECORDtoTEMPLATE_PREVIEW, turns off video stabilization and uses FAST NR/edge/hot-pixel modes, improves FPS range selection (exact fixed range preferred; optional synthetic[fps,fps]viaforceFixedFpsInsideSupportedBand), and adds opt-in Pixsmart EIS (enablePixsmartEisOnRequest).CameraSourcevalidates surfaces before session start;PreviewViewrequests 160×120 preview sizes for power savings.Demo defaults shift to aggressive power settings (QCIF, 15 fps, low bitrates, mono 16 kHz audio, bitrate regulation on) plus a power saving preference (stored but not wired in core from this diff).
Reviewed by Cursor Bugbot for commit 8d7824f. Bugbot is set up for automated code reviews on this repo. Configure here.