Skip to content

Call-first UI: instant listen, Bloub, neural-or-silence TTS - #1

Merged
byte271 merged 15 commits into
mainfrom
cursor/cloud-agent-environment-ee9a
Aug 28, 2026
Merged

Call-first UI: instant listen, Bloub, neural-or-silence TTS#1
byte271 merged 15 commits into
mainfrom
cursor/cloud-agent-environment-ee9a

Conversation

@byte271

@byte271 byte271 commented Aug 28, 2026

Copy link
Copy Markdown
Owner

OpenLive’s default path is now a call, not a lab console.

Feel (GPT-Live)

  • First paint is the listening orb + Mute / End / Settings. No branded splash overlay.
  • The session auto-joins. If the browser blocks the mic until a gesture, the surface says Tap to talk and the first tap on the stage starts the call. Mute is mute, not Start.
  • Brand opens Settings. Latency is off unless you turn it on.
  • Call chrome is pinned to the document root so it cannot be clipped by stage overflow.
  • Demo TTS stays neural or silence. Formant only if you pick it.
  • Sandbox, multi-agent, camera, and diagnostics stay in Settings → Advanced (collapsed by default, grid-safe so the disclosure stays tappable).
  • Tactile sounds live under Appearance (no orphaned Feel block overlapping Advanced).

Desktop CI

  • OPENLIVE_SKIP_GATEWAY_BUILD=1 writes a placeholder openlive-gateway-bundled.exe so clippy’s tauri_build resource check passes without a real gateway binary.
  • Real PNG/ICO/ICNS placeholders are checked in for bundling.
  • Empty macOS entitlements string removed.
  • Tauri CLI install uses cargo install … --force so cached runners on macOS and Windows do not fail when cargo-tauri already exists.

How to try

  1. Open the web UI or desktop shell.
  2. You should see the Bloub face and Listening (or Tap to talk until mic is granted).
  3. Talk. Interrupt. Open Settings → Advanced for lab tools.

Not affiliated with xAI or OpenAI. Face: Bloub (MIT). Controls: Morphicons + Lucide.

The lockfile requires rustc 1.85+ (edition 2024). Pin 1.98.0, compile the workspace, and start the mock gateway on :8787 so Cloud Agents boot ready to work.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d3a1bde5-19a5-4a5b-8f42-bc4e9354ff13

📥 Commits

Reviewing files that changed from the base of the PR and between 4b83c75 and be49d49.

⛔ Files ignored due to path filters (5)
  • apps/openlive-desktop/icons/128x128.png is excluded by !**/*.png
  • apps/openlive-desktop/icons/128x128@2x.png is excluded by !**/*.png
  • apps/openlive-desktop/icons/32x32.png is excluded by !**/*.png
  • apps/openlive-desktop/icons/icon.ico is excluded by !**/*.ico
  • apps/openlive-gateway/web/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (73)
  • .cursor/environment.json
  • .cursor/install.sh
  • .cursor/start-gateway.sh
  • .github/workflows/ci.yml
  • .gitignore
  • THIRD_PARTY_NOTICES.md
  • apps/openlive-desktop/README.md
  • apps/openlive-desktop/build.rs
  • apps/openlive-desktop/icons/icon.icns
  • apps/openlive-desktop/scripts/generate-icons.py
  • apps/openlive-desktop/splash/index.html
  • apps/openlive-desktop/src/main.rs
  • apps/openlive-desktop/tauri.conf.json
  • apps/openlive-gateway/src/config.rs
  • apps/openlive-gateway/src/main.rs
  • apps/openlive-gateway/src/session.rs
  • apps/openlive-gateway/src/session_registry.rs
  • apps/openlive-gateway/src/session_state.rs
  • apps/openlive-gateway/src/webrtc_media.rs
  • apps/openlive-gateway/tests/task_lifecycle.rs
  • apps/openlive-gateway/web/app.js
  • apps/openlive-gateway/web/audio-session.js
  • apps/openlive-gateway/web/bloub-orb.js
  • apps/openlive-gateway/web/call-controls.js
  • apps/openlive-gateway/web/call-entry.js
  • apps/openlive-gateway/web/call-state.js
  • apps/openlive-gateway/web/demo-voice.js
  • apps/openlive-gateway/web/index.html
  • apps/openlive-gateway/web/package.json
  • apps/openlive-gateway/web/scripts/vendor-ui.mjs
  • apps/openlive-gateway/web/settings-store.js
  • apps/openlive-gateway/web/setup-store.js
  • apps/openlive-gateway/web/styles.css
  • apps/openlive-gateway/web/tests/call-entry.test.js
  • apps/openlive-gateway/web/tests/call-state.test.js
  • apps/openlive-gateway/web/tests/demo-voice.test.js
  • apps/openlive-gateway/web/tests/protocol.test.js
  • apps/openlive-gateway/web/tts-client.js
  • apps/openlive-gateway/web/ui.js
  • apps/openlive-gateway/web/vendor/bloub/LICENSE
  • apps/openlive-gateway/web/vendor/bloub/NOTICE
  • apps/openlive-gateway/web/vendor/bloub/engine.js
  • apps/openlive-gateway/web/vendor/bloub/entry.js
  • apps/openlive-gateway/web/vendor/bloub/src/cycles.ts
  • apps/openlive-gateway/web/vendor/bloub/src/decor.ts
  • apps/openlive-gateway/web/vendor/bloub/src/engine.ts
  • apps/openlive-gateway/web/vendor/bloub/src/expressions.ts
  • apps/openlive-gateway/web/vendor/bloub/src/eyefit.ts
  • apps/openlive-gateway/web/vendor/bloub/src/face.ts
  • apps/openlive-gateway/web/vendor/bloub/src/math.ts
  • apps/openlive-gateway/web/vendor/bloub/src/profiles.ts
  • apps/openlive-gateway/web/vendor/bloub/src/repere.ts
  • apps/openlive-gateway/web/vendor/bloub/src/shape.ts
  • apps/openlive-gateway/web/vendor/bloub/src/skins.ts
  • apps/openlive-gateway/web/vendor/bloub/src/states.ts
  • apps/openlive-gateway/web/vendor/licenses/lucide-LICENSE
  • apps/openlive-gateway/web/vendor/licenses/morphicons-LICENSE
  • apps/openlive-gateway/web/vendor/morphicons-entry.js
  • apps/openlive-gateway/web/vendor/morphicons.js
  • apps/openlive-gateway/web/visual-state.js
  • apps/openlive-gateway/web/voice-visualizer.js
  • crates/openlive-provider/src/headless_browser.rs
  • crates/openlive-provider/src/knowledge.rs
  • crates/openlive-provider/src/llm_bridge.rs
  • crates/openlive-provider/src/memory_store.rs
  • crates/openlive-provider/src/mock.rs
  • crates/openlive-provider/src/openai_compatible.rs
  • crates/openlive-provider/src/openai_realtime.rs
  • crates/openlive-provider/src/pending_actions.rs
  • crates/openlive-provider/src/tools.rs
  • crates/openlive-provider/src/user_profile.rs
  • crates/openlive-runtime/src/persistence.rs
  • docs/gpt-live-parity.md

📝 Walkthrough

Walkthrough

The change introduces a call-first OpenLive interface. It adds a local desktop splash flow, Bloub SVG animation, animated call controls, automatic call joining, neural-or-silence TTS behavior, vendored UI assets, and development startup scripts. It also includes lint-oriented Rust refactors and updated documentation.

Changes

OpenLive call surface

Layer / File(s) Summary
Desktop startup and gateway readiness
.cursor/*, apps/openlive-desktop/...
The desktop shell now displays a local splash page, polls gateway health, stages gateway resources, and bundles generated icons.
Gateway TTS and runtime behavior
apps/openlive-gateway/src/*, crates/openlive-provider/src/*, crates/openlive-runtime/src/*
Automatic TTS now uses Piper or silence. Formant synthesis requires explicit selection. Rust fallback and Clippy patterns were refactored.
Call entry and web UI
apps/openlive-gateway/web/{app.js,index.html,styles.css,ui.js,...}
The web app starts on a listening call surface, supports guarded auto-join, adds Mute/End/Settings controls, and moves advanced tools into Settings.
Bloub animation system
apps/openlive-gateway/web/vendor/bloub/*, apps/openlive-gateway/web/bloub-orb.js
A deterministic SVG avatar engine now renders states, expressions, shapes, eye fitting, particles, arcs, and transitions.
Vendored UI assets and validation
apps/openlive-gateway/web/vendor/*, apps/openlive-gateway/web/tests/*, docs/*
Morphicons and Lucide assets are bundled with license notices. New tests cover call entry, voice mapping, TTS policy, and presentation defaults.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Browser
  participant CallEntry
  participant AudioSession
  participant Gateway
  participant BloubOrb

  Browser->>CallEntry: evaluate shouldAutoJoinCall
  CallEntry->>AudioSession: beginConversation after gesture
  AudioSession->>Gateway: establish voice session
  Gateway-->>Browser: voice state and media events
  Browser->>BloubOrb: setMode and setSignals
  BloubOrb-->>Browser: render SVG animation frame
Loading
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cursor/cloud-agent-environment-ee9a

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

cursoragent and others added 2 commits August 28, 2026 05:55
CI uses stable rustc 1.98 with -D warnings while the workspace denies clippy::all and clippy::pedantic, so map_unwrap_or, unnecessary_sort_by, result_large_err, and related lints now fail the rust job.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
Default chrome is the listening orb plus Mute / End / Settings.
Vendors Bloub (MIT) and Morphicons + Lucide. Auto TTS is Piper or
silence — no formant pad. Desktop shows the orb while the gateway
spawns.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
@cursor cursor Bot changed the title Add Cloud Agent environment for Rust 1.98 and mock gateway Call-first UI: Bloub orb, Morphicons, neural-or-silence TTS Aug 28, 2026
cursoragent and others added 4 commits August 28, 2026 06:30
The bundled lucide re-export listed Phone instead of PhoneOff, which
blocked ESM load of call-controls and left the UI on the boot splash.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
Exclude the orb SVG from the global icon stroke reset, use a light
face with dark eyes on black, and hide idle chrome rings/backchannel.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
Boot fade-in left a translateY transform on the signal shell, which
made the fixed dock clip under overflow:hidden. Clear those for minimal UI.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
Skip the branded splash delay, auto-join the session, and treat Mute as mute. Lab chrome stays in Settings. Desktop CI lint no longer requires a prebuilt gateway binary.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
@cursor cursor Bot changed the title Call-first UI: Bloub orb, Morphicons, neural-or-silence TTS Call-first UI: instant listen, Bloub, neural-or-silence TTS Aug 28, 2026
cursoragent and others added 8 commits August 28, 2026 07:21
The call chrome no longer lives inside overflow-hidden stage layout. The OpenLive wordmark opens Settings instead of diagnostics.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
Keep the stage from eating topbar clicks, and let End cancel an in-flight auto-join.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
Move splash and media-generation state above first use so auto-join, brand Settings, and End all wire up on load.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
tauri_build still requires bundle.resources during clippy, so the skip
path writes a placeholder gateway binary. Add PNG/ICO/ICNS placeholders
and drop the empty macOS entitlements string.

The web UI no longer mounts a branded boot splash, keeps lab tools
inside a collapsed Advanced block, and uses call-native copy.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
The settings body is a CSS grid. Forcing min-height:0 on the Advanced
details crushed the closed summary to the border width, so lab tools
were unreachable. Use min-content so the disclosure stays tappable.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
The Feel fieldset used to append after Settings → Advanced once Runtime
was no longer a top-level fieldset, which overlapped the lab disclosure.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
macOS desktop-build failed when the cache already had cargo-tauri:
cargo install exits 101 without --force. Skip install when 2.11.4
is present, otherwise force-install.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
The bash if/then skip broke PowerShell on windows-latest. A single
cargo install --force works on macOS and Windows and tolerates a
cached cargo-tauri binary.

Co-authored-by: Byte271 <byte271@users.noreply.github.com>
@byte271
byte271 marked this pull request as ready for review August 28, 2026 14:05
@byte271
byte271 merged commit 9f954fd into main Aug 28, 2026
8 of 9 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be49d490d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

addTimeline("microphone", `Capture started at ${sampleRate} Hz`);
}

if (epoch !== joinEpoch) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Clean up resources when cancelling an in-flight join

If the user presses End while the new auto-join is awaiting provider/transport setup, joinEpoch changes but the function can still resume, open a socket and start the microphone before reaching this check. Returning here performs no cleanup because endConversation() already ran, leaving the microphone and connection active behind an idle UI; cancellation must be checked before each resource acquisition or explicitly tear down resources on an epoch mismatch.

Useful? React with 👍 / 👎.

Comment on lines +160 to +162
const advance = this.reducedMotion ? 0 : dt * this.motionScale;
this.clock += advance;
this.paint(this.engine.sample(this.clock));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve state changes when motion is disabled

When prefers-reduced-motion is enabled, or the motion slider is set to 0, advance remains zero forever. BotEngine.setState() represents state changes as time-based morphs, so every subsequent sample(this.clock) stays at morph progress zero and the face remains in its previous state instead of reflecting thinking, speaking, yielding, or interruption; suppress ambient animation without freezing state-transition time, or snap transitions to their terminal pose.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants