0.5.209: a waitForServices timeout is thrown on the main thread and kills the app (Android FATAL EXCEPTION; iOS SIGABRT) after the run-without-AI re-point

[emooreatx]

Android, ciris-client 0.5.209 + ciris-server 0.5.203, run #34228782947, live-qa-linux-android → logs/android-logcat.txt, pid 5838:

13:15:48.414  E AndroidRuntime: FATAL EXCEPTION: main
              Process: ai.ciris.mobile.debug, PID: 5838
              java.lang.Exception: Timeout waiting for services (10/22 online)
                at ai.ciris.mobile.shared.viewmodels.StartupViewModel.waitForServices(StartupViewModel.kt:417)
                at ai.ciris.mobile.shared.viewmodels.StartupViewModel$waitForServices$1.invokeSuspend(Unknown Source:14)
                at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:101)
                at android.os.Handler.handleCallback(Handler.java:958)
                …
              Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@216a3c5, Dispatchers.Main.immediate]
13:15:48.473  ActivityManager: Process ai.ciris.mobile.debug (pid 5838) has died: fg  TOP

A waitForServices timeout is raised as a plain Exception inside a coroutine on Dispatchers.Main.immediate with no handler, so it reaches the main thread and the process dies. Context: this is the no-AI session, after the client correctly re-pointed to the node on :4243 (0.5.209's #43 fix) — a node has no 22 agent services to wait for, so the count never arrives and the wait times out. The 10/22 it reports is presumably the last thing the agent showed before it handed off.

iOS, same run, live-qa-macos-ios → cmdlogs/ios-oslog.log: pid 33205 polls http://127.0.0.1:8080/v1/system/health and gets Connection refused for roughly a hundred seconds, then

13:19:10  launchd_sim: UIKitApplication:ai.ciris.mobile [33205]: exited due to SIGABRT | sent by iosApp[33205], ran for 102558ms

A self-sent SIGABRT is what an uncaught Kotlin/Native exception looks like, and the timing matches a services wait giving up — but the Kotlin-side text does not reach oslog, so I am presenting iOS as corroboration rather than the same trace. The consequence on iOS is worse than Android's: nothing restarts the app, so our next leg found no test server at all (no /health from :9091 within 120s).

What I would expect, without prescribing: a services-wait timeout is a UI state ("still starting", "could not reach the agent"), never a throw that escapes the coroutine; and on a run-without-AI install the wait should not be counting agent services in the first place — the backend is a node, and its readiness is /v1/identity (or whatever ActiveBackend names), not 22 services.

This is the third hand-off behaviour after the two on #43, and unlike those it ends the process. On desktop the same wait presumably exists but the app survives; on the phones it is fatal.

[emooreatx]

Fixed on main at d439ba2. Both halves confirmed, and the fatal one is not really about the hand-off — the hand-off just supplied a timeout it had never been given before.

The crash

waitForServices raised a plain throw, and it has two callers:

// StartupViewModel.kt:135 — inside try/catch
try { initializePython(); startFastAPIServer(); waitForServices() }
catch (e: Exception) { … _phase.value = StartupPhase.ERROR }

// StartupViewModel.kt:724 — bare
viewModelScope.launch { waitForServices() }

So whether a timeout was survivable depended on which caller ran it. The bare one is the resume path, on Dispatchers.Main with no handler — which is your FATAL EXCEPTION and, on iOS, the SIGABRT. Your corroboration was right even without the Kotlin text: an uncaught Kotlin/Native exception is exactly a self-sent SIGABRT, and nothing relaunches an iOS app, which is why your next leg found no :9091 at all.

The timeout is now an error state through onErrorDetected — the same path every other startup failure takes. I fixed it inside waitForServices rather than by adding a catch at the bare call site, because a startup wait that kills the app depending on its caller is one refactor away from being fatal again.

The wait itself

You were right that it should not have been counting agent services. After the re-point the backend is the node on :4243, which serves no 22-service roster, so the count could never arrive; the 10/22 is the last thing the agent published before handing off, exactly as you guessed. When ActiveBackend names the node, readiness is the endpoint answering — which the startup poll has already established — so the roster wait is skipped outright.

Worth noting about severity

Of the three hand-off behaviours, this is the only one that ended the process, and the reason is a plain throw in a coroutine rather than anything about nodes. The node case is what produced a timeout on that path for the first time; a slow agent boot could have done the same on any release. So this is a latent crash 0.5.209 exposed rather than one it introduced — which is worth recording, because the same shape may exist elsewhere in startup.

696 shared tests green; desktop, Android and wasm targets compile; gates clean.

Not in a published version. 0.5.210 is publishing now and contains neither this nor #46. Both are on main; I will cut them together and say here when it is out.

[emooreatx]

0.5.211 is on PyPI with both of these fixed — all five wheels and the Android AAR are published. pip install ciris-client==0.5.211.

One caveat so you do not pin the wrong thing: the iOS XCFramework is still building (~2.5h) and attaches to the GitHub release when it lands. Also, 0.5.210 has no XCFramework at all — I cancelled its iOS job to free runners for this cut, after its wheels had already published. So for iOS artifacts, wait for v0.5.211's release assets; do not fall back to v0.5.210.

What 0.5.211 carries for you:

• 0.5.209: a waitForServices timeout is thrown on the main thread and kills the app (Android FATAL EXCEPTION; iOS SIGABRT) after the run-without-AI re-point #47 — the timeout is an error state, not a throw, fixed inside waitForServices so it cannot depend on which caller ran it. And the roster wait is skipped entirely when ActiveBackend names the node, so there is no 22-service count to time out on after the hand-off.
• 0.5.209: after a successful with-AI login the client re-enters the post-setup "holding reconfiguring state while the node restarts" and never leaves Login (macOS) #46 — a live session ends the post-setup hold, so a successful login is no longer thrown away by a re-entered "Restarting your node…".
• Everything from 0.5.209 and 0.5.210: the backend switch actually wired, and the API client moving with it rather than only the node URL.

Expected on your side: the three desktop no-AI legs clear, Android no-AI login passes in-session against the fold, and no fatal exception on either phone. Android reset should still fail on CIRISServer#563 — that is the split-key node-only boot and is neither of ours; a red reset leg there is not any of these issues reopening.

If something is still red, the response text is worth pasting verbatim now: /scroll reports offsets, refusals name what is drivable, and the backend line at startup says which port the client chose ([BACKEND] serving on :4243/health).