Bug Description
AsyncExecutor.close() can block forever. It is called during LocalConversation.close() to release tool executors, so a single stuck portal task wedges conversation shutdown — and with it every subsequent operation that needs the conversation lock.
Two independent causes, both in AsyncExecutor.close() (async_executor.py:104-114):
if portal_cm is not None:
try:
portal_cm.__exit__(None, None, None)
- Remaining tasks are never cancelled. Passing no exception makes anyio take its graceful path,
portal.call(portal.stop, cancel_remaining=False), which waits for in-flight tasks to finish on their own. A task that never completes blocks shutdown permanently — even one that would die instantly if cancelled.
- The thread join is unbounded.
start_blocking_portal() ends in a bare thread.join() with no timeout (anyio from_thread.py). So even with cancellation, a task that cannot be cancelled — anything awaiting inside a worker thread, where anyio cannot deliver cancellation until the thread returns — hangs the join forever.
Cancelling alone does not fix it, and bounding the join alone leaves cause 1 waiting the full timeout on every close. Both need addressing.
AsyncExecutor backs the browser toolset (BrowserToolExecutor), the MCP client (MCPClient), and ACPAgent, so this is not browser-specific.
Expected Behavior
close() returns promptly. Tasks still on the portal are cancelled; if the portal thread cannot be stopped it is abandoned — it is a daemon thread — and a warning is logged.
Actual Behavior
close() blocks indefinitely. Nothing times out, so the process never recovers.
Save the snippet under Steps to Reproduce as repro_close_hang.py in a checkout of this repo and run it:
uv run python repro_close_hang.py
Observed on main (bb26768) and on 1.42.1:
close() STILL BLOCKED after 10s
Expected output is close() returned. The 10s is only the harness's patience — the call never returns at all.
The same hang is reachable from the test suite. With a bounded close() this passes in ~1s; on main test_close_returns_with_task_still_running blocks until its assertion deadline:
uv run pytest tests/sdk/utils/test_async_executor.py -q
It also escapes pytest's own timeout when it happens in teardown: on main, uv run pytest tests/tools/browser_use/test_browser_executor_e2e.py::TestBrowserExecutorE2E::test_navigate_action -q --timeout=180 never returned and had to be killed externally at 400s.
Steps to Reproduce
import threading, time, anyio
from openhands.sdk.utils.async_executor import AsyncExecutor
ex = AsyncExecutor()
ex.portal.start_task_soon(anyio.sleep_forever) # a task still in flight
time.sleep(0.5)
done = threading.Event()
threading.Thread(target=lambda: (ex.close(), done.set()), daemon=True).start()
print("close() returned" if done.wait(10) else "close() STILL BLOCKED after 10s")Prints close() STILL BLOCKED after 10s on main (bb26768) and on 1.42.1. Note anyio.sleep_forever is trivially cancellable — it dies instantly under cancel_remaining=True.
Isolating the two causes:
| task on the portal |
cancel_remaining |
close() |
anyio.sleep_forever() |
False (current behaviour) |
hangs |
anyio.sleep_forever() |
True |
returns |
await run_sync(lambda: time.sleep(3600)) |
True |
hangs |
Production symptom
Hit on an agent-server running the browser tool. LocalConversation.close() closes tool executors in a loop with no timeout (local_conversation.py:2679-2688), reaching BrowserToolExecutor.close(), whose finally calls self._async_executor.close() unbounded (browser_use/impl.py:710-732). The 30s guard on the preceding run_async(self.cleanup, ...) does not cover it.
The server stayed 8h21m in this state. py-spy dump:
Thread 550621 (idle): "asyncio_4"
_wait_for_tstate_lock (threading.py:1167)
join (threading.py:1147)
start_blocking_portal (anyio/from_thread.py:560)
__exit__ (contextlib.py:144)
close (openhands/sdk/utils/async_executor.py:112)
close (openhands/tools/browser_use/impl.py:727)
close (openhands/sdk/conversation/impl/local_conversation.py:2685)
Thread 555902 (idle): "asyncio-portal-705da41d8410"
_cancel_all_tasks (asyncio/runners.py:205)
run_blocking_portal (anyio/from_thread.py:536)
Because the close holds the conversation lock, routes needing a loaded conversation (GET /api/conversations/{id}/events*) blocked forever, while metadata routes (/api/conversations/{id}, /api/conversations/search, /health) kept answering in milliseconds. The service therefore looks healthy while no conversation can be opened. Chromium had already exited; no file lock was held. Only a process restart recovered it.
Acceptance Criteria
Installation Method
uv (workspace checkout), and uvx --from openhands-agent-server==1.42.1
SDK Version
1.42.1, and main at bb26768
Python Version
3.12.3 (production), 3.13.13 (repo venv)
Operating System
Linux
Bug Description
AsyncExecutor.close()can block forever. It is called duringLocalConversation.close()to release tool executors, so a single stuck portal task wedges conversation shutdown — and with it every subsequent operation that needs the conversation lock.Two independent causes, both in
AsyncExecutor.close()(async_executor.py:104-114):portal.call(portal.stop, cancel_remaining=False), which waits for in-flight tasks to finish on their own. A task that never completes blocks shutdown permanently — even one that would die instantly if cancelled.start_blocking_portal()ends in a barethread.join()with no timeout (anyiofrom_thread.py). So even with cancellation, a task that cannot be cancelled — anything awaiting inside a worker thread, where anyio cannot deliver cancellation until the thread returns — hangs the join forever.Cancelling alone does not fix it, and bounding the join alone leaves cause 1 waiting the full timeout on every close. Both need addressing.
AsyncExecutorbacks the browser toolset (BrowserToolExecutor), the MCP client (MCPClient), andACPAgent, so this is not browser-specific.Expected Behavior
close()returns promptly. Tasks still on the portal are cancelled; if the portal thread cannot be stopped it is abandoned — it is a daemon thread — and a warning is logged.Actual Behavior
close()blocks indefinitely. Nothing times out, so the process never recovers.Save the snippet under Steps to Reproduce as
repro_close_hang.pyin a checkout of this repo and run it:Observed on
main(bb26768) and on 1.42.1:Expected output is
close() returned. The 10s is only the harness's patience — the call never returns at all.The same hang is reachable from the test suite. With a bounded
close()this passes in ~1s; onmaintest_close_returns_with_task_still_runningblocks until its assertion deadline:It also escapes pytest's own timeout when it happens in teardown: on
main,uv run pytest tests/tools/browser_use/test_browser_executor_e2e.py::TestBrowserExecutorE2E::test_navigate_action -q --timeout=180never returned and had to be killed externally at 400s.Steps to Reproduce
Prints
close() STILL BLOCKED after 10sonmain(bb26768) and on 1.42.1. Noteanyio.sleep_foreveris trivially cancellable — it dies instantly undercancel_remaining=True.Isolating the two causes:
cancel_remainingclose()anyio.sleep_forever()False(current behaviour)anyio.sleep_forever()Trueawait run_sync(lambda: time.sleep(3600))TrueProduction symptom
Hit on an agent-server running the browser tool.
LocalConversation.close()closes tool executors in a loop with no timeout (local_conversation.py:2679-2688), reachingBrowserToolExecutor.close(), whosefinallycallsself._async_executor.close()unbounded (browser_use/impl.py:710-732). The 30s guard on the precedingrun_async(self.cleanup, ...)does not cover it.The server stayed 8h21m in this state.
py-spy dump:Because the close holds the conversation lock, routes needing a loaded conversation (
GET /api/conversations/{id}/events*) blocked forever, while metadata routes (/api/conversations/{id},/api/conversations/search,/health) kept answering in milliseconds. The service therefore looks healthy while no conversation can be opened. Chromium had already exited; no file lock was held. Only a process restart recovered it.Acceptance Criteria
close()returns even with a task still running on the portalclose()returns even when that task cannot be cancelled, and logs a warningclose()stays idempotent and safe when the portal was never startedInstallation Method
uv(workspace checkout), anduvx --from openhands-agent-server==1.42.1SDK Version
1.42.1, and
mainatbb26768Python Version
3.12.3 (production), 3.13.13 (repo venv)
Operating System
Linux