fix: free local llama-cpp model deterministically to prevent crash on exit#31
Merged
paberr merged 1 commit intoJun 23, 2026
Merged
Conversation
… exit The Llama model was only released via __del__, which runs at interpreter shutdown when module globals (contextlib, llama_cpp's C bindings) are already torn down to None. The model was therefore never freed, so the Metal device finalizer aborted with GGML_ASSERT/SIGABRT, preceded by AttributeError and TypeError from the failed __del__ paths. Add an idempotent close() to the Summarizer contract and call it deterministically from every summarization path (pipeline + search), so the model is freed while the interpreter is still alive. __del__ remains only as a best-effort backstop using plain try/except.
paberr
approved these changes
Jun 23, 2026
paberr
left a comment
Owner
There was a problem hiding this comment.
Thanks for the PR and writeup. I agree with the analysis!
It's good to merge from my side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Running
ownscribewith the local (llama-cpp) summarization backend crashes on exit, after the summary has already been produced and saved:Root cause
The loaded
Llamamodel was only ever released via__del__. The summarizer object survives until interpreter shutdown, at which point module globals are torn down toNone:LlamaCppSummarizer.__del__callscontextlib.suppress(...), but the module-levelcontextlibis alreadyNone→AttributeError.llama_cpp's ownLlama.__del__callsfree_model, but its C-function pointer is alreadyNone→TypeError.Because
free_modelnever runs, the Metal buffers stay allocated. When the process finally reachesexit()→__cxa_finalize→ggml_metal_device_free, the global device still has live residency sets, soGGML_ASSERT([rsets->data count] == 0)aborts the process withSIGABRT.__del__is fundamentally the wrong place for this — at shutdown the model cannot be freed because llama_cpp's C bindings are already gone.Fix
Free the model deterministically while the interpreter is still alive, instead of relying on
__del__:close()plus context-manager protocol (__enter__/__exit__) to theSummarizerbase contract (no-op default; backends without native resources inherit it for free).close()inLlamaCppSummarizerto freeself._llm. It uses a plaintry/exceptrather thancontextlib.suppressprecisely because it may also run from__del__at shutdown whencontextlibisNone.close()deterministically from every summarization path:run_summarizeand_do_transcribe_and_summarize(viatry/finally) inpipeline.py, andask(viawith summarizer:) insearch.py.__del__remains only as a best-effort backstop.Testing
close()(frees model, idempotent, no-op without load, error suppression, context manager) and the inherited no-op contract for non-local backends.ruff check src/ tests/passes.ownscribe summarizeon a real transcript with the cachedphi-4-minimodel → exit code 0, none of theException ignored/GGML_ASSERT/SIGABRTmarkers.