[Bug] Prediction loop stalls on tool-call dispatch; abort times out ("Canceling predictions timed out"); app requires force-restart

App version: 0.4.23 (beta channel, build 1) — Windows Model in use at time of incident: qwen3.8-27b-ud (~16.3 GB, MTP/spec-decode enabled, Vulkan) Repro context: any chat with a tools-provider plugin active; here the plugin was an internal file-tools provider (crunch3r/ai-toolbox). The failing call was a trivial directory search (measured runtime 2–17 ms in every other call of the same session).

Symptom

Mid-conversation, after a tool call is dispatched, the chat UI's tool card shows a running spinner indefinitely. Everything else reports "stopped/idle":

Developer Logs show the engine already released its slot at the end of the previous generation (stop processing, truncated = 0 — i.e. a natural EOS stop, not a truncation or user abort).
Local Server page shows Status: Stopped / Server not running while the model is still loaded (READY).
No further log output appears until ~3 minutes later, when the app's own cleanup path fails and the whole application wedges — only a force-restart recovers it.

Exact sequence from `logs/main.log` (all timestamps local)

Key observations

No user action was taken between the tool dispatch and the failure — the stop button was never pressed, so this is not a cancel-path race from UI input. The engine's "stop processing" line predates the stall and corresponds to a normal end-of-generation.

Expected behavior

A tool call dispatch that fails or stalls should surface an error to the chat UI (tool card transitions to an error state) instead of spinning forever.
The internal abort path ("Canceling predictions") should either succeed within its timeout or escalate with a recoverable state — not produce an unhandled rejection and kill the engine process silently, leaving the app wedged until force-restart.

Attachments (4 screenshots)

screenshot 2026-08-30 174715.png — Developer → Local Server: model READY while server shows "Stopped"; slot release lines visible at bottom of Developer Logs
screenshot 2026-08-30 174735.png — Chat UI ("ai_toolbox Session Handoff"): tool card stuck in running state after engine already stopped
screenshot 2026-08-30 174801.png — Zoomed chat view: same stuck "Model is calling grep_files()" spinner, no result/error ever rendered
screenshot 2026-08-30 174846.png — Developer Logs window: full generation timing for the final successful slot (natural EOS stop) and the subsequent silence

Suggested diagnostics

Check what the default prediction-loop handler does when a tool-call dispatch promise neither resolves nor rejects (missing timeout/abort wiring around provider invocations).
Investigate why "Canceling predictions" times out even though no generation is in flight at that point (engine idle since ~17:46) — suggests the loop was already stuck in a state where its abort check never runs.

[Bug] Prediction loop stalls on tool-call dispatch; abort times out ("Canceling predictions timed out"); app requires force-restart

Symptom

Mid-conversation, after a tool call is dispatched, the chat UI's tool card shows a running spinner indefinitely. Everything else reports "stopped/idle":

Developer Logs show the engine already released its slot at the end of the previous generation (stop processing, truncated = 0 — i.e. a natural EOS stop, not a truncation or user abort).

Local Server page shows Status: Stopped / Server not running while the model is still loaded (READY).

No further log output appears until ~3 minutes later, when the app's own cleanup path fails and the whole application wedges — only a force-restart recovers it.

Expected behavior

A tool call dispatch that fails or stalls should surface an error to the chat UI (tool card transitions to an error state) instead of spinning forever.

The internal abort path ("Canceling predictions") should either succeed within its timeout or escalate with a recoverable state — not produce an unhandled rejection and kill the engine process silently, leaving the app wedged until force-restart.

Attachments (4 screenshots)

screenshot 2026-08-30 174715.png — Developer → Local Server: model READY while server shows "Stopped"; slot release lines visible at bottom of Developer Logs

screenshot 2026-08-30 174735.png — Chat UI ("ai_toolbox Session Handoff"): tool card stuck in running state after engine already stopped

screenshot 2026-08-30 174801.png — Zoomed chat view: same stuck "Model is calling grep_files()" spinner, no result/error ever rendered

screenshot 2026-08-30 174846.png — Developer Logs window: full generation timing for the final successful slot (natural EOS stop) and the subsequent silence

Suggested diagnostics

Check what the default prediction-loop handler does when a tool-call dispatch promise neither resolves nor rejects (missing timeout/abort wiring around provider invocations).

Investigate why "Canceling predictions" times out even though no generation is in flight at that point (engine idle since ~17:46) — suggests the loop was already stuck in a state where its abort check never runs.

ai-toolbox

ai-toolbox

ai-toolbox

ai-toolbox

[Bug] Prediction loop stalls on tool-call dispatch; abort times out ("Canceling predictions timed out"); app requires force-restart

Symptom

Exact sequence from logs/main.log (all timestamps local)

Key observations

Expected behavior

Attachments (4 screenshots)

Suggested diagnostics

[Bug] Prediction loop stalls on tool-call dispatch; abort times out ("Canceling predictions timed out"); app requires force-restart

Symptom

Exact sequence from logs/main.log (all timestamps local)

Key observations

Expected behavior

Attachments (4 screenshots)

Suggested diagnostics

Exact sequence from `logs/main.log` (all timestamps local)

Exact sequence from `logs/main.log` (all timestamps local)