Skip to content

ui: move the agentic loop into @ggml-org/llama-harness, in ui/lib/harness - #55

Draft
allozaur wants to merge 3 commits into
allozaur/feat/voice-conversationfrom
allozaur/refactor/llama-harness-package
Draft

allozaur wants to merge 3 commits into
allozaur/feat/voice-conversationfrom
allozaur/refactor/llama-harness-package

Conversation

@allozaur

@allozaur allozaur commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Overview

Llama Chat's agentic loop becomes a headless package of its own, Llama Harness (@ggml-org/llama-harness), so the loop can run without the UI and an app can use it or bring its own. Llama Chat works as before.

Additional information

Part 5 of 13 of the work-in-progress stack on the fork, based on allozaur/feat/voice-conversation (#65), voice mode, which sits on voice messages and dictation (#64) and the npx package (#69). Its diff is only this layer.

Since 17 November this layer sits above the voice PRs (#64, #65), right under the MCP extension (#61); before that it sat below them, and on 13 November it moved down from the top of the stack. Its commits are the same apart from the Projects tools, which the agentic store now gains in the Projects layer above.

The package is at ui/lib/harness, beside the library in ui/lib/ui: ui/lib holds the packages, one folder each.

Known gaps: the Docker and Nix builds were not run locally. The harness is not published to npm yet, and Llama Chat still uses its own chat service as the inference client, not the package's.

What changed
  • ui/lib/harness holds @ggml-org/llama-harness: plain TypeScript, no UI, no DOM, no dependencies.
    • createHarness(client).run(messages, tools, signal, options) returns the run as an async stream of events: turn_start, text, reasoning, tool_call_delta, timings, message, tool_call, tool_update, tool_result, turn_end, then done or error. It never throws, and it waits for the consumer between events, so an app can store each message before the loop goes on.
    • Messages and tools use the OpenAI shapes. A tool is its definition plus execute. The loop keeps no state and stores nothing.
    • signal stops at once, interrupted ends the run at the next safe point, and maxTurns with onTurnLimit asks before going past the limit.
    • createOpenAIClient is an inference client for any OpenAI-compatible endpoint. It reads llama-server's timings and prompt_progress when they are there.
  • Llama Chat's agenticStore no longer has the loop. It gives the harness ChatService as the inference client and the enabled tools, each behind the permission gate, and it turns the events into the same callbacks as before. Its public API, the gates and the chat store are unchanged.
  • Chat compiles the harness from source through a @ggml-org/llama-harness alias in vite.config.ts (a Vite alias plus a TypeScript path), so it needs no build of the package. The builds that copy or isolate ui/apps/chat bring ui/lib/harness along the same way as ui/lib/ui: CMake stages its sources next to the app, the Dockerfiles add a COPY, the Nix derivation takes ui/ as its source, and sources.cmake watches the harness sources.
  • The UI workflow has a job that type-checks, tests and packs the harness.

Three behaviours differ from before:

  • A call to a tool that was not offered to the model gets an Unknown tool error as its result. Before, a tool that exists but is disabled could still run after the permission prompt.
  • The output of a streaming tool (exec_shell_command) is coalesced: when the UI is slower than the stream, it gets the latest output, not every chunk.
  • The live "streaming tool call" state is also cleared when a turn ends with an error or is stopped.

One existing behaviour is kept as it is and looks like a bug: after Continue at the turn limit, the turn count starts again at 1, so no new assistant message is created for the next turn and it is written into the previous one.

Testing
  • ui/lib/harness: npm run check, npm test (18 tests of the loop and the client) and npm run prepack (build and publint) pass.
  • ui/apps/chat: npm run check, npm run lint, the 833 unit tests and npm run build passed on the earlier top of the stack. In the new position, type-check, lint and the 771 unit tests of this layer pass; the build was not run again. The client tests for tool calls and shell streaming pass.
  • A new unit test, agentic-flow.test.ts, runs the flow end to end with the server and the tools mocked: a tool turn, a denied permission, a failing tool, a streaming tool, an image in a tool result, a steering message, a stream error and an abort. The same test passes on the store as it was before this PR.
  • The CMake asset script (scripts/ui-assets.cmake) stages the app with the harness beside it, builds it with neither package installed and embeds the assets.
  • By hand, with a local llama-server: in Llama Chat, a get_datetime call shows the permission prompt, runs and the model answers with the result. From Node, the built package runs a tool call and a final answer against the same server, with no browser.

Requirements

@github-actions github-actions Bot added documentation Improvements or additions to documentation devops nix labels Nov 10, 2026
@allozaur
allozaur added this pull request to stack #53 November 10, 2026 23:09
@allozaur allozaur changed the title ui: move the agentic loop into @ggml/llama-harness, in ui/lib/harness ui: move the agentic loop into @ggml/llama-harness, in ui/harness Nov 11, 2026
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from f34d624 to 7409202 Compare November 12, 2026 00:00
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from 7409202 to af2c4ab Compare November 12, 2026 00:09
@allozaur allozaur changed the title ui: move the agentic loop into @ggml/llama-harness, in ui/harness ui: move the agentic loop into @ggml/llama-harness, in ui/lib/harness Nov 12, 2026
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from af2c4ab to dff7cd3 Compare November 12, 2026 00:49
@allozaur
allozaur removed this pull request from stack #53 November 12, 2026 01:28
@allozaur
allozaur changed the base branch from allozaur/refactor/ui-folders to allozaur/style/glass-surfaces November 12, 2026 01:28
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from dff7cd3 to 7fd2f56 Compare November 12, 2026 01:29
@allozaur
allozaur added this pull request to stack #58 November 12, 2026 01:29
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch 3 times, most recently from c1882f0 to be6fe36 Compare November 12, 2026 14:29
@allozaur
allozaur removed this pull request from stack #58 November 12, 2026 20:54
@allozaur
allozaur changed the base branch from allozaur/style/glass-surfaces to allozaur/feat/prompt-library November 12, 2026 20:54
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from be6fe36 to a48ffd4 Compare November 12, 2026 20:55
@allozaur
allozaur added this pull request to stack #59 November 12, 2026 20:55
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from a48ffd4 to 8f887f0 Compare November 12, 2026 22:59
@allozaur
allozaur removed this pull request from stack #59 November 12, 2026 22:59
@allozaur
allozaur changed the base branch from allozaur/feat/prompt-library to allozaur/style/glass-surfaces November 12, 2026 22:59
@allozaur
allozaur added this pull request to stack #60 November 12, 2026 23:00
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from 053f0c8 to 3eb58bb Compare November 16, 2026 13:44
@allozaur
allozaur changed the base branch from allozaur/style/glass-surfaces to allozaur/docs/llama-ui-storybook November 16, 2026 13:45
@allozaur
allozaur added this pull request to stack #68 November 16, 2026 13:45
@allozaur allozaur changed the title ui: move the agentic loop into @ggml/llama-harness, in ui/lib/harness ui: move the agentic loop into @ggml-org/llama-harness, in ui/lib/harness Nov 16, 2026
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from 3eb58bb to 23f5ffa Compare November 16, 2026 13:52
@allozaur
allozaur removed this pull request from stack #68 November 16, 2026 16:12
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from 23f5ffa to 64799b7 Compare November 16, 2026 16:12
@allozaur
allozaur changed the base branch from allozaur/docs/llama-ui-storybook to allozaur/feat/llama-chat-npx November 16, 2026 16:13
@allozaur
allozaur added this pull request to stack #71 November 16, 2026 16:13
@allozaur
allozaur removed this pull request from stack #71 November 16, 2026 16:49
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from 64799b7 to b285cda Compare November 16, 2026 16:50
@allozaur
allozaur added this pull request to stack #72 November 16, 2026 16:50
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from b285cda to f98f578 Compare November 16, 2026 17:36
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from f98f578 to 432435d Compare November 16, 2026 18:58
@allozaur
allozaur force-pushed the allozaur/refactor/llama-harness-package branch from 432435d to f69ce45 Compare November 17, 2026 05:21
…/lib/harness

Llama Harness is the agentic loop as headless TypeScript with no UI, no DOM and no dependencies. It streams a turn, runs
the tools the model called, gives it the results, and repeats until the model answers without a tool call. An app gives
it an inference client and tools, and reads the run as a stream of events: run(messages, tools, signal). It ships a
client for any OpenAI-compatible endpoint, so it also runs on its own in Node. The UI workflow type-checks, tests and
packs it.

Assisted-by: Claude (Fable 5.2)
The loop leaves the agentic store, which now gives the harness what the app owns: ChatService as its inference client,
the enabled tools behind the permission gate, and the callbacks that store each turn. Its API and the flow's callbacks
are unchanged. A unit test runs the flow end to end with the server and the tools mocked.

Chat compiles the harness from source through an alias, so the builds that copy or isolate ui/apps/chat now bring
ui/lib/harness along: CMake and Nix through LLAMA_HARNESS_DIR, the Dockerfiles with a COPY.

Assisted-by: Claude (Fable 5.2)
The layout card no longer calls the move planned, the harness is in the
pages, and they say that Llama Chat does not import @ggml-org/llama-ui yet.

Assisted-by: Claude (Opus 5.5 Medium)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops documentation Improvements or additions to documentation nix server/ui

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant