Skip to content

ui: make MCP a built-in extension, on MCP SDK v2 - #61

Draft
allozaur wants to merge 17 commits into
allozaur/refactor/llama-harness-packagefrom
allozaur/refactor/mcp-extension
Draft

allozaur wants to merge 17 commits into
allozaur/refactor/llama-harness-packagefrom
allozaur/refactor/mcp-extension

Conversation

@allozaur

@allozaur allozaur commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Overview

MCP becomes the first built-in extension of Llama Chat: a feature that plugs into a small core and can be switched off in Settings. Its client moves to MCP SDK v2 and speaks the stateless protocol, with the session handshake kept for older servers. Local MCP servers that llama-server runs (stdio) now show up beside the remote ones, with a short how-to for adding one.

Additional information

Part 6 of 13 of the work-in-progress stack on the fork, based on allozaur/refactor/llama-harness-package (#55), Llama Harness, above voice mode (#65). Its diff is only this layer. Projects (#28), Servers (#35) and the prompt library (#29) sit on top of it since 13 November, and the prompt library adds one more extension point, systemPromptSources, for the MCP prompts in its System message menu.

Known gaps:

  • llama-server embeds the app as a single bundle (bundleStrategy: 'single'), so MCP's code ships whether the extension is on or off; nothing of it runs while it is off. A build that splits chunks gives MCP a 438 KB chunk of its own that nothing imports statically.
  • MCP's shared data stays in the core, so conversations and settings read the same either way: its types, enums, constants, utils/mcp.ts, settings keys, the stored message formats and the export that strips server headers. The + button's tooltip still says "MCP Servers".
  • A switch in one tab does not reach other open tabs until they reload. A flow that is running while MCP is switched off was not tested.
  • After a stateless listen stream is reopened there is no re-list, so a change published in the gap is missed. Connections promoted from a health check get no list-changed stream, as before.
  • Not run: the E2E suite, the Docker and Nix builds and the CMake embed. A real model called an MCP tool in the browser only through a stdio server run by llama-server, not through a server the browser connects to.
  • One commit of this layer is a llama-server change (mcp_server in GET /tools), so it can go upstream on its own. Without it, the app guesses a stdio server's name from the part of the tool name before the first _.
What changed

Extensions

  • src/lib/extensions/ holds the registry. An extension is a manifest (id, name, description, load) and one function, activate(ctx), which calls ctx.contribute(point, contribution) and ctx.onDeactivate(cleanup). The app registers its built-in extensions and activates the ones that are on once the stores are ready.
  • Switching an extension off removes its contributions at once and runs its cleanups. The choice is kept in localStorage under LlamaUi.extensions.
  • The core never imports an extension's code; an extension may import the core. A lint rule, local/no-extension-internals, enforces it for every folder under src/lib/extensions/<id>/.
  • Settings has a new Extensions section with a switch per extension and the reason when one failed to start.
  • A setting can name the extension it belongs to: its field shows only while that extension is on, and the stored value stays. The MCP request timeout is the first.

Extension points

Point What the core does with it
tools Lists the extension's tools in groups, runs their calls, shows group icons and names. Stored tool keys keep their shape (mcp-<serverId>:<tool>, mcp:<serverId>).
flow Calls the hook when an agentic flow starts and what it returns when the flow ends.
navigation Draws a sidebar entry: a page on a phone, a dialog on a desktop or tablet.
pages Shows the extension's page at its path through one generic route; a path nothing contributes is a 404.
composerCommands Lists a slash command in the / picker.
composerPickers Mounts a popover picker that a command opens.
composerAttachments Draws pending attachments under the input and adds them to the message on send.
messageRenderers Lets the extension draw a message it claims.
attachmentRenderers Lets the extension draw an attachment it claims.

MCP as an extension

  • All of MCP's code is in src/lib/extensions/mcp/: the client (services/), the host store (stores/), every component, and the glue that contributes to each point. The core barrels no longer export any of it.
  • With MCP off, the sidebar entry, the /prompt command, MCP's tools and #/mcp-servers are gone, its connections are closed and pending resource attachments are dropped. Old messages keep their MCP prompts and resources, drawn as plain text attachments.

MCP SDK v2

  • @modelcontextprotocol/sdk 1.32.1 is replaced by @modelcontextprotocol/client 2.4.0. The demo fixture's server side uses @modelcontextprotocol/server and @modelcontextprotocol/node.
  • Over Streamable HTTP the client negotiates: a server/discover probe gives a stateless connection (MCP 2026-07-28, no session id, list-changed and resource subscriptions over subscriptions/listen); an older server gets the initialize handshake as before. SSE and WebSocket keep the handshake.
  • A probe that fails for a reason other than authentication (a 5xx, an unusable body, a timeout) is followed by one attempt with the session handshake, within the same connect deadline, and the connection log says so.
  • A listen stream that ends while the connection is alive is reopened with backoff. List calls bypass the SDK's response cache, as the stores keep their own.
  • v2 has no WebSocket client transport, so the v1 one is kept in the extension (76 lines).
  • The server card shows the negotiated protocol version; it used to show a fixed 2025-06-18.

Local MCP servers (stdio)

  • A page cannot start a process, so a local MCP server that runs as a command is one llama-server starts from --mcp-servers-config. Its tools used to reach the app as plain server tools under the llama-server group.
  • llama-server now names each such tool's server in GET /tools (mcp_server, beside "type": "mcp").
  • A tool provider can claim llama-server's tools of one /tools type (serverToolType), and MCP claims mcp. They become MCP tools, one group per server (mcp:llama-server:<name>), switched off and allowed like any other MCP server's tools, and still run on llama-server. With MCP off, the core lists them as server tools again.
  • The MCP servers page shows each of these servers as a card marked "Run by llama-server", with its tools and a switch for them, and a collapsible "Use a local MCP server (stdio)" with a config example and the command to copy. This is the flow iMCP's README describes for llama.cpp, and it does not change.

Docs

  • Storybook has a new Architecture › Extensions page: the rule, the points, MCP, and what the Models manager, Projects and the Library still need before they can plug in the same way. None of that is built.
What behaves differently
  • An unknown single-segment path such as #/nope is a 404 from the page's load, with the same page as before.
  • MCP's background health checks start when the extension is activated, just before the root layout mounts.
  • A 404 from an MCP server counts as an expired session only on a connection that has one.
  • A list call with no cursor walks all pages, a list on a capability the server does not advertise returns empty without a request, and client error messages lose the MCP error <code>: prefix. The SDK does these.
  • Connecting to an older server costs one more request, the rejected probe, and the connection log shows it.
  • An MCP reconnect loop ends when its server is removed or the store shuts down.
  • The tools of a stdio server llama-server runs move from the llama-server group to a group of their own, under new keys, so a choice made for them before (switched off, always allowed) does not carry over.
Testing
  • npm run check, prettier --check and eslint pass. 1009 unit tests, 334 client tests and 250 story tests pass, and the production build completes (16 November, after the SvelteKit 3 restack; pwa.spec passes once it reads the fresh build).
  • New tests cover the registry, each extension point from the core side with and without contributions, MCP's glue for each point, switching MCP off and on from Settings, the stored tool keys, both protocol eras against in-process servers, the legacy retry, the listen stream reopening, and the reconnect loop ending.
  • llama-server's test_mcp_servers.py passes (22 tests), with new checks on mcp_server. New unit tests cover the core handing llama-server's mcp tools to the MCP extension and back, their group keys, the policy, running them on llama-server, and the server-name fallback; a new story shows the cards and the how-to.
  • In a browser, against llama-server started with --mcp-servers-config and two stdio servers (the test fixture under echo and my_tools) and a 4B model: both show on the MCP servers page at desktop and phone width, the tools menu lists them as two groups, and the model called echo_add (17 + 25) after the permission prompt ("Allow use of echo_add from echo?"), which llama-server ran, giving 42.
  • In a browser, against the official reference servers on localhost:
    • a stateless server connects with protocol 2026-07-28 and 22 tools;
    • a session-only server connects with 2025-11-25 after the probe is rejected;
    • the same server behind strict CORS, where the browser blocks the probe's preflight, falls back to the handshake and connects;
    • /prompt lists the server's prompts and inserts one into the form;
    • switching MCP off in Settings removes the sidebar entry and /prompt, turns #/mcp-servers into the 404 and survives a reload; switching it on brings all of it back.
  • From Node, with the app's MCPService: connect, list and call tools, prompts, resources and subscriptions against both servers, directly and through llama-server's CORS proxy; a server that answers the probe with 500; and a front that cuts every listen stream.
  • The sidebar, the MCP dialog and page, and the chat form's prompt flow were compared before and after in a browser at desktop and phone width, with the same results.

Requirements

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Nov 13, 2026
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from 98b9026 to d6d9c83 Compare November 13, 2026 20:38
@allozaur
allozaur added this pull request to stack #62 November 13, 2026 20:39
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from d6d9c83 to d48911f Compare November 16, 2026 07:05
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from d48911f to 9374b2e Compare November 16, 2026 08:19
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch 2 times, most recently from 209f536 to 9ed55c6 Compare November 16, 2026 10:42
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from 9ed55c6 to 91b08e6 Compare November 16, 2026 11:28
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from 91b08e6 to 180211b Compare November 16, 2026 11:43
@allozaur
allozaur removed this pull request from stack #62 November 16, 2026 11:56
@allozaur
allozaur changed the base branch from allozaur/refactor/llama-harness-package to allozaur/feat/voice-conversation November 16, 2026 11:56
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from 180211b to 30d4ec9 Compare November 16, 2026 11:56
@allozaur
allozaur added this pull request to stack #66 November 16, 2026 11:57
@allozaur
allozaur force-pushed the allozaur/refactor/mcp-extension branch from 30d4ec9 to 08a82f3 Compare November 16, 2026 12:07
Extensions are how a feature outside the core plugs into the app. Each one
is known by a manifest, its code is imported only when it is activated, and
it adds its parts through extension points. The registry holds those
contributions and removes them, running the extension's cleanup, when the
extension is switched off.

The app starts its built-in extensions once the stores are ready. MCP is
the first one: it is registered here and contributes nothing yet, the next
commits move it behind the extension points one by one.

Assisted-by: Claude (Opus 5.5 Medium)
The tools store, the agentic flow, the tool policy and the tools UI no longer import MCP. The MCP extension contributes a tool provider to the `tools` point, which lists the servers' tools, normalizes their schemas, runs the calls and gives the icons and hints the UI shows, and a hook to the `flow` point, which holds the connections while a flow runs.

The core reads both from the extension registry, so with the MCP extension off the tools UI and the flow work the same, just without the MCP tools. The stored tool and group keys keep their format.

Assisted-by: Claude (Opus 5.5 Medium)
… pages extension points

The sidebar, the routes and the root layout no longer import MCP code. Two new extension points carry what they need instead.

`navigation` is an entry in the sidebar: the MCP extension contributes its item, which the sidebar places between Models and Settings, lists in the search dialog and opens like before, as the MCP servers dialog on a desktop or tablet and as a page on a phone. The sidebar mounts the dialogs of contributed items itself.

`pages` is a page at a path of its own: the static `mcp-servers` route gives way to one generic route that renders the page contributed for the path, and ends in the app's 404 when nothing is contributed for it. `#/mcp-servers` keeps working and the sidebar item is lit while its page shows.

The background MCP health checks move from the root layout into the extension's activation and stop when the extension is switched off.

Assisted-by: Claude (Opus 5.5 Medium)
…xtension points

The chat form no longer imports MCP code. It reads three new extension points instead: composerCommands (slash commands of the `/` picker), composerPickers (popover pickers a command opens) and composerAttachments (attachments held outside the form's files until the message is sent).

The MCP extension contributes the `/prompt` command, its prompt picker and the attached resources through them. The code that turns a picked prompt into a loading chip, then fills it or marks it failed, moved from ChatForm into the extension and works through a small composer handle. The chat store now takes the pending resource extras from composerAttachments when a message is sent.

With the extension on, the form looks and behaves as before. Without it, `/prompt` and the resource list are gone and the rest of the form works unchanged.

Assisted-by: Claude (Opus 5.5 Medium)
…on points

The chat message and the attachments list no longer know about MCP. Two new
extension points, messageRenderers and attachmentRenderers, let an extension
claim a message or an attachment and draw it with its own component. The MCP
extension contributes its prompt message, its prompt chip and its resource
chip through them, and the resource chip now owns the preview dialog that the
attachments list used to hold.

A claimed attachment stays out of the built-in preview. When nothing claims
an MCP prompt or resource, because the extension is off, it is drawn as a
plain text attachment with its name and text, and a message that is only an
MCP prompt is drawn as a normal user message with that chip.

Stories that show MCP prompts or MCP attachments switch the built-in
extensions on in their setup.

Assisted-by: Claude (Opus 5.5 Medium)
MCP's store, service and components move into `src/lib/extensions/mcp/` (`stores/`, `services/`, `components/`), beside the glue that was already there. The core barrels (`$lib/stores`, `$lib/services`, `$lib/components/app` and its `chat`, `dialogs` and `settings` barrels) no longer export any of it, so nothing in the core can reach MCP's code except through the extension points.

Inside the extension a file imports the others by relative path. The documented component barrel moves along as `components/index.ts` and is for tests and stories; the extension's own code does not import it, which removes the cycle through `tools.ts`, the tool group identity, the barrel and the servers settings.

The search results block of a tool call was the last core user of mcpStore: it now takes the icon from `toolsStore.getToolIcon` and reads the CORS proxy flag from the server props, which is what mcpStore did.

A new ESLint rule, `local/no-extension-internals`, enforces the rule for every extension: a module under `src/lib/extensions/<id>/` can only be imported from that folder. The registry's own files stay importable, `builtin.ts` loads each extension's entry with a dynamic import, and `tests/**` is exempt.

MCP's shared data (types, enums, constants, `utils/mcp.ts`, settings keys, stored formats) stays in the core. No behaviour changes.

Assisted-by: Claude (Opus 5.5 Medium)
Storybook now registers and activates the built-in extensions in the preview's `beforeAll`, the way the app does at startup, so every story and story test sees what the extensions add without asking for it.

The stories that did this themselves (the chat form, the MCP prompt message, the attachments list and the sidebar) drop their own activation. The one guarded helper stays in `tests/stories/helpers/extensions.ts`.

Assisted-by: Claude (Opus 5.5 Medium)
Settings has a new section, Extensions: the extensions the app knows, each with its name, its description and a switch. A switch takes effect at once and the registry remembers the choice, so a reload keeps it. An extension that failed to load or activate says why, and switching it off and on tries again.

Switching MCP off now also closes its live connections and drops the resources still attached in the chat form, so nothing of it keeps running and nothing comes back unseen when it is switched on again. Its sidebar entry, its page, the `/prompt` command and its tools already went away with its contributions.

Assisted-by: Claude (Opus 5.5 Medium)
Replace @modelcontextprotocol/sdk with @modelcontextprotocol/client. The client keeps the session handshake, so behaviour does not change.

SDK v2 has no WebSocket client transport, so the v1 one now lives in the app. The demo MCP servers use @modelcontextprotocol/server and @modelcontextprotocol/node.

Assisted-by: Claude (Opus 5.5 Medium)
…ndshake

Streamable HTTP connections probe the server with server/discover (MCP 2026-07-28) and fall back to the initialize handshake on older servers. SSE and WebSocket connections keep the initialize handshake.

The connection log and the server card show the negotiated protocol version. On a stateless connection, resource subscriptions use a subscriptions/listen stream.

Assisted-by: Claude (Opus 5.5 Medium)
- A failed stateless probe (HTTP 5xx, unusable reply, no reply) is retried once with the session handshake, with a warning in the connection log. Authorization failures are not retried.
- A subscriptions/listen stream that the server or a proxy ends is opened again with a backoff, until the connection is closed.
- List calls always reach the server, the SDK cache cannot answer them.
- A 404 means an expired session only on a connection that has a session.
- Remove the unused protocol version constant and update comments that described the session flow as the only one.

Assisted-by: Claude (Opus 5.5 Medium)
…ut down

A reconnect loop kept the server's config it started with, so shutting the
store down between two attempts did not stop it: the next attempt brought a
connection back. Switching the MCP extension off shuts the store down, which
made that reachable from Settings.

The loop now checks that its config is still the registered one before each
attempt and after a connect, and closes a connection that arrives too late.
A shutdown with no live connection forgets the configs too.

Assisted-by: Claude (Opus 5.5 Medium)
A setting can now name the extension it belongs to. Its field is drawn only
while that extension is running; the stored value and its default stay, so
settings export, sync and migrations see the same keys either way.

The MCP request timeout is the first such field: with MCP switched off it
no longer shows in Settings.

Assisted-by: Claude (Opus 5.5 Medium)
A setting can name its extension (`extension` in SETTINGS_REGISTRY), so
its field shows only while the extension is on and the stored value stays
either way. An extension's code ships in the single bundle llama-server
embeds, but none of it runs while the extension is off.

The Architecture › Extensions page that came with this lives with the
Storybook's Architecture section, outside this stack.

Assisted-by: Claude (Opus 5.5 Medium)
An MCP prompt picked into the chat form is an attachment of the app's own
kind, drawn by MCP's renderer. With the extension switched off before the
message is sent, it fell back to the generic file chip with its kind as an
overflowing label. It now reads as a text file once its text has loaded,
the same rule a stored attachment of such a kind already follows.

Assisted-by: Claude (Opus 5.5 Medium)
A tool of an MCP server llama-server runs from --mcp-servers-config is
listed as <server>_<tool>, so a client could only guess the server from
the name, and guess wrong when the server's name has an underscore.
GET /tools now gives it in `mcp_server` for such tools. Built-in tools
are listed as before.

Assisted-by: Claude (Opus 5.5 Medium)
…s to

A page cannot start a process, so a local MCP server that runs as a
command (stdio) is one llama-server starts from --mcp-servers-config. Its
tools reached Llama Chat as plain server tools, under the llama-server
group, unlike the MCP servers the browser connects to by URL.

The MCP extension now takes those tools over: a tool provider can claim
llama-server's tools of one /tools type, and MCP claims `mcp`. They are
listed one group per server, switched off and allowed like the tools of
any other MCP server, and still run on llama-server. The MCP servers page
shows each of these servers as a card marked as run by llama-server, and
explains how to add a local server to llama-server's config. With MCP
switched off, the core lists them as server tools again.

Assisted-by: Claude (Opus 5.5 Medium)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops documentation Improvements or additions to documentation nix server/ui server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant