Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a4445e8
ui: add an extension registry to Llama Chat
allozaur Nov 12, 2026
2ee1e04
ui: list and run MCP tools through the tools extension point
allozaur Nov 12, 2026
27f2015
ui: open MCP from the sidebar and its page through the navigation and…
allozaur Nov 12, 2026
8c26dca
ui: add MCP prompts and resources to the chat form through composer e…
allozaur Nov 12, 2026
61356dd
ui: draw MCP prompt messages and attachments through renderer extensi…
allozaur Nov 12, 2026
4b50e34
ui: keep MCP's code in its extension folder, out of the core barrels
allozaur Nov 12, 2026
297f9a0
ui: switch the built-in extensions on once for all Storybook stories
allozaur Nov 13, 2026
22f70e5
ui: switch extensions on and off in Settings
allozaur Nov 13, 2026
633d641
ui: move the MCP client to MCP SDK v2
allozaur Nov 12, 2026
980f3a2
ui: negotiate the stateless MCP protocol, fall back to the session ha…
allozaur Nov 12, 2026
c3c5be4
ui: keep stateless MCP connections as dependable as session ones
allozaur Nov 12, 2026
38b6f0b
ui: end an MCP reconnect loop when its server is gone or the store sh…
allozaur Nov 13, 2026
e8b0e5b
ui: hide an extension's settings fields while it is off
allozaur Nov 13, 2026
b811de2
ui: say in the README how a setting belongs to an extension
allozaur Nov 13, 2026
19b13e6
ui: draw an unsent attachment of an unclaimed kind as text
allozaur Nov 13, 2026
f63aace
server : name the MCP server of each tool in /tools
allozaur Nov 16, 2026
be69015
ui: list the MCP servers llama-server runs beside the ones it connect…
allozaur Nov 16, 2026
4a1794a
ui: show Models and MCP in the extension tests, now that both wait be…
allozaur Nov 18, 2026
212792c
ui: bring the MCP extension in line with the PRs now below it
allozaur Nov 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions tools/server/README-dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ Get a list of tools, each tool has these fields:
- `tool` (string): the ID name of the tool, to be used in POST call. Example: `read_file`
- `display_name` (string): the name to be displayed on UI. Example: `Read file`
- `type` (string): `"server"` for a server tool, or `"mcp"` for a tool exposed by an MCP server
- `mcp_server` (string, only when `type` is `"mcp"`): the name of the MCP server the tool comes from, as declared in `--mcp-servers-config`. Example: `imcp`
- `permissions` (object): a mapping string --> boolean that indicates the permission required by this tool. This is useful for the UI to ask the user before calling the tool. For now, the only permission supported is `"write"`
- `definition` (object): the OAI-compat definition of this tool

Expand Down
2 changes: 1 addition & 1 deletion tools/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,7 @@ The same JSON can be passed inline with `--mcp-servers-json`. Each entry under `
| `cwd` | working directory of the child process |
| `timeout_ms` | per-tool-call timeout (default: 30000) |

Every server is spawned once at startup to list its tools, then stopped, and respawned on demand when one of its tools is called. Tools are exposed as `<server>_<tool>` alongside the built-in ones: they show up in Llama Chat and in `GET /tools`, and the model calls them like any other tool. A name colliding with an already registered tool is skipped. This is independent of `--tools`, MCP servers can be the only tools available.
Every server is spawned once at startup to list its tools, then stopped, and respawned on demand when one of its tools is called. Tools are exposed as `<server>_<tool>` alongside the built-in ones: they show up in Llama Chat, which lists each of these servers on its MCP servers page, and in `GET /tools`, with `"type": "mcp"` and the server's name in `mcp_server`. The model calls them like any other tool. A name colliding with an already registered tool is skipped. This is independent of `--tools`, MCP servers can be the only tools available.

The child process runs with the same privileges as the server, so only declare commands you trust. As with `--tools`, `--cors-origins` then defaults to `localhost`.

Expand Down
7 changes: 6 additions & 1 deletion tools/server/server-tools.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ static std::string console_output_to_utf8(const std::string & text) {
}

json server_tool::to_json() const {
return {
json res = {
{"display_name", display_name},
{"tool", name},
{"type", type()},
Expand All @@ -99,6 +99,10 @@ json server_tool::to_json() const {
{"uses_cwd", uses_cwd},
{"definition", get_definition()},
};
if (!mcp_server.empty()) {
res["mcp_server"] = mcp_server;
}
return res;
}

static constexpr size_t SERVER_TOOL_GIT_LS_FILES_MAX_OUTPUT = 8 * 1024 * 1024; // 8 MB
Expand Down Expand Up @@ -1814,6 +1818,7 @@ struct server_mcp_tool : server_tool {
{
name = server_name + "_" + tool_name;
display_name = name;
mcp_server = server_name;
permission_write = false;
support_stream = false;
}
Expand Down
1 change: 1 addition & 0 deletions tools/server/server-tools.h
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ struct server_tool {
bool permission_write = false;
bool support_stream = false; // if true, output can be streamed
bool uses_cwd = false; // if true, the tool resolves paths and runs against the working directory
std::string mcp_server; // the MCP server the tool comes from, empty for a built-in tool

virtual ~server_tool() = default;
virtual json get_definition() const = 0;
Expand Down
3 changes: 3 additions & 0 deletions tools/server/tests/unit/test_mcp_servers.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ def get_tool_name(t):
# Verify tool structure
echo_tool = next(t for t in mcp_tools if get_tool_name(t) == "echo_echo")
assert "description" in echo_tool or "definition" in echo_tool
assert echo_tool["type"] == "mcp"
assert echo_tool["mcp_server"] == "echo"
assert all("mcp_server" not in t for t in tools if t["type"] != "mcp")
finally:
server.stop()

Expand Down
3 changes: 3 additions & 0 deletions ui/apps/chat/.storybook/preview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { SurfaceTheme } from '#lib/enums/ui.enums.ts';
import { applySurfaceTheme } from '#lib/utils/css.ts';
// the docs pages' styles, shared with the library's Storybook
import '../../../lib/ui/.storybook/docs.css';
import { activateBuiltInExtensions } from '../tests/stories/helpers/extensions';
import { applyColorScheme } from './color-scheme';
import ModeWatcherDecorator from './decorators/ModeWatcherDecorator.svelte';
import TooltipProviderDecorator from './decorators/TooltipProviderDecorator.svelte';
Expand All @@ -13,6 +14,8 @@ applyColorScheme();
window.matchMedia(MEDIA_QUERIES.PREFERS_DARK).addEventListener('change', applyColorScheme);

const preview: Preview = {
// the app switches its extensions on at startup; the stories get them the same way, once
beforeAll: activateBuiltInExtensions,
parameters: {
controls: {
matchers: {
Expand Down
31 changes: 29 additions & 2 deletions ui/apps/chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,12 +87,13 @@ The app imports the library as `@ggml-org/llama-ui/ui/<name>` and compiles its s
| Layer | Lives in | What it holds |
| -------------- | ------------------------- | ---------------------------------------------------------------------------------------------------- |
| Routes | `src/routes` | `/`, `/chat/[id]`, `/models`, `/mcp-servers`, `/settings`, `/search` |
| App components | `src/lib/components/app` | Chat, navigation, models, MCP, settings and dialogs, grouped by area |
| App components | `src/lib/components/app` | Chat, navigation, models, settings and dialogs, grouped by area |
| UI components | `ui/lib/ui` | shadcn-svelte on bits-ui, from Llama UI |
| Hooks | `src/lib/hooks` | View logic between components and stores: scroll, drag and drop, pickers, keyboard |
| Stores | `src/lib/stores` | Application state as Svelte 5 runes: chat, conversations, the agentic loop, tools, models, settings |
| Agentic loop | `ui/lib/harness` | Llama Harness: streams a turn, runs the tools the model called, repeats until it answers without one |
| Services | `src/lib/services` | Stateless I/O: the llama-server API, IndexedDB, MCP, the Hugging Face Hub, the tool sandbox |
| Services | `src/lib/services` | Stateless I/O: the llama-server API, IndexedDB, the Hugging Face Hub, the tool sandbox |
| Extensions | `src/lib/extensions` | Features that can be switched off, MCP first: what they add to the app's extension points |
| Storage + API | the browser, llama-server | IndexedDB and localStorage; llama-server's `/v1`, `/props`, `/models`, `/tools`, `/slots` |

```mermaid
Expand All @@ -105,6 +106,7 @@ flowchart LR
Services --> DB[("IndexedDB, localStorage")]
Services --> API["llama-server"]
Components -. "imports" .-> UI["Llama UI"]
Ext["Extensions, such as MCP"] -. "extension points" .-> Stores
Stores -. "reactive state" .-> Components
```

Expand Down Expand Up @@ -143,6 +145,31 @@ sequenceDiagram
- **Modalities**: attachments a model cannot read (images for a text-only model, say) are refused before sending, from the model's `/props`.
- **Storage**: conversations and messages in IndexedDB (Dexie), settings and model preferences in localStorage, server state and Hub details in memory only.

### Extensions (`src/lib/extensions/`)

An extension is a feature the app is composed with and that can be switched off. The registry (`registry.svelte.ts`) knows the built-in extensions (`builtin.ts`), activates the ones that are on at startup, and holds what they add to each extension point (`points/`): tools, flow hooks, sidebar entries, pages, slash commands, pickers, pending attachments, and message and attachment renderers.

The core never imports an extension's own code; an extension may import the core. ESLint enforces this: a file outside `src/lib/extensions/<id>/` cannot import from that folder.

Extensions are switched on and off in Settings, in the Extensions section. A switch takes effect at once: the extension's contributions go away and its cleanups run (MCP closes its connections). The choice is kept in localStorage and read at startup.

MCP is the first extension. All of its code is in `src/lib/extensions/mcp/`:

| Part | Responsibility |
| -------------------------- | ------------------------------------------------------------------------------------------------- |
| `index.ts` and glue | The extension: what MCP adds to each extension point (`tools.ts`, `composer.ts`, `pages.ts`, ...) |
| `stores/` (`mcpStore`) | MCP host role: multi-server lifecycle, tool routing, server icons; composes `health`, `resources` |
| `services/` (`MCPService`) | MCP protocol: transports, connect, list/execute tools, prompts, resources |
| `components/` | Server cards and forms, resource browser, dialogs, the prompt picker and the chat renderers |

MCP's shared data stays in the core: types, enums, constants, `utils/mcp.ts`, settings keys and the stored message formats.

MCP servers come two ways. A remote server (Streamable HTTP, SSE or WebSocket) is one the browser connects to by URL, added on the MCP servers page. A local server that runs as a command (stdio) is one llama-server starts from `--mcp-servers-config`, since a page cannot start a process: llama-server lists its tools in `/tools` with the type `mcp` and the server's name in `mcp_server`, and runs them on `POST /tools`. The MCP extension takes those tools over from the core's server tools (`serverToolType` on its tool provider, see `llama-server.ts`), so both kinds show up the same way, one group per server, on the MCP servers page and in the tools menu. With the extension off, the core lists them as server tools again.

A setting can name the extension it belongs to (`extension` in `SETTINGS_REGISTRY`): its field shows only while that extension is on, and the stored value stays either way.

llama-server embeds the app as a single bundle, so an extension's code ships whether it is on or off; nothing of it runs while it is off.

### Cross-origin isolation

llama-server serves the UI with `Cross-Origin-Embedder-Policy: require-corp` and `Cross-Origin-Opener-Policy: same-origin`, so every cross-origin asset needs the other origin's consent. Hugging Face Hub avatars and metadata are requested in CORS mode, which the Hub allows. MCP server icons and search-result favicons go through llama-server's `/cors-proxy` when it runs with `--ui-mcp-proxy`, and fall back to a bundled icon otherwise.
Expand Down
8 changes: 4 additions & 4 deletions ui/apps/chat/demo/fixtures/mcp-servers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
*/

import { DEMO_MCP_PORT, DEMO_MCP_SERVERS, type DemoMcpServer } from './demo-config.ts';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
import { NodeStreamableHTTPServerTransport } from '@modelcontextprotocol/node';
import { McpServer } from '@modelcontextprotocol/server';
import { createServer, type IncomingMessage, type ServerResponse } from 'node:http';
import { z } from 'zod';

Expand All @@ -19,7 +19,7 @@ function buildServer(fixture: DemoMcpServer): McpServer {
for (const tool of fixture.tools) {
server.registerTool(
tool.name,
{ description: tool.description, inputSchema: { query: z.string().optional() } },
{ description: tool.description, inputSchema: z.object({ query: z.string().optional() }) },
async () => ({ content: [{ text: tool.result, type: 'text' }] })
);
}
Expand Down Expand Up @@ -72,7 +72,7 @@ const httpServer = createServer(async (req, res) => {
}

const server = buildServer(fixture);
const transport = new StreamableHTTPServerTransport({
const transport = new NodeStreamableHTTPServerTransport({
enableJsonResponse: true,
sessionIdGenerator: undefined
});
Expand Down
89 changes: 89 additions & 0 deletions ui/apps/chat/eslint.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import simpleImportSort from 'eslint-plugin-simple-import-sort';
import storybook from 'eslint-plugin-storybook';
import svelte from 'eslint-plugin-svelte';
import globals from 'globals';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import ts from 'typescript-eslint';

Expand Down Expand Up @@ -157,6 +158,86 @@ const blankLineBetweenAccessors = {
type: 'layout'
}
};
// The core never imports an extension's own code; an extension may import the
// core. A module under src/lib/extensions/<id>/ can only be imported by a file
// in that same folder. The registry's own files (the index, builtin,
// registry.svelte, types and points/*) stay importable from anywhere, and
// builtin.ts loads each extension's entry with a dynamic import.
const LIB_DIR = fileURLToPath(new URL('./src/lib', import.meta.url));
const EXTENSIONS_DIR = path.join(LIB_DIR, 'extensions');
const EXTENSIONS_PUBLIC_FILES = new Set(['builtin', 'index', 'registry.svelte', 'types']);
const noExtensionInternals = {
create(context) {
// path of a file inside src/lib/extensions as segments, or null for a file outside it
function extensionSegments(file) {
const relative = path.relative(EXTENSIONS_DIR, file);

if (relative.startsWith('..') || path.isAbsolute(relative)) return null;

return relative === '' ? [] : relative.split(path.sep);
}

function check(node, source, isDynamic) {
if (typeof source !== 'string') return;

let target;

if (source.startsWith('#lib/')) {
target = path.join(LIB_DIR, source.slice('#lib'.length));
} else if (source.startsWith('.')) {
target = path.resolve(path.dirname(context.filename), source);
} else {
return;
}

const segments = extensionSegments(target);

if (!segments || segments.length === 0 || segments[0] === 'points') return;

const id = segments[0];

if (segments.length === 1 && EXTENSIONS_PUBLIC_FILES.has(id.replace(/\.[jt]s$/, ''))) return;

const importer = extensionSegments(context.filename);

if (importer && importer.length > 1 && importer[0] === id) return;

const isEntry =
segments.length === 1 || (segments.length === 2 && /^index(\.[jt]s)?$/.test(segments[1]));

// the one place that names the extensions: it loads the entry of each, lazily
if (isDynamic && isEntry && importer?.length === 1 && importer[0] === 'builtin.ts') return;

context.report({ data: { id, source }, messageId: 'internal', node });
}

return {
ExportAllDeclaration(node) {
check(node, node.source.value, false);
},
ExportNamedDeclaration(node) {
if (node.source) check(node, node.source.value, false);
},
ImportDeclaration(node) {
check(node, node.source.value, false);
},
ImportExpression(node) {
if (node.source.type === 'Literal') check(node, node.source.value, true);
}
};
},
meta: {
docs: {
description: "Keep an extension's own code out of the rest of the app."
},
messages: {
internal:
'"{{source}}" is the own code of the "{{id}}" extension. Only files in src/lib/extensions/{{id}}/ can import it; the rest of the app uses the extension points.'
},
schema: [],
type: 'problem'
}
};

export default ts.config(
includeIgnoreFile(gitignorePath),
Expand All @@ -171,6 +252,7 @@ export default ts.config(
local: {
rules: {
'blank-line-between-accessors': blankLineBetweenAccessors,
'no-extension-internals': noExtensionInternals,
'padding-line-between-elements': paddingLineBetweenElements
}
},
Expand All @@ -189,6 +271,8 @@ export default ts.config(
'eol-last': 'error',
// Enforce a blank line between consecutive get/set accessors
'local/blank-line-between-accessors': 'error',
// The core does not import an extension's own code
'local/no-extension-internals': 'error',
// Require a blank line between sibling elements in a Svelte template
'local/padding-line-between-elements': 'error',
// typescript-eslint strongly recommend that you do not use the no-undef lint rule on TypeScript projects.
Expand Down Expand Up @@ -326,6 +410,11 @@ export default ts.config(
files: ['src/**/*.ts'],
languageOptions: { parserOptions: { projectService: true } }
},
{
// Tests and stories exercise an extension's code directly
files: ['tests/**'],
rules: { 'local/no-extension-internals': 'off' }
},
{
// Exclude generated build output and Storybook files from ESLint
ignores: [
Expand Down
Loading
Loading