Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
ea58985
ui : add the providers data layer
allozaur Sep 28, 2026
7f44826
ui : route chat through the active provider
allozaur Sep 28, 2026
e89f121
ui : add the manage providers view
allozaur Sep 28, 2026
8b189e0
ui : show every provider's models in the manager
allozaur Sep 28, 2026
ae13938
ui : resolve the layout probes through the providers api url
allozaur Oct 6, 2026
adec94e
test : seed the picker drawer through the providers data layer
allozaur Oct 8, 2026
1a44f63
ui : keep the selector list open on provider backends
allozaur Oct 8, 2026
2ba6a98
ui : show the selected model of the compat backends
allozaur Oct 8, 2026
884dce9
ui : hide the chat template on the remote providers
allozaur Oct 8, 2026
dd12664
ui : offer the compat providers a selection instead of a load
allozaur Oct 8, 2026
a5b2202
ui : keep the table status working while a provider is active
allozaur Oct 8, 2026
402503e
ui : keep llama.cpp servers as the only providers
allozaur Oct 19, 2026
64fbcc0
ui : address remote providers through a tunnel
allozaur Oct 19, 2026
d589aa1
ui : pair a provider by code or camera
allozaur Oct 19, 2026
a93ef7e
ui : cover the provider transport with tests
allozaur Oct 19, 2026
ccfd4a1
ui : rename the provider layer to servers
allozaur Oct 20, 2026
54e1bd5
ui : rebuild the server card around what it reports
allozaur Oct 20, 2026
27350ad
ui : recover a dropped channel when the tab comes back
allozaur Oct 20, 2026
b3af6a2
ui : show the access code as a QR code
allozaur Oct 20, 2026
6ad8d5c
ui : give servers a dialog of their own
allozaur Oct 20, 2026
d56bdb2
ui : add a server from a drawer on a phone
allozaur Oct 20, 2026
30af466
test : cover the servers surface
allozaur Oct 20, 2026
1e3f136
ui : keep a server's API key when it is read back
allozaur Oct 20, 2026
74c28b5
ui : refuse an endpoint that is not a llama.cpp server
allozaur Oct 20, 2026
31ac099
ui : offer pairing where no server can answer
allozaur Oct 20, 2026
0ce919e
ui : read a remote router's status feed
allozaur Oct 20, 2026
8855c8d
ui : adapt the servers work to the stack
allozaur Nov 2, 2026
c04b7ef
ui : open Servers as a page on a phone, a dialog elsewhere
allozaur Nov 2, 2026
fecf80f
ui: keep local-only requests on the local server, fix hidden models
allozaur Nov 9, 2026
1a51a4a
ui : put remote server pairing behind an experimental setting
allozaur Nov 9, 2026
10110cd
ui: send voice requests to the active server
allozaur Nov 16, 2026
a8fdea5
ui: forget llama-server's MCP servers with its other tools
allozaur Nov 16, 2026
dbc66f2
ui: npx llama-chat --server adds a llama-server to the app
allozaur Nov 18, 2026
00f9404
ui: ask for llama-chat.json only where the CLI put it
allozaur Nov 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions ui/apps/chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,22 @@ It works with a llama-server in either mode: **MODEL** (one model) or **ROUTER**
Llama Chat is published to npm as [`@ggml-org/llama-chat`](https://www.npmjs.com/package/@ggml-org/llama-chat), with [`llama-chat`](https://www.npmjs.com/package/llama-chat) as its short name. It serves the prebuilt app as static files on localhost, or writes them to a folder for any static file server (see [Self-hosting](#self-hosting)); it brings no server and no models of its own. It needs Node.js 20 or later.

```bash
npx @ggml-org/llama-chat # serve it on http://127.0.0.1:3000 and open the browser
npx llama-chat --host 0.0.0.0 --no-open # the same, by its short name, reachable from other devices
npx llama-chat build # write the static app to ./llama-chat instead
npx @ggml-org/llama-chat --server http://127.0.0.1:9931 # serve it on http://127.0.0.1:3000 with that server added
npx llama-chat --host 0.0.0.0 --no-open # the same, by its short name, reachable from other devices
npx llama-chat build --server https://llm.example.com # write the static app to ./llama-chat instead
```

| Flag | Default | What it does |
| --------------------------------- | ----------- | ------------------------------------------------------------------------------------------- |
| `--server <url>` | | A llama-server the app adds on startup, once; repeatable. Its API key goes in the app |
| `--port <n>` | `3000` | Fixed, so the origin, and the chats the browser stores under it, stay the same between runs |
| `--host <addr>` | `127.0.0.1` | `0.0.0.0` to open it to the LAN |
| `--open` / `--no-open` | open | Opens the default browser once it listens |
| `-v`, `--version`, `-h`, `--help` | | As usual |

The bare command is the documented form; `start` is accepted as an alias for it. `build` takes `--out <dir>` (default `./llama-chat`) and `--force`, which lets it write into a folder that is not empty.
The bare command is the documented form; `start` is accepted as an alias for it. `build` takes `--out <dir>` (default `./llama-chat`), `--force`, which lets it write into a folder that is not empty, and `--server`.

How it works: the CLI only serves the files of `dist`, with llama-server's headers for the UI (`index.html` cross-origin isolated, the PWA files revalidated, hashed assets cached for good), and forwards nothing. The browser talks to each llama-server you add in the app by URL directly, so llama-server's own rules apply: start it with `--cors-origins http://127.0.0.1:3000` (the address you open) when it restricts origins, as it does with tools or MCP enabled, and with `--api-key` when it is reachable from the LAN. An `http://` page like this one can reach a plain `http://` llama-server on the LAN, which an `https://` page cannot.
How it works: the CLI only serves the files of `dist`, with llama-server's headers for the UI (`index.html` cross-origin isolated, the PWA files revalidated, hashed assets cached for good), and forwards nothing. `--server` only puts the address in `llama-chat.json` next to `index.html`, which the app reads at startup to add the server; one you remove in the app stays removed. The CLI marks `index.html` with `<meta name="llama-chat-servers" content="llama-chat.json" />`, and only a page with that tag asks for the file (`build --server` writes both). The browser talks to each llama-server directly, so llama-server's own rules apply: start it with `--cors-origins http://127.0.0.1:3000` (the address you open) when it restricts origins, as it does with tools or MCP enabled, and with `--api-key` when it is reachable from the LAN. An `http://` page like this one can reach a plain `http://` llama-server on the LAN, which an `https://` page cannot.

## Self-hosting

Expand Down
135 changes: 118 additions & 17 deletions ui/apps/chat/cli/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,15 @@
*
* It only serves files and forwards nothing: the browser calls each llama-server the app is
* pointed at directly, so llama-server's own rules (--cors-origins, --api-key) apply as they do
* to any other page.
* to any other page. `--server` only tells the app which servers to add: the CLI serves them in
* `llama-chat.json` next to index.html, and `build` writes that file, which the app reads at
* startup. index.html carries a marker for it, so a page served any other way, such as by
* llama-server or the dev server, does not ask for a file that is not there.
*/

import { spawn } from 'node:child_process';
import { createReadStream, readFileSync } from 'node:fs';
import { cp, readdir, stat } from 'node:fs/promises';
import { cp, readdir, readFile, stat, writeFile } from 'node:fs/promises';
import http from 'node:http';
import { extname, isAbsolute, join, normalize, relative, resolve, sep } from 'node:path';
import { fileURLToPath } from 'node:url';
Expand All @@ -28,6 +31,15 @@ export const DEFAULTS = Object.freeze({
port: 3000
});

/** The file next to index.html that lists the servers the app adds at startup. */
export const PRESET_FILE = 'llama-chat.json';

/**
* The marker in index.html that sends the app to `llama-chat.json`. It names the file rather
* than the servers: the service worker keeps index.html, so servers in it would outlive a run.
*/
export const PRESET_META = `<meta name="llama-chat-servers" content="${PRESET_FILE}" />`;

// The same caching llama-server applies to the UI it embeds (tools/server/server-http.cpp):
// index.html and the PWA's update checks are revalidated, hashed assets are cached for good.
const NO_CACHE = new Set([
Expand All @@ -36,7 +48,8 @@ const NO_CACHE = new Set([
'/sw.js',
'/manifest.webmanifest',
'/_app/version.json',
'/build.json'
'/build.json',
`/${PRESET_FILE}`
]);
const CACHE_IMMUTABLE = 'public, max-age=31536000, immutable';
const CACHE_REVALIDATE = 'no-cache';
Expand Down Expand Up @@ -65,13 +78,15 @@ const SERVE_OPTIONS = ['host', 'open', 'port'];
const BUILD_OPTIONS = ['force', 'out'];

export const USAGE = `Usage: llama-chat [options]
llama-chat build [--out <dir>] [--force]
llama-chat build [--out <dir>] [--force] [--server <url>]

Serves Llama Chat on localhost as static files, or, with build, writes them to
a folder for any HTTP server. The browser talks to llama-server directly: start
it with --cors-origins set to this page's address when it restricts origins.

Options:
--server <url> a llama-server the app adds on startup, repeatable; the API
key, if it needs one, goes in the app
--port <n> port to listen on (default: ${DEFAULTS.port})
--host <addr> address to listen on, 0.0.0.0 for the LAN (default: ${DEFAULTS.host})
--open open the browser once it listens (default)
Expand All @@ -82,11 +97,12 @@ Options:
Build options:
--out <dir> folder to write to (default: ./${DEFAULTS.out})
--force write into a folder that is not empty
--server <url> as above, written to ${PRESET_FILE} in the folder

Examples:
npx @ggml-org/llama-chat
npx @ggml-org/llama-chat --server http://127.0.0.1:9931
npx @ggml-org/llama-chat --host 0.0.0.0 --port 3000 --no-open
npx @ggml-org/llama-chat build --out ./public`;
npx @ggml-org/llama-chat build --out ./public --server https://llm.example.com`;

/** The version in this package's package.json. */
export function packageVersion() {
Expand All @@ -97,7 +113,7 @@ export function packageVersion() {
* Reads the command line. No verb serves the app, `start` means the same, and `build` writes it
* to a folder.
* @param {string[]} argv the arguments after the command
* @returns {{ command: 'build' | 'serve', force: boolean, help: boolean, host: string, open: boolean, out: string, port: number, version: boolean }}
* @returns {{ command: 'build' | 'serve', force: boolean, help: boolean, host: string, open: boolean, out: string, port: number, servers: string[], version: boolean }}
*/
export function parseCliArgs(argv) {
const { positionals, values } = parseArgs({
Expand All @@ -111,6 +127,7 @@ export function parseCliArgs(argv) {
open: { type: 'boolean' },
out: { type: 'string' },
port: { type: 'string' },
server: { multiple: true, type: 'string' },
version: { short: 'v', type: 'boolean' }
},
strict: true
Expand Down Expand Up @@ -146,10 +163,54 @@ export function parseCliArgs(argv) {
open: values.open ?? true,
out: values.out ?? DEFAULTS.out,
port,
servers: (values.server ?? []).map(parseServerUrl),
version: values.version ?? false
};
}

/**
* A llama-server address for the app, with or without `http://`, as the app stores it: no
* trailing slash, a path kept (for `--api-prefix`).
* @param {string} value
*/
export function parseServerUrl(value) {
let url;

try {
url = new URL(/^[a-z][a-z\d+.-]*:\/\//i.test(value) ? value : `http://${value}`);
} catch {
throw new Error(`invalid --server: ${value}`);
}

if (url.protocol !== 'http:' && url.protocol !== 'https:') {
throw new Error(`invalid --server: ${value} (use an http:// or https:// address)`);
}

if (url.search || url.hash || url.username || url.password) {
throw new Error(`invalid --server: ${value} (no query, fragment or credentials)`);
}

return url.href.replace(/\/+$/, '');
}

/**
* index.html with the marker for `llama-chat.json` at the end of its head, once.
* @param {string} html
*/
export function withPresetMeta(html) {
if (html.includes('name="llama-chat-servers"')) return html;

return html.replace('</head>', `\t${PRESET_META}\n\t</head>`);
}

/**
* The contents of `llama-chat.json`.
* @param {string[]} servers
*/
export function presetFile(servers) {
return `${JSON.stringify({ servers: servers.map((url) => ({ url })) }, null, '\t')}\n`;
}

/**
* Maps a request path to a file under `root`, or null when it falls outside it.
* @param {string} root
Expand Down Expand Up @@ -181,7 +242,10 @@ function resolveStatic(root, pathname) {
*/
function sendFile(req, res, file, pathname, stats) {
const name = pathname === '/' ? '/index.html' : pathname;
const etag = `W/"${stats.size.toString(16)}-${Math.floor(stats.mtimeMs).toString(16)}"`;
// index.html goes out with the marker for llama-chat.json, and an ETag of its own
const body =
name === '/index.html' ? Buffer.from(withPresetMeta(readFileSync(file, 'utf8'))) : null;
const etag = `W/"${(body?.length ?? stats.size).toString(16)}-${Math.floor(stats.mtimeMs).toString(16)}"`;

res.setHeader('ETag', etag);
res.setHeader('Cache-Control', NO_CACHE.has(name) ? CACHE_REVALIDATE : CACHE_IMMUTABLE);
Expand All @@ -201,7 +265,7 @@ function sendFile(req, res, file, pathname, stats) {
'Content-Type',
MIME_TYPES[extname(file).toLowerCase()] ?? 'application/octet-stream'
);
res.setHeader('Content-Length', stats.size);
res.setHeader('Content-Length', body?.length ?? stats.size);
res.writeHead(200);

if (req.method === 'HEAD') {
Expand All @@ -210,6 +274,12 @@ function sendFile(req, res, file, pathname, stats) {
return;
}

if (body) {
res.end(body);

return;
}

createReadStream(file).pipe(res);
}

Expand All @@ -224,11 +294,13 @@ function sendError(res, status, message) {
}

/**
* The HTTP server: the files of `dist`, read-only.
* @param {{ distDir?: string }} [options]
* The HTTP server: the files of `dist`, read-only, and `llama-chat.json` with the servers given,
* if any: index.html always points the app at it.
* @param {{ distDir?: string, servers?: string[] }} [options]
*/
export function createChatServer({ distDir = DIST_DIR } = {}) {
export function createChatServer({ distDir = DIST_DIR, servers = [] } = {}) {
const root = normalize(distDir);
const preset = Buffer.from(presetFile(servers));

return http.createServer(async (req, res) => {
const { pathname } = new URL(req.url ?? '/', 'http://localhost');
Expand All @@ -240,6 +312,17 @@ export function createChatServer({ distDir = DIST_DIR } = {}) {
return;
}

if (pathname === `/${PRESET_FILE}`) {
res.writeHead(200, {
'Cache-Control': CACHE_REVALIDATE,
'Content-Length': preset.length,
'Content-Type': MIME_TYPES['.json']
});
res.end(req.method === 'HEAD' ? undefined : preset);

return;
}

const file = resolveStatic(root, pathname);
const stats = file ? await stat(file).catch(() => null) : null;

Expand All @@ -255,11 +338,12 @@ export function createChatServer({ distDir = DIST_DIR } = {}) {

/**
* Writes the app to a folder: the files of `dist` without source maps, ready for any static
* file server. Refuses a folder that is not empty unless `force` is set, and a folder in `dist`.
* @param {{ distDir?: string, force?: boolean, out: string }} options
* file server, and `llama-chat.json` when servers are given, with the marker in index.html. Refuses a folder that is not empty
* unless `force` is set, and a folder in `dist`.
* @param {{ distDir?: string, force?: boolean, out: string, servers?: string[] }} options
* @returns {Promise<string>} the absolute path of the folder
*/
export async function buildStatic({ distDir = DIST_DIR, force = false, out }) {
export async function buildStatic({ distDir = DIST_DIR, force = false, out, servers = [] }) {
const source = resolve(distDir);
const target = resolve(out);
const inside = relative(source, target);
Expand All @@ -276,6 +360,13 @@ export async function buildStatic({ distDir = DIST_DIR, force = false, out }) {

await cp(source, target, { filter: (file) => !file.endsWith('.map'), recursive: true });

if (servers.length > 0) {
const index = join(target, 'index.html');

await writeFile(join(target, PRESET_FILE), presetFile(servers));
await writeFile(index, withPresetMeta(await readFile(index, 'utf8')));
}

return target;
}

Expand Down Expand Up @@ -339,10 +430,17 @@ export async function main(argv) {

if (options.command === 'build') {
try {
const target = await buildStatic({ force: options.force, out: options.out });
const target = await buildStatic({
force: options.force,
out: options.out,
servers: options.servers
});
const shown = relative(process.cwd(), target) || '.';

console.log(`Llama Chat ${packageVersion()} written to ${target}`);

for (const server of options.servers) console.log(`Adds the llama-server at ${server}`);

console.log(`Serve it with any static file server, e.g.: npx http-server ${shown}`);
} catch (error) {
console.error(`llama-chat: ${error instanceof Error ? error.message : error}`);
Expand All @@ -352,7 +450,7 @@ export async function main(argv) {
return;
}

const server = createChatServer();
const server = createChatServer({ servers: options.servers });

return new Promise((resolve) => {
server.on('error', (error) => {
Expand All @@ -373,6 +471,9 @@ export async function main(argv) {
const url = `http://${shown.includes(':') ? `[${shown}]` : shown}:${address.port}/`;

console.log(`Llama Chat ${packageVersion()} on ${url}`);

for (const server of options.servers) console.log(`Adds the llama-server at ${server}`);

console.log(
`If llama-server restricts origins (tools or MCP on), start it with --cors-origins ${url.slice(0, -1)}`
);
Expand Down
8 changes: 8 additions & 0 deletions ui/apps/chat/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions ui/apps/chat/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@
"typescript-eslint": "8.72.1",
"unified": "11.0.5",
"unist-util-visit": "5.1.0",
"uqr": "0.1.3",
"uuid": "13.0.2",
"vaul-svelte": "1.0.0-next.7",
"vite": "8.3.6",
Expand Down
2 changes: 2 additions & 0 deletions ui/apps/chat/src/app.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import type {
ApiChatCompletionStreamChunk,
ApiChatCompletionToolCall,
ApiChatCompletionToolCallDelta,
ApiChatCompletionUsage,
ApiChatMessageContentPart,
ApiChatMessageData,
ApiContextSizeError,
Expand Down Expand Up @@ -72,6 +73,7 @@ declare global {
ApiChatCompletionResponse,
ApiChatCompletionStreamChunk,
ApiChatCompletionToolCall,
ApiChatCompletionUsage,
ApiChatCompletionToolCallDelta,
ApiChatMessageData,
ApiChatMessageContentPart,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
{/snippet}

{#snippet body()}
{#if gauge.activeModelId !== null && !gauge.isActiveModelLoaded}
{#if gauge.canLoadActiveModel && gauge.activeModelId !== null && !gauge.isActiveModelLoaded}
<ContextGaugeLoadModel
isLoading={gauge.isActiveModelLoading}
modelId={gauge.activeModelId}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,14 @@
</DropdownMenu.Item>
{:else}
{@const source = toolsStore.getToolSource(toolName)}
{@const providerName =
{@const serverName =
source === ToolSource.SERVER
? TOOL_SERVER_LABELS[ToolSource.SERVER]
: source === ToolSource.CUSTOM
? TOOL_SERVER_LABELS[ToolSource.CUSTOM]
: 'MCP Tools'}
<DropdownMenu.Item onclick={() => onDecision(ToolPermissionDecision.ALWAYS_SERVER)}>
Approve all tools from {providerName}
Approve all tools from {serverName}
</DropdownMenu.Item>
{/if}
</DropdownMenu.Content>
Expand Down
Loading