Skip to content

ui: run Llama Chat with npx @ggml-org/llama-chat - #69

Draft
allozaur wants to merge 2 commits into
allozaur/docs/llama-ui-storybookfrom
allozaur/feat/llama-chat-npx
Draft

allozaur wants to merge 2 commits into
allozaur/docs/llama-ui-storybookfrom
allozaur/feat/llama-chat-npx

Conversation

@allozaur

@allozaur allozaur commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Overview

Llama Chat can be published to npm and run with npx @ggml-org/llama-chat (or npx llama-chat). It serves the app on localhost and connects it to a llama-server, so you can use it locally or self-host it without building it.

Additional information

Part 2 of 13 of the work-in-progress UI stack on the fork, based on allozaur/docs/llama-ui-storybook (#67), whose Architecture › Channels page served as the spec. Voice messages and dictation (#64) and the rest of the stack sit on top of it. Nothing is published to npm: the packages go out by hand with an npm one-time password, as @ggml-org/llama-ui does.

Known gap: chat.llama.app is not deployed here. dist is ready for any static host, but until the app can add a server by URL (#35), a hosted copy reaches only a llama-server on its own origin.

Where the docs disagreed, and what this PR picks
  • Package name: the Channels page wanted an unscoped llama-chat with @ggml-org/llama-chat as the import name. npx llama-chat only resolves an unscoped package, and npx @ggml-org/llama-chat only a scoped one, so both exist. @ggml-org/llama-chat carries the app and the CLI (bin: llama-chat). llama-chat (in alias/, already reserved on npm as a 0.0.0 placeholder) depends on it at the same version and only runs its CLI.
  • --server: the page has it add and select the server in the app, and the app open on an add-server screen without it. Neither exists at this point in the stack (adding servers by URL comes with ui: run chat against more llama.cpp servers, added by URL or paired through a tunnel #35). Until then the CLI forwards the app's API calls to --server, which defaults to llama-server's own http://127.0.0.1:8080. The Channels page now says so.
  • Port: the page asks for "a fixed port" and draws localhost:3000, so the default is 3000.
  • Version: 0.1.0 for the first release, down from the unused 1.0.0. The page's "versioned with the release" can follow once a release job publishes it.
  • Publish workflow: none. @ggml-org/llama-ui has none in the stack either, and the page's release flow (one CI job per llama.cpp release) is a separate change.
What changed
  • package.json: private is gone. It adds bin, files (bin, cli and dist without source maps), exports (./cli, ./server, ./dist/*), repository, license, engines (Node 20+) and publishConfig.access: public. There is no prepack build: this folder's .npmrc sets ignore-scripts, so npm would never run it. The publishing steps build dist explicitly and check the packed file list instead.
  • CLI (bin/cli.js, cli/server.js): plain node:http, no dependencies. A request is answered from dist when a file there matches its path, and forwarded to the llama-server otherwise, without its Origin and with the server's Host, as the Vite dev proxy does, so the server needs no --cors-origins. Streams pass through, and closing one stops the generation upstream. Static files get the headers llama-server sends for the UI: index.html cross-origin isolated and revalidated, service-worker.js, sw.js, the manifest, build.json and _app/version.json revalidated, everything else cached for good, with ETags. Flags: --server, --port, --host, --open/--no-open, --version, --help. start is accepted as an alias for the bare command. A busy port fails with a message rather than moving, since the stored chats belong to the origin. A --server path is kept, for servers started with --api-prefix.
  • alias/: the unscoped llama-chat package.
  • README: Run it with npx, Self-hosting, and Publishing to npm.
  • Storybook: Architecture › Channels describes the command as it ships.
  • LICENSE: MIT, as in ui/lib/ui, so the package carries it.
Testing
  • npm ci, then npm run check (0 errors), npm run lint, and npx vitest --project=unit --run (803 tests on the SvelteKit 3 stack, 11 of them new in tests/unit/cli.test.ts: flags, static headers and 304s, forwarding with Origin stripped and the prefix kept, no paths outside dist, 502 when the server is down, and the alias kept on the app's version) all pass.
  • npm run build, then npm pack: 3.5 MB, 77 files, no source maps.
  • npx ./ggml-org-llama-chat-0.1.0.tgz --server http://127.0.0.1:9931 --no-open against a running llama-server in router mode, before and after the SvelteKit 3 restack: the model list loaded, and a message streamed a reply. Moving from the old build to the new one, the open tab showed the update prompt, so the revalidated service worker picks up a new version.
  • Both tarballs installed together: npx llama-chat --version prints 0.1.0 through the alias.

Requirements

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Nov 16, 2026
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 71ce01e to fc4995e Compare November 16, 2026 16:12
@allozaur
allozaur added this pull request to stack #71 November 16, 2026 16:13
@allozaur
allozaur removed this pull request from stack #71 November 16, 2026 16:49
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from f96dd32 to 7499bd4 Compare November 16, 2026 16:50
@allozaur
allozaur added this pull request to stack #72 November 16, 2026 16:50
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 7499bd4 to 4484a36 Compare November 16, 2026 17:36
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 4484a36 to 33387cb Compare November 16, 2026 18:58
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 33387cb to 21c2294 Compare November 17, 2026 05:21
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 21c2294 to a75153d Compare November 17, 2026 05:55
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from a75153d to b69e0d5 Compare November 17, 2026 10:31
@allozaur
allozaur removed this pull request from stack #72 November 17, 2026 11:15
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from b69e0d5 to b6f2fe6 Compare November 17, 2026 11:16
@allozaur
allozaur added this pull request to stack #76 November 17, 2026 11:16
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch 2 times, most recently from d00fa73 to d529f0a Compare November 17, 2026 12:44
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from d529f0a to 14f2bf9 Compare November 17, 2026 13:05
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 14f2bf9 to 6a8df71 Compare November 17, 2026 13:13
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 6a8df71 to 04f251b Compare November 17, 2026 13:20
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 04f251b to 7ce9420 Compare November 17, 2026 13:44
@allozaur
allozaur removed this pull request from stack #76 November 17, 2026 13:44
@allozaur
allozaur added this pull request to stack #78 November 17, 2026 13:44
@allozaur
allozaur removed this pull request from stack #78 November 17, 2026 16:25
@allozaur
allozaur added this pull request to stack #79 November 17, 2026 16:26
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch 4 times, most recently from fb34676 to 4441784 Compare November 17, 2026 17:30
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 4441784 to 390311e Compare November 17, 2026 22:00
Makes @ggml-org/llama-chat publishable: drops private, adds bin,
files, exports, repository and a prepack build. Its CLI serves the
prebuilt dist on localhost (3000 by default) and forwards every other
request to a llama-server (--server, default http://127.0.0.1:8080),
with the headers llama-server sends for the UI. A thin unscoped
llama-chat package in alias/ forwards to it, so npx llama-chat works.

Assisted-by: Claude (Opus 5.5 Medium)
…ore packing

SvelteKit 3 adds service-worker.js next to sw.js, and llama-server
revalidates it, so the CLI does too. The folder's .npmrc sets
ignore-scripts, so prepack never ran: it goes, and the publishing steps
build dist explicitly and check the packed file list.

Assisted-by: Claude (Opus 5.5 Medium)
@allozaur
allozaur force-pushed the allozaur/feat/llama-chat-npx branch from 390311e to 4312fbd Compare November 17, 2026 22:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation server/ui

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant