Repository navigation
Conversation
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 3, 2026 10:02
62dbc6b to
8bda0fc
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 9, 2026 10:05
8bda0fc to
4af2ddd
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 9, 2026 10:37
4af2ddd to
39b0d62
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
3 times, most recently
from
November 9, 2026 12:44
7fbabe8 to
e7a2c6b
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 9, 2026 14:35
e7a2c6b to
233d012
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 9, 2026 15:06
233d012 to
8be6ea1
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 9, 2026 15:24
8be6ea1 to
6a2834b
Compare
allozaur
force-pushed
the
allozaur/feat/servers
branch
from
November 9, 2026 16:03
6a2834b to
c4d1026
Compare
A provider is added from a preset card or by hand: the dialog probes the endpoint, reads a refusal as a llama.cpp server that wants a key, and shows the connection test as a status block. Presets carry their official artwork and a saved backend keeps its branding through the favicon fallback. The manager dialog gains its third view, and the settings save stops clobbering keys that other surfaces write. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
The table grows a section per enabled provider, each with its own mark, error and loading state, and a provider filter with live repo counts. Rows gain their capability gates back, the draft column with its use-as-draft action, and the quant badge names the provider on its rows. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
The key check reads the providers api url and waits for the server capabilities, and the model status feed survives tab switches instead of tearing down on every one. Split from the shell polish: the layout depends on the providers layer. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
The models store derives its list from the backends now, so the drawer test seeds the active models instead of assigning the derived list, and runs inside the tooltip provider the app shell supplies. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
The active backend of an external provider has no /props, so its role degrades to MODEL while the selector still has a list to show. The trigger opened the manager instead of the list. Gate the trigger on the selectable list instead of the router role. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
A compat backend serves the selection itself and reports no load state, so the selected model sat inside its provider section, possibly cut off by the display cap. Give it a leading section of its own, in the selector list and the manager table, the way loaded models lead the llama-compat ones. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
A remote provider serves its own template and reports no /props, so the section would never show one. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
The pane passed no canToggleLoad, so the load button defaulted to visible for models of backends that cannot load or unload. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
The status cell branched on the active backend's role, so rows collapsed to the served-by placeholder once a provider was the selected backend, and the state lookup resolved the backend by raw-name first match, which a same-named provider catalog entry shadowed: loading a loaded local model then failed with 'model is already running'. Branch the cell per row instead, resolve the state from the local feed first and the backends' own listings after, and start the local feed while a remote backend is active: the feed reports the local server's models, so the local server's role decides. Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
Drop the OpenAI-compatible backend dialect from the providers work: every backend now speaks the llama.cpp API, so there is nothing to negotiate. Goes with it: the protocol enum and its capability and wire-quirk tables, the adapter registry (it had a single implementation, and both protocols resolved to it), the chat and models path overrides, the endpoint probe that classified a URL as llama or OpenAI, and the hosted-provider presets with their recommendation block. ChatService now builds llama.cpp requests directly and decodes its stream through chat-wire, keeping the timings the server reports instead of the wall-clock fallback a foreign endpoint needs. The gates that read those capabilities collapse to their llama.cpp branch: load, unload, slots, props, the resumable-stream endpoints and pre-encode are simply available. A row of another server loads through that server, and the switch that keeps local server state now keys on the built-in backend rather than on a protocol.
A remote llama.cpp server is one the UI reaches over a WebRTC data channel instead of a base URL. It stays a provider: it is listed, named, enabled and disabled, and its models appear in the selector like everyone else's. What changes is where a provider's requests are addressed. A remote provider has no URL of its own, so it is given a synthetic origin whose host carries its id (`https://<id>.remote.llamacpp.invalid`, the suffix being reserved so the host can never resolve). Every service already builds its URLs through `apiUrl`, so nothing in the services had to learn about tunnels: `getBackendBaseUrl` hands back the synthetic origin, and a fetch wrapper installed in `hydrateStores` routes a request by that host instead of by the page origin. Routing on the address is what makes this safe to run next to other providers. The first remote-access build tunneled everything aimed at the page origin, so one tunnel was the most it could hold, and a remote provider with a URL of its own would have slipped past it. Now a request aimed at a tunnel origin with no live tunnel is refused out loud, and because the origin cannot resolve, a call site that ever bypasses the resolver dies in the browser rather than reaching the server that happens to serve this UI. A request queued while the handshake ran is refused the same way if the channel died underneath it. Tunnels open lazily: only the provider the app opens with is handshaked during startup, the rest when one of their models is picked or Connect is hit on their card. A provider that is disabled or removed loses its data channel, and its code - which is a credential - is dropped with it. Three things this forces into the open, all of them pinned to the built-in provider back when no second provider could exist: - `validateApiKey` probed the active provider with the key of the local one, so a remote provider answering 401 raised the API key splash over a credential belonging to another machine. - `streamTool` ran tools on the local server while the model asking for one could be anywhere. A tool now runs where its tokens are generated. - settings export scrubbed `apiKey` and MCP headers but exported the provider list whole, keys included. Remote providers now leave the export entirely: without their code they name nothing, and with it they are a key. The tunnel itself comes from ggml-org#24577 (ngxson), with the base-path stripping dropped: a request already carries the path the host's server serves. Co-Authored-By: ngxson <ngxson@gmail.com>
The providers view gains the second kind of provider. A remote one shows its code masked with a reveal, reports the tunnel and the server behind it as two things that can fail - Connecting, N models, Disconnected, or Not connected with a Connect button - and forgets its code when it is deleted. Pairing is a code, and 40 characters of near-nonsense is a poor thing to type on a phone, so a browser that ships a native BarcodeDetector gets a Scan button that reads the QR code llama-server prints with --connect. No dependency, no camera unless the user asks, and everywhere the detector is missing the field is simply filled by hand. Pairing saves the provider and then opens its tunnel, because the tunnel belongs to the provider and has to be able to find it. A failed handshake therefore leaves the provider in place with a retry on its card rather than rolling the pair back: the code the user pasted is worth keeping. With no local server and no provider the view says so plainly - this page has no server of its own - and offers the code first, which is the state a build served from the network opens in. That replaces the blocking splash of the first remote build: the app boots, the built-in provider is dropped once its probe fails, and what is left is an empty provider list that knows what to ask for.
The addressing rules are the cheap part to check and the expensive part to get wrong: a provider id that stops round-tripping through its host, a code alphabet that quietly accepts a pasted `I`, or a stored list that lets a URL provider claim the tunnel host all mean a request goes somewhere it should not. The client test drives the installed transport itself and asserts the part that matters most - a refused request left nothing on the server serving the UI - and the unit test pins the shapes the transport reads: codes, origins, and the provider list that decides what may be routed at all.
The provider and backend vocabulary is retired across the data layer, the stores, the components and the tests: the list, the view and the noun are servers. Assisted-by: DeepSeek-V4.1-Flash
A local card shows the host side of a server started with --connect, the status words name the state instead of hiding it in a tooltip, a stalled handshake can be dropped, and the access code can be copied off the card. Assisted-by: DeepSeek-V4.1-Flash
A backgrounded tab on iOS or Android loses its peer connection, which leaves the card on a failure the user can only clear by hand. Coming back now asks for the channels that were already wanted, once the network has settled. Assisted-by: DeepSeek-V4.1-Flash
The code a host prints can be scanned off the screen by the phone that pairs with it, and the settings export says it carries access codes. Assisted-by: DeepSeek-V4.1-Flash
The models dialog stays desktop-only, so a phone had no way into its server list; the sidebar now carries a Servers entry that opens the list on its own. Assisted-by: DeepSeek-V4.1-Flash
The add and pair form becomes a bottom sheet on a phone: its actions stay in thumb reach and its fields scroll under them, while a desktop keeps the dialog. Assisted-by: DeepSeek-V4.1-Flash
The card's state table, the tunnel recovery and a handshake the user gives up on, the QR round trip through the browser's own detector, and the phone's way into the servers dialog. Assisted-by: DeepSeek-V4.1-Flash
The parser dropped apiKey, so a key typed in the add or edit form was written to the settings and never seen again: every request went out without it and a key-protected server answered 401. A paired server keeps its key too, which is what makes --connect together with --api-key possible. Assisted-by: DeepSeek-V4.1-Flash
The UI drives llama.cpp servers only: it loads and unloads models, reads the router feed and its chat template. /props is asked once the URL settles, and a definite answer that it is something else blocks saving the server. An endpoint that does not answer, or guards /props with a key, stays unknown, so a working server behind a reverse proxy is never locked out. Assisted-by: DeepSeek-V4.1-Flash
The chat banner is the surface left when nothing can serve a request, and it stopped at a retry that cannot succeed. It now carries the door to the servers dialog, where a server is paired. Assisted-by: DeepSeek-V4.1-Flash
A load on a server behind a tunnel was a spinner with no progress, since the feed was pinned to the local origin. The active server's own feed is read at its origin while it is reachable, and an operation that waits on a load keeps its polling as the fallback, so a feed that stops or never starts cannot hang a row. Assisted-by: DeepSeek-V4.1-Flash
The sidebar entry test renders the footer items through onAction, the way the stack's SidebarNavigationActions takes them, and the files the replay touched get their import order, class member order and formatting back. Assisted-by: Claude (Opus 5.5 Medium)
On a phone the sidebar and the chat's no-server banner open #/servers, with the menu key and a title like the Settings and MCP pages. Assisted-by: Claude (Opus 5.5 Medium)
The CORS proxy, its auth header, streamed tool calls and the API key check followed the active server, so with a remote server active, MCP traffic went through someone else's machine and streamed tools ran where the tool list did not come from. Pin them to the local server, like the tool list. Hidden models: read ids saved before server-qualified ids as the local server's, and check the active server's raw ids against qualified ones, so hidden models stay hidden and out of the automatic pick. A conversation's model now stays on the active server when that server has it too. Assisted-by: Claude (Opus 5.5 Medium)
Pairing over WebRTC still sends the pass code to whichever peer answers on the public tracker, so it ships off by default as an Experimental setting that names that gap and the planned mutual challenge-response. While it is off the servers form offers URLs only and no tunnel opens; turning it off closes the ones already open. Assisted-by: Claude (Opus 5.5 Medium)
Transcription, speech and the realtime socket go to the server in use, through apiUrl, and the models that hear or speak come from its list. A websocket is not tried for a remote server, which a browser socket can not reach with its tunnel or key. Assisted-by: Claude (Opus 5.5 Medium)
With no local llama-server, or with it switched off, the tools store drops llama-server's tool list. The MCP servers it runs (stdio), which the MCP extension lists from that same listing, now go with it. Assisted-by: Claude (Opus 5.5 Medium)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Chat can run against more than the server the app was loaded from: other llama.cpp servers are added by URL, or paired by an access code or its QR code and reached through a tunnel, and every server's models show up in the selector and the Models page. On a phone, Servers is a page of its own, like Settings and MCP. Pairing is experimental and off by default, behind a setting.
Additional information
Part 8 of 13 of a work-in-progress stack on the fork, based on
allozaur/feat/projects. Its diff is only this layer, and it is written on the design system's tokens and components (#22, lower in the stack).Since 13 November the stack has MCP as an extension (#61) below this layer. Two things here follow from that: Servers sits after the contributed MCP entry in the sidebar footer, so the order stays Models, MCP, Servers, Settings; and the page of an extension starts the extensions in its own load, because this layer moves the rest of the startup after the load.
The same work as ggml-org#29586, rebased onto this stack from its newer local branch (20 October: the rename to servers, the tunnel and pairing). The upstream PR and its branch are unchanged.
Pairing still sends the access code's pass code to whichever peer answers on the public tracker, so it sits behind "Enable remote server pairing" (Settings, General), marked Experimental and off by default, with a note naming that gap. While it is off, servers are added by URL only and no tunnel opens. A mutual challenge-response, where neither side sends the pass code, is the planned fix.
What changed in the rebase
The work was written before master squash-merged Discover (ggml-org#29584) and the Models Manager as a route (ggml-org#30988), and before this stack, so each commit was replayed with only its own change applied onto the current code.
hydrateStores(migrations, settings, servers) now runs before the stack'sinitStoresandloadAppRoute.chat-wire.tsreportsfinish_reason.Commits (31)
Testing
uqr@0.1.3, which the QR code commit adds).#/servers, with the menu key and a title like the Settings and MCP pages, and the sidebar marks it; on desktop it stays a dialog with no change of route. DESIGN.md lists it with the other phone pages.Requirements
Claude (Opus 5.5 Medium)