Skip to content

UI: Add tool calling Web Search (SearXNG) and fetch_url - #506

Open
tapstoop wants to merge 24 commits into
theroyallab:mainfrom
tapstoop:feat/ui-tool-calling
Open

tapstoop wants to merge 24 commits into
theroyallab:mainfrom
tapstoop:feat/ui-tool-calling

Conversation

@tapstoop

@tapstoop tapstoop commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Is your pull request related to a problem? Please describe.

The built-in chat UI gives the model no tools. This is now allowing to use web search and direct url fetch straight out of the UI, as well as adding tools advanced menu, and toggles for each tool.

This is using the already existing API: /v1/chat/completions accepts tools and returns tool_calls.

Why should this feature be added?

This is to improve TabbyAPI native UI, and allow models to be more powerful inside of it.
This change stays opt-in. Nothing changes until tools_config.yml exists and enables a tool. Users who never touch the file see one new gear button that does nothing until the server enables the tools.

The server keeps the safety in its own hands:

  • The SearXNG token stays server-side. It never reaches a client, a log line, or the UI.
  • fetch_url refuses private, loopback and link-local addresses. The same check applies to redirects. allow_private_urls exists for trusted networks and defaults to false.
  • The server stops each download at 1 MiB, even when the origin sends no length header. The server refuses non-text content.
  • Hard ceilings cap every size option. A client can lower a limit but cannot raise it past the ceiling. A file with errors keeps the previous options and logs a redacted error.
  • The loop stops after 8 tool rounds. The UI cuts each tool result at 2000 chars.

Examples

Enable the tools:

cp tools_config_sample.yml tools_config.yml
# set searxng.url and searxng.enabled: true, then save. The watcher applies
# the change without a restart.

Use them in the UI:

  1. Click the gear button next to the composer.
  2. Turn on Web search.
  3. Ask a question that needs fresh information, for example "What changed in the latest Python 3.13 patch release?"
  4. The model calls web_search. The UI shows an approval card with the arguments.
  5. Click Approve to run the tool. Click Deny to return an error to the model.
  6. The tool result goes back to the model as a tool message. The model continues with the result.

If Auto-approve tool calls is on, the UI runs tools without approval. Each message shows its tool calls in a collapsible "Tools (N)" box. Calculator and Date and time run in the browser, so they need no server config.

Call the endpoints directly:

curl -s http://127.0.0.1:5000/v1/tools/web_search \
  -H "Authorization: Bearer <admin key>" -H "Content-Type: application/json" \
  -d '{"query": "python 3.13 release notes", "max_results": 3}'

The three endpoints use the same API key check as every other endpoint. GET /v1/tools/status reports what the server has enabled. It never includes the SearXNG URL or token.

Additional context

What the change contains:

  • common/tools_config.py loads tools_config.yml and watches it for changes.
  • endpoints/OAI/utils/web_tools.py holds the SearXNG search and the page fetch with their SSRF guards.
  • endpoints/OAI/router.py adds POST /v1/tools/web_search, POST /v1/tools/fetch_url and GET /v1/tools/status.
  • endpoints/ui/static/* adds the tool switches, the approval cards, the tool loop and the "Tools (N)" boxes.
  • docs/10.-Tool-Calling.md explains setup, the API reference, the hard limits, the UI loop and troubleshooting.
  • tools_config_sample.yml shows every option with its default.

tests/test_tools_config.py and tests/test_tools_web.py add 50 tests. They need no server, no model, and no GPU:

$ python -m pytest tests/test_tools_config.py tests/test_tools_web.py -q
50 passed in 1.49s
$ ruff check .
All checks passed!

The rebase and the tests ran against current main (9979d01).

Two judgement calls worth a look in review:

  • Tools live in a new tools_config.yml, not in config.yml. The new file reloads while the server runs, and config.yml stays untouched.
  • Client requests can lower a limit but never raise it past the ceiling. One bad client cannot make the server fetch 100 pages of 1 MiB each.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant