Repository navigation
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Is your pull request related to a problem? Please describe.
The built-in chat UI gives the model no tools. This is now allowing to use web search and direct url fetch straight out of the UI, as well as adding tools advanced menu, and toggles for each tool.
This is using the already existing API:
/v1/chat/completionsacceptstoolsand returnstool_calls.Why should this feature be added?
This is to improve TabbyAPI native UI, and allow models to be more powerful inside of it.
This change stays opt-in. Nothing changes until
tools_config.ymlexists and enables a tool. Users who never touch the file see one new gear button that does nothing until the server enables the tools.The server keeps the safety in its own hands:
fetch_urlrefuses private, loopback and link-local addresses. The same check applies to redirects.allow_private_urlsexists for trusted networks and defaults to false.Examples
Enable the tools:
Use them in the UI:
web_search. The UI shows an approval card with the arguments.toolmessage. The model continues with the result.If Auto-approve tool calls is on, the UI runs tools without approval. Each message shows its tool calls in a collapsible "Tools (N)" box. Calculator and Date and time run in the browser, so they need no server config.
Call the endpoints directly:
The three endpoints use the same API key check as every other endpoint.
GET /v1/tools/statusreports what the server has enabled. It never includes the SearXNG URL or token.Additional context
What the change contains:
common/tools_config.pyloadstools_config.ymland watches it for changes.endpoints/OAI/utils/web_tools.pyholds the SearXNG search and the page fetch with their SSRF guards.endpoints/OAI/router.pyaddsPOST /v1/tools/web_search,POST /v1/tools/fetch_urlandGET /v1/tools/status.endpoints/ui/static/*adds the tool switches, the approval cards, the tool loop and the "Tools (N)" boxes.docs/10.-Tool-Calling.mdexplains setup, the API reference, the hard limits, the UI loop and troubleshooting.tools_config_sample.ymlshows every option with its default.tests/test_tools_config.pyandtests/test_tools_web.pyadd 50 tests. They need no server, no model, and no GPU:The rebase and the tests ran against current
main(9979d01).Two judgement calls worth a look in review:
tools_config.yml, not inconfig.yml. The new file reloads while the server runs, andconfig.ymlstays untouched.