Repository navigation
fix(web): Harden web fetch - #283
marcelklehr wants to merge 1 commit into
Conversation
WSHAPER
left a comment
There was a problem hiding this comment.
Disclaimer: Claude Sonnet 5.5 Review
The approach is good (urlparse instead of startswith, exact host/port matching, per-hop redirect validation, streamed reads capped at 100 kB), but a few things need fixing or confirming first.
Should fix
-
Malformed URLs will raise instead of returning an error.
urlparse(url).pathraisesValueErroron something likehttp://[::1, andparsed.portraises onhost:abcorhost:99999. The URL comes from the model, so treat it as untrusted. Today this propagates out ofis_an_internal_file_linkand_split_authority_path, soweb_fetchcrashes instead of returning a clean message. Wrap both in try/except and returnFalseor an error string. -
is_an_internal_file_linkwent from sync to async. Any caller that still calls it withoutawaitgets a coroutine, which is always truthy, so every URL would count as an internal link and fail silently. Please grep for other callers (other tools, tests) to confirm they're all updated. -
lib/url_guard.pyisn't in this diff.web.pyimportsBlockedUrlError,PublicAddressResolverandassert_public_urlfrom it. I couldn't review the core of the SSRF logic. I'd also verify thatniquests.AsyncSession(resolver=...)accepts your resolver class as written, since resolver interfaces are easy to get subtly wrong.
Worth a look
-
Scheme isn't checked before the Nextcloud-host bypass.
is_known_nc_hostis scheme-agnostic, softp://cloud.example.com/...skipsassert_public_url, which is where the http/https check lives. niquests will probably reject it anyway and you catchRequestException, but a cheapscheme in ('http', 'https')check at the top ofvalidate_hopis better than relying on that. -
Redirect loop edge case. On the last iteration a redirect closes
resand validatesnext_url, then falls into thefor/else, which callsres.close()a second time. It's probably harmless, but it's wasted work and a double close. -
Proxies. With an env proxy, the proxy resolves the target itself, so the connect-time rebinding protection doesn't apply there. Your comment says as much; just make sure that's an accepted trade-off.
NO_PROXYis also ignored by thegetproxies()filter.
Nits
- The docstring rewording in
files.py("agree with the pattern", "with the internal link to that file") is unrelated churn, and "must match" was clearer. This text is what the LLM reads, so I'd revert it. - The two
todo:comments (multimodal, PR #211) don't belong in this change. - There's no test coverage in the diff. The cases worth testing are
cloud.example.com.evil.net, a/f/123in the query string, a sub-path install, redirect to a private IP, and a server that ignoresRange.
This is also several changes in one: URL matching, SSRF guard, streaming, and error-format changes (HTTP errors now return a message instead of raw text). Splitting it up would help review. If it stays as one commit, the message should explain why: the old startswith check was bypassable via host-prefix tricks, and web_fetch could be redirected to internal targets.
Assisted-by: ClaudeCode:claude-opus-5.5 Signed-off-by: Marcel Klehr <mklehr@gmx.net>
6388590 to
53233df
Compare
|
"crashes" is overstated. graph.py:82 (handle_tool_error) catches tool exceptions and passes them back to the model as an error message |
Valid concern, nothing to fix. The only caller is web.py:39, and it awaits. |
🤖 AI (if applicable)