Description
HtmlConverter converts only the <body> element when present, and the whole document otherwise. Since BeautifulSoup's html.parser does not synthesize or relocate elements the way a browser does:
- a valid HTML5 document without
<body> tags leaks its <title> text into the markdown body
- any content placed before/after
<body> is silently dropped
Reproduction
from io import BytesIO
from markitdown import MarkItDown, StreamInfo
html = b'<html><head><title>Page Title</title></head><p>hello</p></html>'
MarkItDown().convert_stream(BytesIO(html), stream_info=StreamInfo(extension='.html')).markdown
# 'Page Title\n\nhello' <-- title leaks into the body
html2 = b'before<body><p>in body</p></body>after'
# 'before' and 'after' are dropped entirely
Expected behavior
Head metadata stays out of the markdown (except when used as the doc title), and stray content outside <body> is preserved, matching what a browser considers rendered content.
Environment
markitdown main (cc0ca9e), Python 3.12
Description
HtmlConverterconverts only the<body>element when present, and the whole document otherwise. Since BeautifulSoup'shtml.parserdoes not synthesize or relocate elements the way a browser does:<body>tags leaks its<title>text into the markdown body<body>is silently droppedReproduction
Expected behavior
Head metadata stays out of the markdown (except when used as the doc title), and stray content outside
<body>is preserved, matching what a browser considers rendered content.Environment
markitdown main (
cc0ca9e), Python 3.12