On recent Python releases, a </ followed by a letter with no closing > drops the rest of the document from the output. It doesn't matter whether it sits inside a code span. No error or warning is raised.
import markdown
markdown.markdown("Run `cmd 2</dev/null` here.\n\nThis paragraph disappears.")Python 3.13.5 and earlier:
<p>Run <code>cmd 2</dev/null</code> here.</p>
<p>This paragraph disappears.</p>
Python 3.13.6 and later:
Everything after </dev/null is gone, including the paragraphs that follow. We hit this with shell redirections such as 2</dev/null and cat </etc/hosts in documentation. The page built cleanly and silently lost its second half.
Versions
With markdown 3.11, the snippet loses text on Python 3.12.14, 3.13.6 through 3.13.9, 3.13.15 and 3.14.8, and works on 3.13.4 and 3.13.5. It also works on Python 3.12.0, 3.12.8, 3.13.0 and 3.13.3 with markdown 3.8. On Python 3.14.8, markdown 3.7, 3.9, 3.10 and the current master all lose the text. The trigger is the Python version, not the markdown version.
Cause
The change comes from CPython gh-135462 (backported to 3.13 as python/cpython@4455cbabf). Its commit message says that end-of-file errors are now handled as the HTML5 spec describes: "comments and declarations are automatically closed, tags are ignored". An end tag that is still open when the input ends is discarded along with the rest of the input. Feeding x 2</dev/null y\n\nnext to a bare html.parser.HTMLParser (with convert_charrefs=False) and calling close() gives these handle_data calls:
- parser.py just before that commit:
'x 2', '<', '/dev/null y\n\nnext'
- parser.py at that commit:
'x 2'
markdown/htmlparser.py overrides parse_endtag for </ followed by a non-letter, and close() hands any leftover rawdata to handle_data. Since gh-135462, the base class's close() consumes the unfinished end tag, so nothing is left over.
Whether the text survives depends on what comes after the tag name, which follows the new tag-end matching in html.parser. The tail is lost in these cases:
`</dev` z > q`</ab c` z`</dev/` > q
The tail survives in these:
`</dev/null` z > q`</ab c>``</a>`
#1578 also involves </ in code spans, but there </ is followed by a non-letter and the result is a hang. #1547 covers the same set of CPython changes, from their effect on the test suite.
On recent Python releases, a
</followed by a letter with no closing>drops the rest of the document from the output. It doesn't matter whether it sits inside a code span. No error or warning is raised.Python 3.13.5 and earlier:
Python 3.13.6 and later:
Everything after
</dev/nullis gone, including the paragraphs that follow. We hit this with shell redirections such as2</dev/nullandcat </etc/hostsin documentation. The page built cleanly and silently lost its second half.Versions
With markdown 3.11, the snippet loses text on Python 3.12.14, 3.13.6 through 3.13.9, 3.13.15 and 3.14.8, and works on 3.13.4 and 3.13.5. It also works on Python 3.12.0, 3.12.8, 3.13.0 and 3.13.3 with markdown 3.8. On Python 3.14.8, markdown 3.7, 3.9, 3.10 and the current
masterall lose the text. The trigger is the Python version, not the markdown version.Cause
The change comes from CPython gh-135462 (backported to 3.13 as python/cpython@4455cbabf). Its commit message says that end-of-file errors are now handled as the HTML5 spec describes: "comments and declarations are automatically closed, tags are ignored". An end tag that is still open when the input ends is discarded along with the rest of the input. Feeding
x 2</dev/null y\n\nnextto a barehtml.parser.HTMLParser(withconvert_charrefs=False) and callingclose()gives thesehandle_datacalls:'x 2','<','/dev/null y\n\nnext''x 2'markdown/htmlparser.pyoverridesparse_endtagfor</followed by a non-letter, andclose()hands any leftoverrawdatatohandle_data. Since gh-135462, the base class'sclose()consumes the unfinished end tag, so nothing is left over.Whether the text survives depends on what comes after the tag name, which follows the new tag-end matching in
html.parser. The tail is lost in these cases:`</dev` z > q`</ab c` z`</dev/` > qThe tail survives in these:
`</dev/null` z > q`</ab c>``</a>`#1578 also involves
</in code spans, but there</is followed by a non-letter and the result is a hang. #1547 covers the same set of CPython changes, from their effect on the test suite.