Repository navigation
Merged
Conversation
Every scanner that finds the `)` closing a command substitution -- the
lexer's tokenize_subshell, the word reparser, the expander's
find_cmd_sub_end, span_skip -- drives the shared casescan automaton, and
it knew quotes and case patterns but nothing else a script can hold.
bash parses $( ) as a script, so three things it accepts broke here:
x=$(echo a # it's a comment unexpected EOF looking for `''
)
x=$(# ( unexpected EOF looking for `)'
echo ok)
x=$(cat <<EOF the body's `'` opened a quote,
it's and a body line `a ) b` closed
EOF the substitution early
)
casescan now owns both spans. A `#` that begins a word (after an
unescaped blank, a newline, `;` `&` `|`, or the `$(` itself) runs to the
newline; after any other `(` it does not, so zsh's `(#i)` and `${(#)x}`
keep their meaning, and `$(a)#b` stays one word. A `<<word` owes a body
that the next newline skips through the delimiter line, compared with
the same quote removal hd_delim applies. `<<<` and `<<=` owe nothing,
and neither does `<<` inside (( )) or $(( )) -- a `(` straight after a
`(` opens arithmetic, where `<<` is a shift; without that, a multi-line
$( ) holding `echo $((1<<2))` would have swallowed its own tail.
Keeping all of it in the one automaton is what keeps the lexer's span
and the expander's span the same span.
Refs #139
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
`nvm install 22` printed "syntax error near unexpected token `newline'"
twice and found no version. nvm quotes a multi-line awk program right
before a heredoc:
VERSIONS="$( { command awk '{
...
}' \
| nvm_grep -w "$PATTERN" \
| $SORT_COMMAND; } << EOF
$VERSION_LIST
EOF
)"
String execution (the $( ) body, eval, source) pulls heredoc bodies out
before parsing, and found the operators by lexing ONE PHYSICAL LINE at a
time. A line that closes a multi-line quote -- `b' | cat; } << EOF` --
lexes on its own as an OPEN quote, so the `<<` was never seen: its body
ran as commands, and the fallback that went looking for it read past the
end of the script buffer (an ASan heap-buffer-overflow; the release
build reads whatever follows the buffer).
A heredoc body starts after the newline TOKEN that ends its command line,
and a newline inside a quote, `...`, $( ) or ${ } is word text, not that
token. collect_specs and heredoc_incomplete now lex a logical line: the
physical line grows while the lexer reports a lexeme still open, and the
bodies of its operators start after its last physical line, which is
where bash reads them (`cat <<A; echo "x<NL>y"` reads A's body after the
`y"` line). A joined line that lacks the byte the lexer is waiting for
cannot close the lexeme, so it is joined without re-lexing: a long quoted
awk or python program costs one lex per line holding a quote, not one
per line.
Refs #139
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
A string run by eval or source whose heredoc has no body in its own text
was handed the caller's heredoc stream -- NULL at top level, which means
"read it live", which for a script means rl.buff, the SCRIPT's buffer:
eval 'cat <<EOF' bash: warning, then runs `hello` and `EOF`
hello as commands, then prints `after`
EOF hellish: printed `hello` -- the eval ate the
echo after script's next lines
That is also the fallback nvm's misread heredoc fell into (#139): the
body scan there stops at a NUL, rl.buff is a length-counted vector with
none, and a delimiter that never came walked off the end of it.
A string whose heredocs found nothing in its own text now gets an empty
stream: the heredoc ends at once with bash's "delimited by end-of-file"
warning, and the lines after the eval or the `.` run as commands. The
one exception is a $( ) body, flagged by the forked child for the one
string it runs: bash reads `x=$(cat <<E)` on into the lines after it, and
that path is kept as it was. And capture_heredoc_from_buff writes the
missing terminator past rl.buff's length before scanning, so that scan
is bounded whoever reaches it.
Refs #139
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
f() { cat <<EOF; }
body
EOF
f
segfaulted -- the published 3.1.3 release exits 139 on it, the ASan
build trips ft_assert in heredoc_redir. A heredoc in a function or loop
body is re-read at every call, so its raw body must be captured onto the
AST node; try_defer_heredoc only did that when the body had been
pre-extracted into hd_src. A one-line definition is complete before its
body is read, so nothing pre-extracted it, and the fallback wrote the
body into THIS command's redirect slot. The slot died with the command,
and the first call of f indexed it.
capture_heredoc_live reads the body from the live input instead -- the
same reader write_heredoc uses, so it works from a script, from -c, and
at an interactive prompt alike -- and keeps it on the node, always ending
on a delimiter line so each call does not re-warn about end-of-file.
Found while tracing #139; filed against the same symptom as #138.
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
The second half of #139. nvm is SOURCED, and source runs its text in chunks (#105). A chunk that parses incomplete grows to the next hazard boundary, and while growing only `<<` is a hazard -- so the retreat to a line start could land right after a trailing `\`: { command awk '{ ... }' \ <- chunk ended here | nvm_grep -w "$PATTERN" \ | $SORT_COMMAND; } << EOF The tokenizer strips the join, so the chunk `{ ... }' \` has no NEWLINE token after it, and an open group ending that way parses as an ERROR, not as "needs more": the chunk was replayed instead of grown, and the replay reported the error. The comment above str_span said the retreat could never land on a join; that held only for a chunk's first span. The span is now always stretched to the end of its logical line. Stretching a growth span is always safe (it re-parses a bigger chunk); a `\` that only looks like a join makes the span longer, nothing else. The span logic moves to exec_string5.c, unchanged otherwise. Refs #139 Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
tests/scripts/52_heredoc_logical_line.sh runs under hellish and bash
--posix in the script corpus (stdout, status, and a clean sanitizer
report): the nvm shape distilled, multi-line quotes before a `<<` in
$( ), backticks, pipes and groups, comments and heredoc bodies inside
$( ), shifts that are not heredocs, one-line function and loop bodies
whose heredoc follows the line, a sourced function whose continuations
must survive the chunker, and an eval that must not read the script's
next lines. Against the 3.1.3 code it exits 2 with 62 lines of diff.
tests/function_heredoc_tty_test.py is the interactive half: the body of
`f() { cat <<EOF; }` arrives through readline's `heredoc>` prompt, a
different reader than a script's buffer, and the old shell died on the
first call. It waits on output, not on the clock, and runs in well
under a second.
Refs #139
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
The ubuntu 22.04 rung of the platform matrix went red on PR #141 before a line was compiled: an Ubuntu mirror mid-sync served an index that disagreed with its files -- File has unexpected size (1622802 != 1622804). Mirror sync in progress? E: Failed to fetch .../libexpat1_2.4.7-1ubuntu0.9_amd64.deb 404 Acquire::Retries does not cover either (nothing dropped; the index is stale for a minute), and the cure is to fetch it again a little later. So the apt branch retries update AND install together, four attempts, 15s/30s/45s apart, beside the zypper retry this file already carries for the same reason. A mirror that keeps failing still fails the build. Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
fix(heredoc): nvm's heredocs, $( ) comments and bodies, and a function-body heredoc segfault
f() { local v=$1; printf '[%s]' "$v"; }; f 'x y'
bash: [x y] hellish: [x]
bash (and POSIX for export/readonly) expands a NAME=value argument of a
declaration utility as one field: no field splitting, no pathname
expansion. hellish did that only for a command word of literally
`export`; local, declare, typeset and readonly split theirs, so
`local v=$1` kept the first word and `local v=$x` with x='a *' was
cut at the blank. The omz sudo widget's
local old=$1 new=$2 space=${2:+ }
then turned `vim f` into `sudo/f` on ESC ESC instead of `sudo -e f`
(#137): `new` was `sudo`, `-e` a stray option, and `space` empty.
decl_word_kind recognises the five utilities, and a bare `command` in
front of one, which bash also honours; `builtin local` and `command -p
declare` still split, as they do in bash. The flag the expander carries
is renamed from `export` to `decl`, which is what it now means, and the
duplicate is_export declaration in expander_private.h goes with it.
Refs #137
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
setopt zsh; alias ll='ls -l'; echo "[${aliases[ll]}]"
zsh: [ls -l] hellish: []
The dialect's `aliases` WRITE side already reached the alias table
(#114, `aliases[k]=v` defines an alias); a READ fell through to the
plain, empty variable. So `${aliases[ll]}`, `${(k)aliases}` and
`${#aliases}` all saw nothing, and oh-my-zsh's sudo widget -- on ESC ESC,
installed and on by default -- could not learn what an alias runs:
local realcmd="${${(Az)aliases[$cmd]}[1]:-$cmd}"
In the dialect the parameter now IS the table: expand_special_dyn answers
`aliases` with the table built as an associative value, so every reader
that already understands one -- a subscript, (k), (v), ${#...}, the flags
-- works unchanged. It is returned borrowed, like any variable's value,
from one of two state-owned slots: the previous read's value stays valid,
so two reads alive at once cannot see each other's free. Both slots are
freed with the session. In the bash dialect `aliases` is an ordinary
variable, as before.
Refs #137
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
Pressing ESC ESC printed, from oh-my-zsh's sudo widget (#137): hellish: ${${(Az)aliases[$cmd]}[1]:-$cmd}: bad substitution hellish: ${__hsh_nest_0[1]:-$cmd}: bad substitution Two shapes reached expand_param_format, the scalar engine, which has no subscript -- `name[sub]`, alone or before an operator, is expand_array_elem_op's form, one layer up at the token level: * a flag on a subscripted name: ${(U)e[2]}, ${(z)aliases[ll]}; * a subscript AND an operator after a nested flagged expansion: ${${(Az)x}[1]:-d}. The subscript alone already worked, through zsh_bare_nested. zsh_body_eval runs a ${...} body through the token-level expander, and both the flag operand (zf_inner) and the nested operator's scratch body now use it. And the nested-with-operator shape is caught at the token level by zsh_nested_op, beside zsh_bare_nested: only there is the quoting known, and it decides whether a flagged inner value is still an array -- so `[1]` picks the first WORD of `${(Az)...}` (`ls`), not its first character (`l`). The bind-and-apply core that both layers use is one function, zsh_nested_apply; zd_nested keeps the text path for the operands that reach it without a token. Graded against a real zsh 5.8 (tests/zsh_nested_op_test.py): every double-quoted form, and every single-word unquoted one, matches. Refs #137 Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
fc matched its options as whole words -- `-l` and `-lr` -- so every other spelling of a listing fell into EDIT mode and started $EDITOR on the history: `fc -ln -1`, `-nl`, `-lnr`, `-l -n`. oh-my-zsh's sudo widget asks `$(fc -ln -1)` for the previous command when ESC ESC hits an empty line (#137), which ran vim inside a command substitution. The listing itself differed from bash's too: * the running fc line counted as the most recent entry, so `-1` named fc itself (bash records a command before it runs, and fc looks past it); inside a zle widget the edited line is not recorded yet, so -1 is the previous command, as zsh code expects; * one operand listed the last 16 instead of that entry to the end; * `-n` after `-l` was read as an operand; * entries printed as `N<TAB>cmd`, not bash's `N<TAB> cmd` (`<TAB> cmd` with -n); * a string operand, `fc -l ech`, was not looked up. fc_parse (builtin_fc3.c) clusters options the POSIX way, stops at `--`, and reads `-` followed by a digit as the negative OPERAND it always is in fc. The listing resolves operands against the entries before this line, lists first-after-last backwards and lets -r flip that, and prints bash's layout. Checked case by case against an interactive bash on a pty; `fc -l 3 1` follows POSIX (3 2 1) where bash, in that one position, printed only entry 1. `-s` and `-e -` (re-run without an editor) are not implemented yet; they are now refused out loud instead of falling into edit mode like every other unrecognised spelling did. Refs #137 Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
Four gates, one per level the bug lived at:
* tests/assign_shaped_words (golden, vs bash --posix): 14 cases --
local/declare/typeset/readonly/declare -x/local -r keep an unquoted
$1 as one field, through a bare `command` too, while `builtin` and
`command -p` still split, as in bash;
* tests/zsh_nested_op_test.py (-c, no terminal): `aliases` reads, flags
on subscripts, subscript + operator after a nested expansion. Each
expectation is what zsh 5.8 printed; they are asserted so the check
runs without zsh, and re-verified against one when it is found;
* tests/fc_listing_tty_test.py: 18 listing forms, each expectation from
an interactive bash on a pty;
* tests/omz_sudo_widget_test.py: the real plugin, the real key. ESC ESC
on a plain command, on `sudo cmd`, on the editor, on `sudo -e`, on an
alias of the editor and on an empty line, each checked by RUNNING the
line with a fake sudo and vim that print their arguments. Cached like
plugin_corpus_test.py; skips, out loud, with no copy and no network.
Against develop 11 of the 14 golden cases fail (the other three are
the controls that must split, and did), 16 of 19 zsh cases, all 18 fc
forms, and 7 of the 9 widget checks; all pass here.
tests/helpers/ptyexpect.py is the pty driver the two terminal tests
share. It waits for the text it needs and moves on the moment it
arrives, instead of sleeping a fixed worst case after every key: 18 fresh
shells in 0.4s, six ESC ESC sessions in 0.2s. It lives in helpers/ so
pty_suite.sh, which runs every tests/*.py, does not take it for a test.
Refs #137
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
$ ./life (the file is there)
hellish: ./life: No such file or directory
Issue #133. Every way a path can fail to run read the same:
* handle_direct_path_error forced errno to ENOENT before printing, so
a file without +x, a directory, `file/x` (ENOTDIR) and a symlink
loop (ELOOP) all said "No such file or directory" -- and the last
two exited 127 where bash exits 126. It now reports stat's or
access's own errno, EISDIR for a directory, and derives the status
from it: 127 only for ENOENT.
* after execve failed, only EACCES was reported. A script whose `#!`
interpreter is missing, or a program whose loader is -- execve's
ENOENT on a file that exists, the likeliest meaning of "./life is
there but will not run" -- printed nothing and exited 127.
* a binary execve refused (ENOEXEC) went to the /bin/sh fallback,
which printed "./x: 1: <garbage>: not found", exited 127, and ran
the rest of the file when its first line held no NUL.
exec_diag.c follows bash's shell_execve: a directory, a file that is
not executable, E2BIG/ENOMEM, ENOENT ("cannot execute: required file
not found"), a `#!` script the kernel refused ("INTERP: bad
interpreter", the CR of a CRLF script spelled ^M), and otherwise the
errno's own text; 127 only for ENOENT. Before the /bin/sh fallback,
the first 80 bytes are checked as bash's check_binary_file does (an ELF
header, or a NUL on the first line -- the first two after "#!"), and a
binary is refused with "cannot execute binary file: Exec format error",
126. An empty file and a text script without "#!" still run.
map_errno_to_exit and the status macros it alone used go: they
encoded 127 for ENOTDIR and ENOMEM, both 126 in bash.
The hang in the report did not reproduce: the child exits at once on
every one of these paths, on a terminal with job control too.
Refs #133
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
… prompt tests/scripts/53_exec_diagnostics.sh runs every failure shape through the corpus (graded against bash --posix, ASan-scanned): a file without +x, a directory, a missing path, ENOTDIR, ELOOP, a missing `#!` interpreter, a CRLF shebang, a directory as the interpreter, a bogus ELF, a NUL on the first line, a NUL on the second after "#!" (runs), a script without "#!" (runs), an empty file (status 0), and a script found on PATH (named by its full path). Each prints its diagnostic and status on stdout, with the "script: line N:" prefix stripped. Against develop 18 of its 31 lines differ; all match here. Its fixtures live in a mktemp directory, since verify_alloc.sh runs the corpus from tests/scripts itself. tests/exec_error_tty_test.py types six of them at an interactive, job-control prompt -- where the report came from -- and requires bash's message, the prompt back, and bash's status from `echo $?`. A hang is a timeout, a named failure. 66 ms. It uses the ptyexpect helper introduced for #137 (the same file, byte for byte). Refs #133 Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
fix(#137): ESC ESC (omz sudo widget) — `aliases`, nested subscripts, declaration words, fc listing
eval $'. /dev/null; false\n# note'; echo $?
bash: 1 hellish: 0
eval and source run their text in chunks cut at hazard lines (`.`,
alias, shopt, heredocs -- issue #105). A chunk that holds no statement
at all -- the blank lines or comments after the last hazard -- still
"ran", with status 0, and that became the status of the whole string.
So a sourced rc file ending in a comment after a `. other` line
reported success whatever its last command did.
run_stmt_list now answers -1 for a chunk that ran nothing, and
exec_chunks keeps the last real status. An eval of only whitespace,
newlines or a comment is still 0, as in bash.
tests/chunk_status (golden, vs bash --posix): 13 cases. The five that
end on a comment or blank tail after a hazard fail on develop; the rest
are the controls (whitespace-only eval, a heredoc as the last command,
a sourced file) that must keep passing.
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
printf 'true\n./nope\n' > t.sh; hellish t.sh
bash: t.sh: line 2: ./nope: No such file or directory
hellish: t.sh: line 3: ./nope: No such file or directory
ctx -- the "script: line N:" every runtime error starts with -- was set
only by the READER, from the line it had read up to. Scripts arrive in
batches, a small one all at once, so by the time a command ran the
reader stood past the batch: every error in it named the line after the
last one. Inside a sourced file the error named the script that did the
sourcing, at a line of its own, and $LINENO there was the `.` line's.
Each simple command now moves ctx to where its first token sits, as it
starts (exec_where.c, from note_cmd_lineno):
* in a sourced file's chunk: "FILE: line N" (with the shell's name in
front at a prompt, as bash does) -- err_pos is the chunk, err_line
its first line, advanced chunk by chunk by exec_chunks;
* in the top-level text: "script: line N";
* anywhere else -- a function body, an eval string -- ctx stays at the
call site, which is what $LINENO reports there too.
`source` saves and restores the whole frame, ctx string included
(where_push/where_pop), so the next error after it names the caller's
line. $LINENO inside a sourced file now reads that file's line, like
bash's. A syntax error inside an eval in a sourced file names the file
and the eval's line (bash: "F: eval: line N"), where it said
"hellish: line 1"; source_diag_test.py pinned the old wording and now
pins the file and the line.
Line numbers are resolved from the last one, forward or back (a file
run top to bottom is scanned once, a loop rescans only its body), and a
new line in the same file rewrites the digits at the end of ctx in
place: a 200k-iteration loop costs 274 ms against 269 before, and a
6000-line source is no slower (40 ms against 45 -- exec_chunks no
longer measures the string twice).
Two miscounts the new lines exposed, both wrong on develop too:
* a script's same-line heredoc body is taken straight from the input
buffer, and the cursor skipped it without counting its lines: every
line after a top-level heredoc was numbered short by its body;
* split_heredocs dropped body lines from the text a cycle or a sourced
chunk is lexed from, so lines after one were short there as well --
the top-level path patched this with an estimate that was wrong for
more than one heredoc. Each body line now leaves an empty line
behind, so offsets map to source lines exactly and the estimate goes
(an empty line is a separator wherever the operator line's own
newline already is; the parse does not change).
tests/scripts/54_error_line_numbers.sh folds each error into stdout for
the corpus to diff against bash: the batch, a sourced file and a loop in
it, a nested source, the lines right after each source returns, an eval
in a sourced file, a heredoc before the error, a loop at top level, and
$LINENO at each. 22 of its 24 lines differ on develop. Not covered, and
documented: a function body still reports its call site, as $LINENO
does (its tokens are copies with no position).
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
./script (#!/nonexistent/interp)
bash 5.2: ./script: cannot execute: required file not found 127
bash 5.3: ./script: /nonexistent/interp: bad interpreter:
No such file or directory 126
The previous commit followed bash 5.2's shell_execve, which reports an
ENOENT from execve as "required file not found" before it looks for a
`#!` line. The suite's pinned oracle is 5.3.9, which reads the `#!`
line first: a script whose interpreter is missing -- a CRLF shebang
too, `/bin/sh^M` -- is a bad interpreter, status 126. Only a file with
no `#!` whose loader is missing (a binary for another system) is still
"required file not found", 127.
CI caught it (Tests: 53_exec_diagnostics.sh, lines 11-14, graded
against 5.3.9); the local oracle was 5.2 because ftp.gnu.org is not
reachable from where this was written. Checked now against a 5.3.9
built from the upstream tarball and patches 001-009: the corpus
script, and ./life (ELF, missing loader: 127), ./script (126),
./badinterp (126, no "line N:"), ./macho (126) by hand.
tests/exec_error_tty_test.py pins the 5.3 wording for ./nointerp.
Refs #133
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
arena-stress (512-byte parse chunks + ASan), gen dense seed=1:
heap-buffer-overflow, READ in strlen <- xg_alt_group <- tokenizer
<- update_prompt (input.c)
The previous commit made split_heredocs leave an empty line for each
heredoc body line it removes, pushed with vec_push_char. That helper
writes the terminating NUL only while len < cap: when the '\n' landed
on the buffer's last byte -- a file ending in a heredoc body, at the
right size -- the stripped text had no NUL, and the tokenizer, which
takes it as a C string, read past the end. vec_push_nstr reserves
room for the NUL every time.
The regular ASan build had slack after every buffer, so only the
512-byte-chunk stress build, over its generated rc corpus, lands on
the exact boundary; that job (Hard corpus + allocator parity) is the
regression gate, and passes again: `sh tests/alloc_stress.sh` is all
clean.
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
bindkey '^[[A' history-beginning-search-backward # Up: prefix search
bindkey '^[[B' history-beginning-search-forward
is how a zsh config asks for "Up searches the history for what I typed"
(#136). It did nothing, twice over:
* the sequence was handed to readline as written, and `^[[A` is zsh's
caret notation -- readline bound the four characters ^ [ [ A, so the
arrow never reached it. So did every `^R`, `^E`, `^[b` a zsh config
or plugin wrote that way. zle_keyseq translates what only zsh has
(^X, ^[, ^?, \E) and passes the backslash escapes the two share;
* a built-in widget name bound the key to the shell-widget dispatcher,
which found no widget by that name and returned -- the key went
dead. zle_bind_install now binds such a key to readline's own
command (zle_rl4.c maps the names that differ: history-beginning-
search-* and oh-my-zsh's up/down-line-or-beginning-search to
history-search-*, history-substring-search-up/down, up/down-history,
expand-or-complete ...; the rest are shared). Directly, not through
the dispatcher: readline's history searches keep their place by
checking the previous command was themselves, so a routed key would
restart the search from the line the last press found.
A widget a plugin registered under a built-in's name keeps the
dispatcher -- zsh-history-substring-search defines exactly those.
`zle history-beginning-search-backward` from inside a widget runs the
same command and refreshes BUFFER after it.
Refs #136
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
…om an rc
bind '"\e[A": history-search-backward' # Up: prefix search
bind 'set completion-ignore-case on'
bind -p | grep search
`hellish -c 'bind -p'` said "bind: command not found". `set -o emacs`
and `set -o vi` were the only line-editing control an rc had: no
history-prefix search on Up, no inputrc setting, no rebinding C-w
(#134 item 7; with the bindkey fix, the core of #136).
bind [-m keymap] [-lpsvPSVX] [-f file] [-q name] [-u name] [-r keyseq]
[keyseq:function | readline-line], as bash has it:
* bindings, -f, -u and -r are RECORDED (bind_lines.c) and replayed
before each read, and again from the start when the editing mode
changes -- an rc runs before readline exists, and the mode's keymap
replaces whatever was bound before it; the same ordering bindkey
already had to follow. -f becomes readline's own `$include`, its
path made absolute while it still means what the caller meant;
* the listings (-p -P -v -V -s -S -l) and -q bring readline up first
(rl_preinit), so `bind '...'; bind -p` in a script lists what it
just bound; their text is readline's own, word for word;
* checked when called, as bash checks them: the keymap (-m), -u's
function, -f's file; bash's messages, statuses (2 for a malformed
call, 1 for "not bound to any keys"), bash's order of actions, and
its warning that line editing is off in a script.
-x (a key that runs a shell command) is refused out loud for now rather
than bound to a key that would do nothing; -X lists no such bindings.
Refs #134, #136
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
…prompt tests/bind_builtin (golden, graded against bash --posix): 27 cases -- type, the empty and `--` calls, every error path and its status (bad option, missing argument, unreadable -f file, bad keymap, unknown -q / -u function), binding then -q / -p / -P, -u and -r taking a binding away, -m binding into vi-insert and querying there and not in emacs, `set` lines through -v / -V, -l, -lp, -s macros, -f FILE, -X. All 27 fail on develop (no bind) and match bash here. Readline's own defaults (what /etc/inputrc binds) and ESC-sequence spellings are never pinned: the system readline and bash's bundled one print `\e` differently. tests/history_prefix_search_test.py: four history lines that print four different words, then a typed prefix, arrows and Enter -- the word that prints is the line the keys recalled. Up and Up again walk back through the lines starting with the prefix, Down walks forward, for bindkey with zsh's names and caret notation, bind with readline's, oh-my-zsh's up-line-or-beginning-search, and `zle history-beginning-search-backward` from inside a user widget; with no binding, Up stays plain history (the control). 8 of the 9 cases fail on develop. 0.2 s, waiting on output. Refs #134, #136 Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
bind -x '"\C-t": __fzf_select' # what fzf's bash bindings do
The previous commit refused -x out loud. It is now bash's
bind_keyseq_to_unix_command and bash_execute_unix_command:
* parsing: the key sequence must be double-quoted, a colon or blanks
separate it from the command, which may be quoted either way; bash's
three messages (first non-whitespace character is not `"', no
closing quote, missing separator) and status 1;
* the key becomes a widget named "bind -x SEQ" -- no `zle -N` can
collide with a name holding a space -- marked bash-style, and runs
through the same dispatcher (zle_rl5.c): the visible line is
cleared, READLINE_LINE, READLINE_POINT and READLINE_MARK are set
(exported; point and mark counted in characters, not bytes, as bash
counts them), the command runs, a changed READLINE_LINE becomes the
line with the cursor at its end, a valid READLINE_POINT / _MARK then
places cursor and mark, the three names are unset, and the prompt
and line are drawn again;
* -X lists the bindings as bash does, the command re-escaped so the
line reads back; bind -p leaves them out, as bash does.
Refs #134
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
tests/bind_builtin: 10 more golden cases -- -X after -x, the three parse errors and their status, a blank separator with a quoted command, a colon with no blank, a command holding quotes and a backslash (re- escaped by -X), a rebinding replacing the command, bind -p leaving -x keys out, a bare -x. All match bash; all fail on develop. tests/bind_x_tty_test.py, at an interactive prompt (each expectation checked against an interactive bash 5.3.9 driven the same way): the command rewrites READLINE_LINE and moves READLINE_POINT to 0, so what is typed next lands in front; a command that prints mid-line leaves the line intact to be run; READLINE_POINT counts characters (an e-acute is one); the three variables are gone once it has run. 0.05 s. Refs #134 Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
fix(#133): say why a path could not run, with bash's words and status
… does
A script whose background job dies from a signal gets bash's unsolicited
report on stderr. bash prints the pid with %5ld (print_pipeline in
jobs.c), so a pid under 10000 comes padded:
t.sh: line 41: 4657 Killed sleep 9 (bash)
t.sh: line 41: 4657 Killed sleep 9 (hellish)
The report is now printed with %5d, the width `jobs -l` already uses.
17_bg_signal_report.sh didn't catch this. Its normalisation only matched
an unpadded pid, so on a machine whose pids had wrapped below 10000
(pid_max is 32768 here) bash's own line went through unnormalised and the
test failed on correct output. It now keeps the pid's width, not its
value: a field of five columns or more becomes PREFIX, and a narrower one
shows up as NARROW-PID. Correct code passes at any pid. The unfixed
shell fails whenever the pid is under 10000. Reproduced by cycling pids
below 3000 first: the unfixed binary differs from bash 5.3.9 on both
reports, and the fixed one matches.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
…iltin completion_posix_test.py reads the dispatch table and requires TAB at an empty prompt to offer exactly that set. Registering `bind` without adding it to g_builtins made it fail with "missing from the offer: ['bind']", which is the drift that test was written to catch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RAmeHfJNm7XjYbMNrQqvkG
fix(diagnostics): runtime errors and $LINENO name the right line — and the sourced file
Mount Univers42/claude-deal-with-the-devil as the .claude submodule (relative URL, like the other submodules) and add CLAUDE.md with the build/test/architecture notes future sessions need. .gitignore stopped ignoring .claude and *CLAUDE*, which hid both.
prompt_expand_p_test.py failed about one run in ten (develop CI, both heaps): at_p waited for ">\r\n", then sliced to the end of everything read so far. The read that delivered the marker can also carry the next prompt, so the value came back as "PROMPT >\nPROMPT". The value is now framed by split markers (B''EG / E''ND, so the echoed command line cannot match) and sliced up to where END was consumed. Every assertion is unchanged. Verified: 0/30 failures in-process, 0/15 with HELLISH_RL_FORK=1 (was 2/20). Mutants: a one-byte change to the expected prompt turns 10 checks red; at_p returning a constant turns 6 red.
fix(test): ${PS1@P} capture ends at its marker, not at the read's end
…136) `git`<Up> now recalls the last line that starts with "git", and further presses keep walking back through those lines only. On an empty line Up and Down are plain previous/next history, as before. This is readline's own history-search-backward/-forward, on by default in interactive shells. HELLISH_HISTORY_PREFIX turns it off when set to 0 or off (any case). It is read from the shell's variables at every prompt, so a value set in the rc file, inherited from the environment or typed at the prompt takes effect on the next line. Unsetting it turns prefix search back on. The user's own bindings always win: `bind`, `bindkey`, `bind -r` and a ~/.inputrc binding to anything other than the stock function are left alone. To tell an explicit `bind '"\e[A": previous-history'` from the default, the arrow slots are parked on a marker before the user's bind/bindkey lines replay. Whatever a replay leaves on a slot is the user's, and that slot is never touched again. Non-interactive shells are untouched: `bind -p` in -c mode (which also runs rl_preinit) still prints exactly what bash 5.3.9 prints. Known limitation: an inputrc line that binds an arrow explicitly to the stock previous/next-history looks the same as the default, so prefix search replaces it. Tests: history_prefix_search_test.py, 149 checks in-process and with HELLISH_RL_FORK=1: - the default; 0, off and OFF; inherited, rc and runtime toggles; readonly; PROMPT_COMMAND; - \eOA/\eOB; vi insert mode and a vi/emacs switch; - inputrc, bind, bindkey and bind -r winning; - no match, a full-entry match, a trailing space, the cursor mid-line, UTF-8, multi-line entries; - history -c and -d; the oldest and newest ends; ^C mid-walk; - --norc, a login shell, and bind -p against bash. rl_inproc_test's revert case pins plain history with HELLISH_HISTORY_PREFIX=0. The golden suite passes 5363/5363 before and after. Mutants, all killed: never install; always on; drop the stock-binding guard; no restore on off; swapped directions; ignore user slots; a case-sensitive "off"; ignore \eO; skip the vi keymap.
chore(claude): add the devil kit as .claude and a CLAUDE.md
feat(history): Up/Down search history by the text before the cursor (#136)
`make cl<TAB>` now completes `clean`. The targets are read from the first
regular file of GNUmakefile, makefile and Makefile in the current
directory (GNU make's order), parsed in the shell itself: make is never
run and nothing forks.
Completion goes through a small provider table, {command, function}, so
the next per-command completion is one row and one function rather than
another branch. `make` and `gmake` are its first rows, matched by
basename and behind sudo, env X=1, && and (.
Precedence keeps every existing behaviour:
- a user `complete` spec (under shopt -s progcomp) wins;
- no Makefile, a directory, an unreadable file, an option word, a word
with = or /, -C/-f on the line, or no matching target all fall back to
filename completion, exactly as before.
Parsing:
- recipe lines (TAB), comments, define/endef, assignments (:=, ::=, ?=,
+=, !=), directives, % and $ targets and .SPECIAL targets are skipped;
- `a b c:` and `a::` count;
- a backslash-newline is one space, as in GNU make;
- CRLF is undone;
- include/-include/sinclude are followed one level only, which also
keeps circular includes safe;
- files are capped at 4 MiB.
The parsed list is cached in t_shell, keyed on the main Makefile's dev,
ino, mtime and size, and freed in free_all_state. An included file
changing alone does not invalidate the cache (documented). Strings
handed to readline use rl_dup (libc); everything else uses xmalloc.
Tests: tests/complete_make_test.py, 24 groups, in-process and with
HELLISH_RL_FORK=1:
- file precedence; symlink, directory, unreadable and empty files;
- every skip rule; continuations; CRLF; includes, missing and circular;
- 5000 targets within a 2 s bound; duplicates;
- -C/-f; sudo, env, &&, gmake; user spec precedence and complete -r;
- cache invalidation on size, on mtime and on cd;
- filename fallback; a mid-word TAB; complete -p parity with bash.
Also green: completion_posix, completion_opts, make completion-test (the
SAFE=0 build), help-test, verify_alloc.sh, the golden suite (5363/5363),
and ASan with leak detection on the cache paths.
Mutants, all killed: provider never called; provider before progcomp;
:= accepted; %/$ filter off; recipe lines kept; includes ignored; cache
never invalidated; Makefile before GNUmakefile; continuations not
joined; no filename fallback.
feat(completion): make <TAB> completes the Makefile's targets (#140)
With HELLISH_AUTOSUGGEST=1, the most recent history line that starts with what is typed is shown dim after the cursor. It is never part of the line. Right, End and Ctrl-F accept it; with no suggestion those keys do what they always did. Off by default; read at every prompt, so it can be turned on and off at the prompt, from the rc or from the environment. HELLISH_SUGGEST_MAX (default 1000) bounds the history scan. POSTDISPLAY, with zsh's meaning, is exposed to zle widgets. It holds the text painted after the cursor, so a plugin can show its own hint and a widget can read the current suggestion. This is the core interface proposed on #140. How it works: - The suggestion is computed inside the redisplay function, right after rl_redisplay(), so it always matches the key just typed. While typing along the suggestion it is reused in O(1); otherwise hist_cmds is scanned newest first, up to HELLISH_SUGGEST_MAX entries. - It is painted from the cursor in one write, merged with RPROMPT's frame. RPROMPT is dropped for a frame when the ghost reaches it. When nothing changes on screen, nothing is written, so the per-key syscall budget holds. - It is wiped (at an absolute column) on Enter, ^C and zle accept-line, so it never stays in scrollback. A resize marks it stale. No ghost is shown when: - the cursor is not at the end of the line; - the prompt is PS2, the line has several rows, or it would wrap; - an incremental search is running, or vi is in command mode; - TERM is dumb. History entries with control bytes (C0, DEL, C1) or newlines are never suggested, so a history line cannot write escape sequences to the terminal. The painter's width check also stops at any character with no width, which covers a widget-set POSTDISPLAY. The key-slot logic from #136 is now a shared module (rl_slot*.c, replacing rl_hist_prefix.c/rl_hist_slot.c), used by both history-prefix and the accept keys. It keeps the same rule: a key the user bound themselves is never touched. Tests: tests/autosuggest_test.py, 202 checks covering 23 cases, both readers, with a sanitizer check per session: - the default, runtime on and off, HELLISH_SUGGEST_MAX; - each accept key, and what the keys do with no suggestion; - Enter, ^C, ^D; Up/Down; ^R; vi command mode; - escape and multi-line entries; truncation; UTF-8; RPROMPT; - a 4 KB paste, a resize, TERM=dumb; - a user binding winning; POSTDISPLAY in a widget. frontend_budget_test.py gained autosuggest cells: 6.3-6.7 syscalls and 1.2-1.6 writes per key, against a limit of 9 and 3. Full pty suite 118/0. Also green: the golden suite 5363/5363, history_prefix_search 149/0, verify_alloc, help-test. Mutants, all killed: never paint; ignore the variable; no control-byte filter; paint off end-of-line; no wipe on Enter; oldest match; no truncation; accept ignores the ghost; write on every key (the budget test catches it); ignore HELLISH_SUGGEST_MAX; POSTDISPLAY not collected; user binding loses.
Agent-Logs-Url: https://github.com/Univers42/hellish/sessions/0fd1c2d4-549d-41ea-93e3-a5c0f0330add Co-authored-by: SMOSTAFAH1 <131867400+SMOSTAFAH1@users.noreply.github.com>
feat(editor): inline history suggestions as dim ghost text (#140)
…curl-installations [WIP] Fix incomplete output from curl installations
Place the installer guidance at the 4095-byte boundary so a fixed-size printf buffer truncates it after 'Manually'. Co-authored-by: Codex <codex@openai.com>
Run the installer fixture through hellish and assert the full PATH instruction survives the former 4095-byte boundary. Co-authored-by: Codex <codex@openai.com>
Account for the colored 'hellish:' prefix so the old 4095-byte buffer would print 'Manually' and drop the rest. Co-authored-by: Codex <codex@openai.com>
Keep the fixture's ANSI and newline escapes as single backslashes so its output matches the installer path. Co-authored-by: Codex <codex@openai.com>
Ensure the generated padding is followed by the PATH instruction rather than a literal variable reference. Co-authored-by: Codex <codex@openai.com>
…-curl-installations
Member
Author
|
🫡 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Closes #
How I verified it
make re(SAFE=1 debug) builds clean, no warningsmake re OPT=1(SAFE=0 optimized) builds clean, no warningsmake testis green (libc heap)cd tests && ./verify_alloc.shgreen on both SAFE=1 and SAFE=0HELLISH_ALLOC_STATS(SAFE=0)
huge args, deep nesting)
make normclean on touched files (no in-function comments, ≤80 cols,no new globals)
Notes / trade-offs