diff --git a/.claude b/.claude new file mode 160000 index 00000000..bf956c08 --- /dev/null +++ b/.claude @@ -0,0 +1 @@ +Subproject commit bf956c083736fdf34445e574c8940bac73c69105 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6fe44336..f05e8e8e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -254,6 +254,8 @@ jobs: # debug build runs this sweep from the pty job; this is its release run. - name: ${...} quoting sweep against the RELEASE build run: python3 tests/expansion_quote_sweep_test.py build/bin/hellish + - name: curl installer output regression (#164) + run: python3 tests/curl_install_output_test.py build/bin/hellish # The suite diffs hellish against bash byte-for-byte, so anything that # steers bash steers the verdict -- and the people most likely to run it # are the people whose rc exports the most. A ~/.hellishrc framework diff --git a/.gitignore b/.gitignore index 5ad567ec..a22b4fc3 100755 --- a/.gitignore +++ b/.gitignore @@ -9,7 +9,6 @@ out.dot tests/results.log tests/bench_results.txt tests/agnostic_results.tsv -.claude/ # secrets / credentials — never commit .env @@ -44,8 +43,5 @@ tests/glob-zoo/tab file # Host-runnable static binaries produced by `make static`. dist/ -.claude/* -.claude -.CLAUDE -*CLAUDE* site/ +*.env diff --git a/.gitmodules b/.gitmodules index 809cdfb3..7f577967 100755 --- a/.gitmodules +++ b/.gitmodules @@ -12,3 +12,6 @@ path = tests/inception url = ../Inception.git branch = feat/scripts-run-with-launching-shell +[submodule ".claude"] + path = .claude + url = ../claude-deal-with-the-devil.git diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..49fa0ffd --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,77 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +hellish is a from-scratch POSIX(+bashisms, + an opt-in zsh dialect) shell in C, written as a 42 project. Its behaviour is defined by diffing against a **pinned bash 5.3.9** (`bash --posix`) and, for the zsh dialect, a pinned zsh 5.9. In-depth docs: `DEV_DOC.md`, `CONTRIBUTING.md`, and `src//README.md` for each module. + +## Setup + +- Submodules are required: `vendor/libft` (libft + the `ft_malloc` allocator) and `vendor/scripts` (dev tooling). If the build can't find libft, run `git submodule update --init --recursive`. +- Build the oracles once: `make oracle` (→ `~/bash-5.3.9`) and `make zsh-oracle` (→ `~/zsh-5.9`). The golden suite compares against these, not the system bash. +- `make` with no target prints the help page, which lists every target with a description. + +## Build + +| Command | Result | +|---|---| +| `make all` | `MODE=debug`: `-O0 -g3`, ASan+LSan, libc malloc (`SAFE=1`) → `build/bin/hellish` | +| `make OPT=1 all` / `MODE=release` | `-O3 -flto`, `ft_malloc` (`SAFE=0`), no sanitizers | +| `MODE=relwithdebinfo` | `-O2 -g`, `ft_malloc` | +| `make re` / `make re OPT=1` | full rebuild; both must build cleanly before a PR | + +- `-Wall -Wextra -Werror`: a warning fails the build. Add flags with `EXTRA_CFLAGS=…`. Passing `CFLAGS=…` replaces the whole flag set. +- Each mode/SAFE combination has its own object dir (`build/obj--`), but they all link to the same binary path. +- Sources are found with `find src` (excluding `src/platform/`), so a new `.c` file needs no Makefile edit. Only `src/platform/$(TARGET)/` (currently `posix`) is compiled in. Platform-specific code goes in per-file implementations there, not in `#ifdef`s. +- Debug knobs: `--debug=lexer|parser|ast`, `HELLISH_ALLOC_STATS=1` (live bytes at exit, the leak oracle on `SAFE=0`), `HELLISH_BANNER=0`, `HELLISH_NO_UPDATE_CHECK=1`. + +## Test + +A green `make test` alone isn't enough. The test layers are: + +- **Golden suite**: `make test`. Each category in `tests/` is a file with one command per line. Every case runs through `hellish -c` and the pinned `bash --posix`, and stdout, exit status and any written files are diffed. + - Run one category: `cd tests && ./tester ` (`-v` for verbose). + - **A new category file must be added to `test_lists` in `tests/tester`**, or it silently never runs. + - The tester `chmod 000`s `tests/test_files/invalid_permission`. If a run is interrupted, `chmod 755` it before git operations. +- **pty/interactive tests**: `make pty-test`. It discovers every `tests/*.py` automatically. Run a subset with `tests/pty_suite.sh -- `. Each `.py` takes the shell path as its only `argv[1]`; keep it that way. + - Many `make *-test` targets run one pty test; see `make help`. +- **Whole scripts**: `tests/run_scripts.sh`. **Hard corpus**: `tests/hard/run.sh`. +- **Allocator parity**: `cd tests && ./verify_alloc.sh` builds and diffs both `SAFE=1` and `SAFE=0` against bash. Run `make test-release` after touching memory code (ASan and `-O3` can disagree). +- `make arena-stress`: parse arena under ASan with tiny chunks. `make help-test`: every builtin must have a `help` entry. +- `make norm` runs 42 norminette (it only reports, and always exits 0). +- Pre-PR gates: `make re`, `make re OPT=1`, `make test`, `tests/run_scripts.sh`, `tests/hard/run.sh`, `make pty-test`, `cd tests && ./verify_alloc.sh`, `make norm`. + +Leak checking: ASan/LSan only work on `SAFE=1`. On `SAFE=0` use `HELLISH_ALLOC_STATS=1`. + +## Architecture + +One command's path through the shell (`src/core/shell.c`: `main` → `on()` → `repl_shell` → `off()`): + +1. **Input** (`src/infrastructure/input*.c`, readline in `rl_*`/`history*`): one unified line buffer (`t_rl`) for tty, file, `-c` and pipe input, plus multi-line joining and heredoc collection (`src/heredoc/`). +2. **Alias expansion** (`src/alias/`) produces `state->alias_exp`, which is what gets lexed. +3. **Lexer** (`src/lexer/`) and **parser** (`src/parsing/`) build the AST (`incs/ast.h`, `src/infrastructure/ast*.c`). +4. **Reparser** (`src/word_splitting/`) runs between parsing and expansion. It splits coarse `AST_WORD` tokens into quote- and variable-aware sub-tokens and re-tags assignment words. +5. **Expander** (`src/expander/`) handles parameter/arith/command/process substitution, IFS splitting, brace expansion, globbing (`src/glob/`) and redirect construction. `expand_zsh_*` is the zsh dialect. +6. **Execution** (`src/execution/`, fork/job parts in `src/platform/posix/`, `src/job_control/`) runs builtins (`src/builtins/`, dispatch table plus `help_data*.c`), functions, pipelines and compound commands. +7. After each REPL turn, the per-command AST, redirects and input are freed, so memory stays flat over long scripts. + +**`t_shell`** (`incs/shell.h`) holds all the shell's state and is passed everywhere. New state goes there, or in a struct you pass down. **Never add a global variable.** The few legacy globals (word slab, parse arena, env index, readline completion iterators, dir stack) are scheduled for cleanup and aren't precedent. + +**Memory model** (`src/helpers/README.md`): +- All allocation goes through `xmalloc`/`xfree`. They route to either libc or `ft_malloc` at compile time, and a pointer must never cross heaps. +- The **parse arena** (`parena_*`) is a bump allocator, open only around the main cycle's parse. Nested parses (`eval`, `source`, cmdsub bodies) fall through to the heap. `parena_free` is a no-op for arena pointers. Heap memory attached to an arena tree must call `parena_note_attach()`. Before sharing a struct between nodes, check `parena_owns()`; issue #94 was a double free caused by skipping that check. +- The **word slab** (`word_slab_push`) holds argv strings. Values that can escape into the environment (assignments, `for` values) must live on the general heap. +- Shutdown goes through `free_all_state`. Its ordering is deliberate. +- Bugs in libft/ft_malloc get fixed in the submodule's own repo, then the pointer gets bumped here. Don't work around them from the shell. + +## Code style (42 Norm, enforced) + +- At most 5 functions per file and 80 columns per line. No `for` loops and no ternaries. Every file starts with the standard 42 header. Because files hit the 5-function limit, modules have many `foo2.c`, `foo3.c` files; continue that pattern. +- Comments are block comments placed **before** a function or at file scope, never inside a function body. They explain the *why* (root cause, invariant, issue number). Complex functions get a Doxygen block (`@brief`/`@param`/`@return`, plus the hard part). + +## Workflow + +- Every fix ships with a test that covers the bug and nearby cases. +- Branch from `develop` (named `type/short-description`). PRs target `develop`; `main` and `develop` are protected. +- Commits use Conventional Commits (`type(scope): imperative subject`, types `feat|fix|refactor|perf|test|docs|chore|style`), enforced by the hook from `./vendor/scripts/install-hooks.sh`. The body explains the root cause and what the test proves. +- **Don't add `Co-authored-by:` trailers for AI tools.** +- After changing builtins, regenerate the wiki page with `make docs-builtins`. diff --git a/USER_DOC.md b/USER_DOC.md index e9a4da0a..ee39d119 100644 --- a/USER_DOC.md +++ b/USER_DOC.md @@ -97,6 +97,52 @@ A syntax error in any file hellish loads is reported once, with the file and line, and the line itself — the same two lines bash prints — and the lines before it still run. +## The prompt + +`PS1` is the prompt, and it is bilingual: bash's backslash escapes (`\u`, +`\w`, `\$`, `\[ \]`…) and zsh's percent escapes (`%~`, `%F{cyan}`, +`%(?.a.b)`…) work in the same string. `PROMPT` is the zsh spelling with +zsh's exact rules, and wins when both are set; `RPROMPT` is the right +prompt. The full escape list is section 1 of `hellishrc.example`, and +[wiki/interactive.md](wiki/interactive.md#prompt-) has the rest. + +What is live at every render, and what is not: + +| in the prompt | rendered? | +|---|---| +| `$VAR`, `${VAR}`, `${VAR:-x}`, `$?` and the other specials | yes, every time the prompt is drawn | +| `$((…))` | yes | +| `$(…)` and backquotes | **no** — the text stays on screen as written; it would fork on every redraw | + +For a computed segment, set a variable from a hook and put the variable in +the prompt. `HELLISH_PRECMD_FUNCS` runs once before each prompt +(`hellishrc.example`, section 5b): + +```sh +_branch() { BR=$(git branch --show-current 2>/dev/null); } +HELLISH_PRECMD_FUNCS=(_branch) +PS1='${BR:+($BR) }\w \$ ' +``` + +**`prompt` is a shell function, not a builtin.** The installer seeds it +into `~/.config/hellish/rc.d/40-prompt-switch.hsh`, next to the 29 themes +in `~/.config/hellish/themes/`. `type prompt` says whether it is loaded; +from a checkout, `HELLISH_THEMES=share/themes; . share/rc.d/40-prompt-switch.hsh` +loads it by hand. + +**`\A` is not bash's 24-hour clock here.** It is the animated glyph, and it +was there first. A bash `PS1` pasted in with `\A` shows the glyph — or +nothing, since the animation is off by default — where bash shows `14:30`. +Write `\D{%H:%M}` or `%T` for the clock. + +**The animation is off by default, on purpose.** `HELLISH_ANIM=spinner` +(or `pulse`, `ember`) makes `\A` repaint the prompt row about ten times a +second while the shell is idle. That repaint has been reported to corrupt +the prompt on some terminals: an escape sequence printed as text +(`8;2;90;96;106m`), or a cut glyph shown as `�`. It could not be +reproduced on demand, so it ships off. If you turn it on and see that, +`HELLISH_ANIM=off` is the fix. + ## Check that it works ```sh @@ -111,11 +157,62 @@ conf list # plugin framework: what is on and off If a terminal ever misbehaves after an experiment: `reset` — and the escape hatches above get you back to your previous shell at any time. +## Completing `make` targets + +`make ` / `make cl` completes target names from the Makefile in the +current directory (also `gmake`, `/usr/bin/make`, and behind `sudo`, `env X=1`, +`&&`, `(`; a wrapper option that takes an argument, as in `sudo -u root make`, +falls back to filenames). The file is parsed in the shell itself; make is never run. + +- Files read: the first regular file of `GNUmakefile`, `makefile`, `Makefile` + (GNU make's order, symlinks followed), at most 4 MiB. No such file, a + directory, or an unreadable file means plain filename completion. +- Offered: the names on `target:` / `a b c:` / `a::` lines. Not offered: + variable assignments, directives, recipe lines, `define` blocks, names with + `%` or `$`, names starting with `.`; text after the colon is ignored. +- `include` / `-include` / `sinclude` of literal file names are followed ONE + level deep (what an included file includes is ignored). Changing only an + included file does not refresh the cache; touch the Makefile. +- Cache: the targets are reused while the Makefile's name, inode, mtime and + size are unchanged (one `stat` per TAB). +- A word starting with `-` or containing `=` or `/` keeps filename completion, + and so does any `make` line with `-C`, `--directory`, `-f`, `--file` or + `--makefile` before the cursor, or a prefix that matches no target + (`make out.txt`). +- A `complete` spec for `make` (under `shopt -s progcomp`) wins over all of + this; `complete -r make` brings the targets back. + ## Knobs | variable | effect | |---|---| | `HELLISH_BANNER=0\|1` | welcome panel off / on | +| `HELLISH_HISTORY_PREFIX=0\|off` | Up/Down plain history; default is prefix search, read at every prompt | +| `HELLISH_AUTOSUGGEST=1\|on` | inline suggestion, off by default: the newest history line that starts with what you typed, shown dim after the cursor (see below); read at every prompt | +| `HELLISH_SUGGEST_MAX=N` | how many of the newest history lines the suggestion looks at (default 1000; not a number, or below 1, is the default); read at every prompt | | `HELLISH_NO_UPDATE_CHECK=1` | never check for updates, no badge | | `HELLISH_NO_ANIM=1` | no startup animation | | `HELLISH_NO_EXEC=1` | user-install hook: skip for this one session | + +### Inline suggestion + +With `HELLISH_AUTOSUGGEST=1` (set it at the prompt or in the rc file) the +newest history line that starts with what you have typed appears dim after +the cursor. It is a hint, not part of the line: **Right**, **End** or +**Ctrl-F** at the end of the line insert it; anywhere else, and when there +is no hint, those keys do what they always did. Enter runs only what you +typed, and the hint is wiped from the screen. `unset HELLISH_AUTOSUGGEST` +(or `=0`) turns it off at the next prompt. A key you bound yourself +(`bind` / `bindkey` / `.inputrc`) is never taken over. + +Never suggested: a line with a newline, a tab or any other control character +(an old `printf '\e[31m'` cannot repaint your prompt), and nothing at all +while you search (Ctrl-R), in vi command mode, on a continuation prompt, with +the cursor inside the line, or on a line that would not fit the terminal's +width (the hint is cut at the edge, never wrapped). A right prompt +(`RPROMPT`) is hidden while the hint reaches it and comes back when it does +not. + +For zsh widgets (`zle -N`), the variable `POSTDISPLAY` is the text shown +after the cursor: read it in a widget to get the current hint, assign it +(`POSTDISPLAY=' '`) to show your own until the line next changes. diff --git a/docker/Dockerfile b/docker/Dockerfile index 8bfb1cb3..3a1b97e3 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -44,6 +44,11 @@ FROM ${BASE} # in was a red rung on a commit the rung had built fine an hour earlier. # A mirror that keeps failing four times still fails the build -- it is # the transient, not the broken, that is retried. +# apt gets the same loop, update included: an Ubuntu mirror mid-sync serves +# an index that disagrees with its files ("File has unexpected size ... +# Mirror sync in progress?", or a 404 on a .deb the index already moved +# past), and only a fresh update a little later cures it. It turned the +# ubuntu 22.04 rung red on PR #141 before a line was compiled. ARG CC=cc RUN set -eux; \ @@ -51,10 +56,13 @@ RUN set -eux; \ apk add --no-cache build-base readline-dev ncurses-dev git bash \ coreutils python3 clang; \ elif command -v apt-get >/dev/null 2>&1; then \ - apt-get update; \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - build-essential libreadline-dev git ca-certificates bash \ - python3 clang; \ + n=0; until apt-get update -o Acquire::Retries=3 \ + && DEBIAN_FRONTEND=noninteractive apt-get install -y \ + -o Acquire::Retries=3 --no-install-recommends \ + build-essential libreadline-dev git ca-certificates bash \ + python3 clang; do \ + n=$((n + 1)); [ "$n" -ge 4 ] && exit 1; sleep $((n * 15)); \ + done; \ rm -rf /var/lib/apt/lists/*; \ elif command -v pacman >/dev/null 2>&1; then \ pacman -Sy --noconfirm --needed base-devel readline git bash \ diff --git a/hellishrc.example b/hellishrc.example index 5dc76aed..94344039 100644 --- a/hellishrc.example +++ b/hellishrc.example @@ -70,6 +70,15 @@ # # \S is usually the nicer answer than $? anyway: it shows " ✘N" only after a # failure and stays invisible when everything is fine. +# +# $((...)) is live too. $(...) is NOT: command substitution in a prompt is +# never run -- it stays on screen as the text you wrote -- because it would +# fork on every redraw. Compute the value in a hook (section 5b) and put the +# variable in the prompt; the hook runs once per prompt, not per redraw: +# +# _branch() { BR=$(git branch --show-current 2>/dev/null); } +# HELLISH_PRECMD_FUNCS=(_branch) +# PS1='${BR:+($BR) }\w \$ ' # ── ANIMATION: OFF BY DEFAULT, AND THAT IS DELIBERATE ────────────────── # A live style makes \A repaint the prompt rows in place ~10x/second diff --git a/incs/casescan.h b/incs/casescan.h index 6ab16ab5..b69b7cf3 100644 --- a/incs/casescan.h +++ b/incs/casescan.h @@ -21,9 +21,16 @@ `(a*)`, extglob `@(a|b)`, body subshells and nested substitutions all raised the depth first, so they still close normally. Users: the lexer's tokenize_subshell, the word reparser's - reparse_envvar_paren, the expander's find_cmd_sub_end. The cmdsub fast - path's csf_skip_csub stays naive on purpose: a mis-scan there makes - eligibility reject and fork, which is correct by construction. */ + reparse_envvar_paren, the expander's find_cmd_sub_end, span_skip. The + cmdsub fast path's csf_skip_csub stays naive on purpose: a mis-scan + there makes eligibility reject and fork, correct by construction. + The same automaton owns the two other spans inside a substitution + whose bytes are not shell text (issue #139): a `#` comment, and a + heredoc body. bash parses $( ) as a script, so `$(# it'secho x)` + and `$(cat <a ) 'bE)` are fine there; a scanner that only + knows quotes sees an open quote or an early `)` in both. Keeping them + here, in the one scanner every user shares, is what keeps the lexer's + span and the expander's span the same span (casescan2.c). */ #ifndef CASESCAN_H # define CASESCAN_H @@ -34,20 +41,49 @@ paren counting (the pre-#95 behaviour). */ # define CASESCAN_MAX 32 +/* Heredoc operators whose bodies can be owed at once (`cat < arith; -1 = not */ } t_casescan; void casescan_init(t_casescan *cs, int len); /* Feed the UNQUOTED, UNESCAPED character at s[*i]; the caller handles quotes and backslash itself (clearing cs->cmdpos when it does). - Advances *i past what was consumed (one char, or a whole keyword) and - returns the paren-depth delta to apply: +1, -1 or 0. */ + Advances *i past what was consumed (one char, a whole keyword, a whole + comment, a heredoc operator with its word, or the owed heredoc bodies + after a newline) and returns the paren-depth delta: +1, -1 or 0. + Every caller starts past the `$(`, so s[*i - 1] is always readable. */ int casescan_step(t_casescan *cs, const char *s, int *i, int depth); +/* casescan2.c / casescan3.c: the non-shell spans. Each returns true when + it consumed s[*i] (and what followed); false leaves s[*i] to the + paren/word steps. `<<` inside (( )) or $(( )) is a shift, never a + heredoc: a `(` right after a `(` opens that arithmetic span. */ +bool casescan_in(const t_casescan *cs, const char *s, int j); +bool casescan_comment(t_casescan *cs, const char *s, int *i); +bool casescan_heredoc_op(t_casescan *cs, const char *s, int *i); +bool casescan_bodies(t_casescan *cs, const char *s, int *i); +int casescan_paren(t_casescan *cs, const char *s, int *i, int depth); + #endif diff --git a/incs/expander.h b/incs/expander.h index 5c5b855d..137d3a36 100644 --- a/incs/expander.h +++ b/incs/expander.h @@ -31,7 +31,11 @@ typedef struct s_expander_simple_cmd t_ast_node *node; /* the simple command, for looking at siblings */ t_ast_node *curr; /* shorthand for children[i] */ bool found_first; /* true once the command word is seen */ - bool export; /* true when expanding an `export` command */ + bool decl; /* a declaration utility: export, readonly, + declare, typeset, local -- its NAME=value + arguments are not field-split */ + bool decl_next; /* the command word was a bare `command`: + the next word decides `decl` */ bool in_db; /* true inside [[ ]]: no split/glob, case rules */ bool pat_next; /* [[ only: next word sits after ==/=/!= */ int exit_stat; /* running exit status for the command */ @@ -42,6 +46,10 @@ typedef struct s_expander_simple_cmd # define EW_KEEP_AS_ONE 1 # define EW_NO_GLOB 2 +/* decl_word_kind results (expander/utils.c). */ +# define DECL_UTILITY 1 +# define DECL_COMMAND 2 + /* Expander functions */ void expand_word(t_shell *state, t_ast_node *node, t_vec *args, bool keep_as_one); diff --git a/incs/ft_builtins.h b/incs/ft_builtins.h index 1496091e..34b364be 100644 --- a/incs/ft_builtins.h +++ b/incs/ft_builtins.h @@ -137,6 +137,7 @@ int builtin_vcs_info(t_shell *state, t_vec argv); int builtin_zstyle(t_shell *state, t_vec argv); int builtin_zle(t_shell *state, t_vec argv); int builtin_bindkey(t_shell *state, t_vec argv); +int builtin_bind(t_shell *state, t_vec argv); int builtin_add_zsh_hook(t_shell *state, t_vec argv); /* Fire chpwd_functions after a successful cd; see builtin_zsh_hook.c. */ void run_chpwd_hooks(t_shell *state); diff --git a/incs/history.h b/incs/history.h index f0a2b251..9b342821 100644 --- a/incs/history.h +++ b/incs/history.h @@ -14,10 +14,14 @@ # define HISTORY_H # include "shell.h" +# include /* hist_cmds is the truth; readline's own list is a mirror kept in step so the arrow keys agree with what `history` prints. + append_fd is known by identity too (append_dev / append_ino): the user + can redirect or close that number, and then it is theirs (history_fd.c). + appended / readmark exist only for `history -a` and `history -n`, which are defined against "what this session has already written / already read" rather than against the whole list. quiet_expand suppresses the @@ -29,6 +33,8 @@ typedef struct s_history bool quiet_expand; bool recorded; /* this cycle's line is already in the list */ int append_fd; + dev_t append_dev; + ino_t append_ino; size_t appended; size_t readmark; t_vec hist_cmds; diff --git a/incs/lexer.h b/incs/lexer.h index 8d353c97..5fe56af1 100644 --- a/incs/lexer.h +++ b/incs/lexer.h @@ -124,5 +124,6 @@ void advance_bs(char **str); int create_token_consume(char *start, int fd_len, t_tt tt, t_token *out); int check_fd_redirect(char *str, t_token *out); +int io_number_at(t_deque_tok *toks, const char *s); #endif \ No newline at end of file diff --git a/incs/prompt.h b/incs/prompt.h index a983c0ef..6468323a 100644 --- a/incs/prompt.h +++ b/incs/prompt.h @@ -92,6 +92,23 @@ typedef struct s_rl char *shown_prompt; /* the prompt on screen, while reading */ int mode_applied; /* edit_mode readline's keymap is set to */ size_t binds_done; /* zle bindings already installed */ + size_t bind_lines_done; /* `bind` requests already applied */ + bool slot_live; /* interactive read: the slot hooks are on */ + int feat_want[2]; /* per feature (rl_slot.c): wanted now */ + int feat_on[2]; /* what was applied; -1 = nothing yet */ + unsigned feat_open; /* features whose slots this preinit parked */ + unsigned slot_bits; /* slots we bound to a replacement */ + unsigned slot_user; /* slots the user bound: hands off */ + int sg_max; /* HELLISH_SUGGEST_MAX: entries scanned */ + char *sg_ent; /* history entry the suggestion comes from */ + char *sg_prev; /* the line sg_ent was searched for */ + bool sg_known; /* sg_prev / sg_ent describe a finished scan */ + char *sg_txt; /* ghost for the line on screen, or NULL */ + char *sg_post; /* POSTDISPLAY a widget set (zle_rl5.c) */ + char *sg_post_at; /* the line it was set for */ + char *sg_pub; /* what a widget was handed as POSTDISPLAY */ + bool sg_painted; /* a ghost is on screen right now */ + int sg_col; /* column it starts at */ /* --- RPROMPT: rendered here, painted by the editor (rl_rprompt.c) --- */ t_string rp_txt; /* rendered right prompt, width markers stripped */ int rp_w; /* its terminal width; 0 = nothing to paint */ @@ -118,7 +135,9 @@ int rl_read_inproc(t_shell *state, char *prompt); char *rl_editor_enter(t_shell *state, char *prompt); void rl_editor_exit(t_shell *state); t_string prompt_normal(t_shell *state); -t_string prompt_more_input(t_shell *state, struct s_parser *parser); +char *prompt_expand_p(t_shell *state, const char *fmt); +char *prompt_more_input(t_shell *state, struct s_parser *parser); +char *prompt_ps2(t_shell *state, const char *fallback); void buff_readline_init(t_rl *ret); bool rl_eof_exit_ok(t_shell *state); void update_ctx(t_shell *state); diff --git a/incs/sh_alias.h b/incs/sh_alias.h index 7ac9a603..a571e362 100644 --- a/incs/sh_alias.h +++ b/incs/sh_alias.h @@ -96,6 +96,7 @@ bool alias_table_empty(t_hash *aliases); char *alias_scan_line(t_hash *aliases, const char *input); void alias_scan_update(struct s_shell *state); +char *zsh_aliases_view(struct s_shell *state); bool asc_push(t_ascan *a, const char *name, size_t nlen, const char *val); void asc_pop(t_ascan *a); bool asc_active(t_ascan *a, const char *w, size_t len); diff --git a/incs/sh_error.h b/incs/sh_error.h index 6ac13625..7da382f8 100644 --- a/incs/sh_error.h +++ b/incs/sh_error.h @@ -24,6 +24,7 @@ void err_2(t_shell *state, char *p1, char *p2); t_ast_node unexpected(t_shell *state, t_parser *parser, t_ast_node ret, t_deque_tok *tokens); void zsh_brace_hint(t_shell *state, t_deque_tok *tt); +bool sh_interactive(t_shell *state); char *parse_err_ctx(t_shell *state, t_deque_tok *tt, const char *at); void parse_err_echo(t_shell *state, t_deque_tok *tt, diff --git a/incs/shell.h b/incs/shell.h index 978abf6e..29186cb2 100644 --- a/incs/shell.h +++ b/incs/shell.h @@ -122,6 +122,8 @@ char *x_getcwd(void); /* release the pushd/popd dir stack at exit; builtins/builtin_dirstack.c */ void free_dirstack(t_shell *state); void free_compspecs(t_shell *state); +/* release the `make ` target cache; completion/complete_make_cache.c */ +void mk_cache_free(t_shell *state); /* remember / recover a finished bg child's status; src/execution/bg_done.c */ void bg_done_record(t_shell *state, pid_t pid, int status); int bg_done_take(t_shell *state, pid_t pid, int *status); @@ -240,6 +242,17 @@ typedef struct s_call_frame unsigned int shopt; } t_call_frame; +/* A sourced chunk's text as its statements run, and the last token whose + line was resolved in it -- the next resolves from there, forward or back + (exec_where.c). base is NULL outside a sourced file's chunk. */ +typedef struct s_srcpos +{ + const char *base; + size_t len; + const char *memo; + int memo_line; +} t_srcpos; + /* One saved variable for function scope: its value at the moment it was made local / before positional params were replaced, restored on return. */ typedef struct s_scope_save @@ -307,6 +320,20 @@ typedef struct s_bg_done int status; } t_bg_done; +/* The parsed targets of the cwd's Makefile for `make `, valid while + name, device, inode, mtime and size are unchanged + (src/completion/complete_make.c). */ +typedef struct s_make_cache +{ + char *name; + long long dev; + long long ino; + long long size; + long long mtime_s; + long long mtime_ns; + t_vec targets; /* char *, xmalloc'd */ +} t_make_cache; + typedef struct s_shell { /* --- I/O and readline state --- */ @@ -359,9 +386,17 @@ typedef struct s_shell /* The file whose text exec_string is running, so a parse error can say "FILE: line N" the way bash does (error_where.c); NULL for eval, traps, -c and the REPL, whose ctx already says where. err_line is - the file line the chunk under replay starts on. */ + the file line the running chunk starts on, err_pos that chunk, so a + runtime error and $LINENO name the file line too. ctx_src and + ctx_line are what ctx names now, and ctx_num where its line number + starts when ctx has room to rewrite it in place (0 when not): a new + line costs a few digits, a new file a rebuild (exec_where.c). */ const char *err_src; int err_line; + t_srcpos err_pos; + const char *ctx_src; + int ctx_line; + size_t ctx_num; /* Silence the "not supported" notes while oh-my-zsh's plugins load through the shim (omz_shim.c): the user did not write those lines. */ bool zunsup_quiet; @@ -452,7 +487,13 @@ typedef struct s_shell bool cycle_streamed; /* ranges parse+exec'd in-stream */ char *hd_src; /* raw heredoc body string (pre-expand) */ size_t hd_pos; /* read position within hd_src */ + bool hd_from_caller; /* one-shot: the next exec_string + is a $( ) body, whose unterminated + heredoc reads on in the caller's + input as bash's does; eval and + source never do (#139) */ char *hd_stripped; /* tab-stripped heredoc body */ + size_t hd_stripped_len; /* its length while it is set */ bool gather_in_func; /* true while gathering heredocs */ bool gathering_compound; /* mid incomplete compound cmd */ /* --- readline and PRNG --- */ @@ -479,8 +520,12 @@ typedef struct s_shell t_vec var_attrs; /* declare -i/-n attribute table */ t_vec dirstack; /* pushd/popd dir stack */ t_vec compspecs; /* t_compspec: `complete` registrations */ + t_make_cache make_cache; /* `make ` targets (one entry) */ /* --- alias and command cache --- */ t_hash aliases; /* alias name -> t_alias_entry */ + char *aliases_view[2]; /* zsh `aliases` as read: the + table as an assoc (alias_view.c) */ + int aliases_view_at; /* slot the last read returned */ t_hash cmd_cache; /* command name -> resolved path cache */ int edit_mode; /* 0=vi, 1=emacs (rl_editing_mode) */ /* --- argv slab pool (zero-malloc fast path for simple commands) --- */ diff --git a/incs/sys.h b/incs/sys.h index 855dcc1b..9440b3f3 100644 --- a/incs/sys.h +++ b/incs/sys.h @@ -43,11 +43,6 @@ getcwd() failed: No such file or directory\n" # define BLACK_HOLE "/dev/null" # define CMD_OPT "-c" # define EXIT_CMD_NOT_EXEC 126 -# define PERMISSION_DENIED 126 -# define NO_SUCH_FILE_OR_DIR 127 -# define NO_SUCH_DIR 127 -# define OUT_OF_MEM 127 -# define IS_A_DIR 126 # define EXIT_CMD_NOT_FOUND 127 # define EXIT_GENERAL_ERR 1 # define IS_DOLLAR '$' diff --git a/incs/zle.h b/incs/zle.h index 1c2e0c11..986f41e9 100644 --- a/incs/zle.h +++ b/incs/zle.h @@ -46,6 +46,7 @@ typedef struct s_zle_widget { char *name; char *fn; + bool bashx; } t_zle_widget; t_vec *zle_widgets(void); @@ -67,23 +68,47 @@ typedef struct s_zle_bind t_vec *zle_binds(void); void zle_bind_raw(t_zle_bind *b); void zle_bind_add(const char *seq, const char *widget); +char *zle_keyseq(const char *s); const char *zle_bind_widget(const char *seq); void zle_binds_free(void); +/* `bind` (bind_lines.c): readline requests recorded until readline can + take them, replayed by bind_line_apply before each read. */ +typedef struct s_bind_line +{ + char op; + char *map; + char *line; +} t_bind_line; + +t_vec *bind_lines(void); +void bind_line_add(char op, const char *map, const char *line); +void bind_lines_free(void); +void bind_line_apply(t_bind_line *b); + /* The readline side. zle_active() is false outside the editor, which is what the bare `zle` guard in every plugin tests. */ t_shell **zle_state_cell(void); bool zle_active(void); void zle_enter(t_shell *state); int zle_dispatch(int count, int key); + +/* A built-in widget is readline's own command (zle_rl4.c): the key is + bound to the function itself, never through zle_dispatch. t_zle_fn is + readline's rl_command_func_t, spelled without its header. */ +typedef int (*t_zle_fn)(int count, int key); + +t_zle_fn zle_builtin_func(const char *w); +void zle_run_x(t_shell *state, t_zle_widget *w); +void zle_bind_install(t_zle_bind *b); void zle_do_redisplay(void); void zle_do_message(const char *msg); void zle_do_kill_buffer(void); -/* A widget's BUFFER/LBUFFER/RBUFFER/CURSOR are scoped to it: the values - from before are saved and put back (or the names unset) afterwards -- - see zle_params.c. */ -# define ZLE_NPARAMS 4 +/* A widget's BUFFER, LBUFFER, RBUFFER, CURSOR and POSTDISPLAY are scoped to + it: the values from before are saved and put back (or the names unset) + afterwards -- see zle_params.c. */ +# define ZLE_NPARAMS 5 typedef struct s_zle_saved { @@ -102,6 +127,12 @@ void rl_prompt_repaint(t_shell *state, bool force); built-in widget that edits the line directly must call this, or the dispatcher's write-back restores the text it just removed. */ void zle_publish(t_shell *state); + +/* POSTDISPLAY, the text painted after the cursor (rl_suggest3.c): handed + to a widget with the rest, and taken back from it afterwards. */ +void rl_suggest_publish(t_shell *state); +void rl_suggest_collect(t_shell *state); +void rl_suggest_unpaint(void); void zle_do_accept_line(void); int zle_option(t_shell *state, t_vec argv, size_t i); diff --git a/share/rc.d/40-prompt-switch.hsh b/share/rc.d/40-prompt-switch.hsh index 75fe66cb..8e398569 100644 --- a/share/rc.d/40-prompt-switch.hsh +++ b/share/rc.d/40-prompt-switch.hsh @@ -49,22 +49,20 @@ prompt_list() { # Render every theme, then put back the one that was active. # -# `print -rP` is the renderer, not a reimplementation of one: it runs the -# same code the prompt itself does, so a preview cannot disagree with what -# you get after switching. It handles both spellings -- the % escapes of a -# PROMPT theme and the backslash escapes of a PS1 one -- because the zsh -# frontend rewrites into the backslash language and they share the engine. +# Each theme goes through the renderer its own prompt uses, never a +# reimplementation, so a preview cannot disagree with what you get after +# switching. A PROMPT theme is zsh's `%` language, which `print -rP` +# renders exactly as the prompt does. A PS1 theme is `${PS1@P}`: bash's +# prompt-expansion operator, which runs the PS1 rule itself -- both escape +# languages, and a variable's value left as it is. That last part is why +# this is not `print -rP` too: print -P scans what a variable expanded to +# for escapes again, as zsh's does, and the PS1 renderer does not, so a +# PS1 segment holding `100%done` or `\w` previewed as a prompt that never +# appears (issue #134, section 1). # # The \001/\002 pair is stripped: those are the zero-width markers `\[` and # `\]` become, invisible to a line editor and line noise on a terminal. # -# A PS1 theme has its percent signs doubled first. `print -P` always enters -# through the zsh frontend, and a bare `%` there is an escape -- so a PS1 -# containing `$(date +%H:%M)` previewed as `$(date +%H:)`, because -# %M is zsh for the full host name. Doubling makes each one literal, which -# is what the backslash renderer would have done with it anyway. Without -# this the preview quietly lies about any prompt containing a percent. -# # A two-row theme takes two rows here. Flattening it would show a prompt # that does not exist. # @@ -82,7 +80,7 @@ prompt_preview() { if [ -n "${PROMPT:-}" ]; then print -rP "$PROMPT" | tr -d '\001\002' else - print -rP "${PS1//%/%%}" | tr -d '\001\002' + printf '%s\n' "${PS1@P}" | tr -d '\001\002' fi done if [ -n "$was" ]; then diff --git a/src/alias/alias_view.c b/src/alias/alias_view.c new file mode 100644 index 00000000..bfb190e7 --- /dev/null +++ b/src/alias/alias_view.c @@ -0,0 +1,63 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* alias_view.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "sh_alias.h" +#include "shell.h" +#include "env.h" + +/* zsh's `aliases` READ side (issue #137). The write side already reached +** the alias table (expand_zsh_aliases.c, #114); a read saw the plain, +** empty variable, so `${aliases[ll]}` was "" and `${(k)aliases}` listed +** nothing. oh-my-zsh's sudo widget -- bound to ESC ESC, on by default -- +** asks `${${(Az)aliases[$cmd]}[1]:-$cmd}` for the command an alias runs. +** +** In the dialect the parameter IS the table: a read builds the table as +** an associative value, so every reader that already understands one -- +** a subscript, (k) and (v), ${#...}, the flags -- works unchanged. The +** value is returned borrowed, like any variable's, from one of two +** state-owned slots: the one before is still valid, so two reads alive at +** once (`${aliases[a]}${aliases[b]}`) cannot see each other's free. */ + +static void view_push(t_string *out, const t_alias_entry *a) +{ + if (out->len > 1) + vec_push_char(out, ARR_RS); + vec_push_str(out, a->name); + vec_push_char(out, ARR_US); + vec_push_str(out, a->value); +} + +char *zsh_aliases_view(t_shell *state) +{ + t_hash_entry *e; + t_string out; + size_t i; + int slot; + + vec_init(&out); + out.elem_size = 1; + vec_push_char(&out, ARR_ASSOC_MAGIC); + e = (t_hash_entry *)state->aliases.ctx; + i = 0; + while (e && i < state->aliases.cap) + { + if (e[i].key && e[i].value) + view_push(&out, (t_alias_entry *)e[i].value); + i++; + } + vec_push_char(&out, '\0'); + slot = state->aliases_view_at ^ 1; + xfree(state->aliases_view[slot]); + state->aliases_view[slot] = (char *)out.ctx; + state->aliases_view_at = slot; + return (state->aliases_view[slot]); +} diff --git a/src/builtins/builtin_bind.c b/src/builtins/builtin_bind.c new file mode 100644 index 00000000..4420db90 --- /dev/null +++ b/src/builtins/builtin_bind.c @@ -0,0 +1,150 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* builtin_bind.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "builtins_private.h" +#include "zle.h" + +/* bash's `bind`: key bindings and readline settings from a config file. +** +** bind '"\e[A": history-search-backward' Up searches by prefix +** bind 'set completion-ignore-case on' +** bind -p | grep search what is bound now +** +** Before it, `set -o emacs` / `set -o vi` were the only line-editing +** control an rc had: no history-prefix search on Up, no inputrc setting, +** no rebinding C-w (#134 item 7, and the core half of #136). Lines are +** recorded, not applied: readline may not exist yet, and the editing +** mode's keymap replaces whatever was bound before it (bind_lines.c). +** The listings bring readline up first, so they show what a prompt would. +** +** -x (a key that runs a shell command, READLINE_LINE and all) is bash's +** own layer on top of readline; it is a widget here (builtin_bind3.c). */ + +/* The option letter at av[*i][*j]; one that takes an argument takes the + rest of the word or the next one, like getopt. NULL when there is none. + Either way the word it came from is used up: *j is left on its last + byte (the caller steps past it), which for an empty word wraps to + SIZE_MAX so the step lands on its NUL. */ +static const char *bind_optarg(t_vec argv, size_t *i, size_t *j) +{ + char **av; + char *arg; + + av = (char **)argv.ctx; + arg = av[*i] + *j + 1; + if (!*arg) + { + if (*i + 1 >= argv.len) + return (NULL); + arg = av[++*i]; + } + *j = ft_strlen(av[*i]) - 1; + return (arg); +} + +static const char **bind_slot(t_bindopt *o, char c) +{ + if (c == 'm') + return (&o->map); + if (c == 'q') + return (&o->query); + if (c == 'u') + return (&o->unbind); + if (c == 'r') + return (&o->remove); + if (c == 'f') + return (&o->file); + if (c == 'x') + return (&o->unix_cmd); + return (NULL); +} + +static void bind_letter(t_bindopt *o, t_vec argv, size_t *i, size_t *j) +{ + char c; + const char **slot; + size_t n; + + c = ((char **)argv.ctx)[*i][*j]; + slot = bind_slot(o, c); + if (slot) + { + *slot = bind_optarg(argv, i, j); + if (!*slot) + o->missing = c; + } + else if (ft_strchr("lpPvVsSX", c)) + { + n = ft_strlen(o->list); + if (!ft_strchr(o->list, c) && n + 1 < sizeof(o->list)) + o->list[n] = c; + } + else + o->bad = c; +} + +/* Options up to the first word that is not one; returns its index. */ +static size_t bind_parse(t_bindopt *o, t_vec argv) +{ + char **av; + size_t i; + size_t j; + + av = (char **)argv.ctx; + i = 1; + while (i < argv.len && av[i][0] == '-' && av[i][1] + && !o->bad && !o->missing) + { + if (!ft_strcmp(av[i], "--")) + return (i + 1); + j = 1; + while (av[i][j] && !o->bad && !o->missing) + { + bind_letter(o, argv, &i, &j); + j++; + } + i++; + } + return (i); +} + +/* bash's order: listings, -f, -q, -u, -r, -x, -X, then the bindings. A request + that readline would carry out later is checked now, as bash checks it: + the keymap, -u's function, -f's file. */ +int builtin_bind(t_shell *state, t_vec argv) +{ + t_bindopt o; + size_t i; + int st; + + o = (t_bindopt){0}; + i = bind_parse(&o, argv); + if (o.bad || o.missing) + return (bind_usage(state, o.bad, o.missing)); + if (o.map && !bind_rl_keymap_ok(o.map)) + return (ft_eprintf("%s: bind: `%s': invalid keymap name\n", + state->ctx, o.map), 1); + st = bind_list_all(state, o.map, o.list); + st |= bind_file(state, o.map, o.file); + if (o.query) + st |= bind_rl_query(state, o.map, o.query); + st |= bind_unbind(state, o.map, o.unbind); + if (o.remove) + bind_line_add('r', o.map, o.remove); + if (o.unix_cmd) + st |= bind_unix(state, o.unix_cmd); + if (ft_strchr(o.list, 'X')) + bind_x_list(); + while (i < argv.len) + bind_line_add(0, o.map, ((char **)argv.ctx)[i++]); + return (st); +} diff --git a/src/builtins/builtin_bind2.c b/src/builtins/builtin_bind2.c new file mode 100644 index 00000000..2fdd6caa --- /dev/null +++ b/src/builtins/builtin_bind2.c @@ -0,0 +1,80 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* builtin_bind2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "builtins_private.h" +#include "zle.h" + +/* bash's wording, and its status 2 for a malformed call. */ +int bind_usage(t_shell *state, char bad, char missing) +{ + if (missing) + ft_eprintf("%s: bind: -%c: option requires an argument\n", + state->ctx, missing); + else + ft_eprintf("%s: bind: -%c: invalid option\n", state->ctx, bad); + ft_eprintf("bind: usage: bind [-lpsvPSVX] [-m keymap] [-f filename] " + "[-q name] [-u name] [-r keyseq] [-x keyseq:shell-command] " + "[keyseq:readline-function or readline-command]\n"); + return (2); +} + +/* Every readline listing asked for, in bash's order whatever the order + given. -X, the `bind -x` bindings, comes later (builtin_bind3.c). */ +int bind_list_all(t_shell *state, const char *map, const char *list) +{ + const char *order; + + order = "lpPsSvV"; + while (*order) + { + if (ft_strchr(list, *order)) + bind_rl_list(state, map, *order); + order++; + } + return (0); +} + +/* -f FILE: bash reads it on the spot, and so reports a missing file on + the spot; the reading itself waits with everything else, as readline's + own `$include`. A relative name is made absolute now, while it still + means what the caller meant. */ +int bind_file(t_shell *state, const char *map, const char *file) +{ + char *line; + + if (!file) + return (0); + if (access(file, R_OK) != 0) + return (ft_eprintf("%s: bind: %s: cannot read: %s\n", + state->ctx, file, strerror(errno)), 1); + if (file[0] == '/' || !state->cwd.ctx) + line = ft_strjoin("$include ", file); + else + line = ft_asprintf("$include %s/%s", (char *)state->cwd.ctx, file); + if (!line) + return (1); + bind_line_add(0, map, line); + xfree(line); + return (0); +} + +/* -u NAME: unbind every key running it. */ +int bind_unbind(t_shell *state, const char *map, const char *name) +{ + if (!name) + return (0); + if (!bind_rl_fn_ok(name)) + return (ft_eprintf("%s: bind: `%s': unknown function name\n", + state->ctx, name), 1); + bind_line_add('u', map, name); + return (0); +} diff --git a/src/builtins/builtin_bind3.c b/src/builtins/builtin_bind3.c new file mode 100644 index 00000000..65e625e6 --- /dev/null +++ b/src/builtins/builtin_bind3.c @@ -0,0 +1,124 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* builtin_bind3.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "builtins_private.h" +#include "zle.h" + +/* bind -x '"KEYSEQ": COMMAND' -- a key that runs a shell command. +** +** The key becomes a widget of its own, named "bind -x KEYSEQ" (a name no +** `zle -N` can collide with), whose function is the command and which runs +** bash's way (zle_rl5.c: READLINE_LINE / _POINT / _MARK). Parsing follows +** bash's bind_keyseq_to_unix_command: the sequence must be double-quoted; +** a colon or blanks separate it from the command, which may be quoted with +** either quote. */ + +/* bash's isolate_sequence: from `i`, past blanks, one field -- delimited + when it opens with a quote. Returns its end, *start its first byte, or + -1 after saying why. The key sequence is the field at 0, and it alone + must be double-quoted. */ +static int bind_isolate(t_shell *state, const char *s, int i, int *start) +{ + char delim; + bool key; + + key = (i == 0); + while (s[i] == ' ' || s[i] == '\t') + i++; + if (key && s[i] != '"') + return (ft_eprintf("%s: bind: %s: first non-whitespace character " + "is not `\"'\n", state->ctx, s), -1); + delim = 0; + if (s[i] == '"' || s[i] == '\'') + delim = s[i++]; + *start = i; + while (s[i] && s[i] != delim) + { + if (s[i] == '\\' && s[i + 1]) + i++; + i++; + } + if (delim && s[i] != delim) + return (ft_eprintf("%s: bind: no closing `%c' in %s\n", + state->ctx, delim, s), -1); + return (i); +} + +static void bind_x_add(const char *seq, const char *cmd) +{ + char *name; + t_zle_widget *w; + + name = ft_strjoin("bind -x ", seq); + if (!name) + return ; + zle_widget_add(name, cmd); + w = zle_widget_get(name); + if (w) + w->bashx = true; + zle_bind_add(seq, name); + xfree(name); +} + +/* -x LINE */ +int bind_unix(t_shell *state, const char *line) +{ + int ks; + int i; + int cs; + char *seq; + char *cmd; + + i = bind_isolate(state, line, 0, &ks); + if (i < 0) + return (1); + seq = ft_strndup(line + ks, (size_t)(i - ks)); + while (line[i] && line[i] != ':' && line[i] != ' ' && line[i] != '\t') + i++; + if (line[i] != ':' && line[i] != ' ' && line[i] != '\t') + return (xfree(seq), ft_eprintf("%s: bind: %s: missing separator\n", + state->ctx, line), 1); + i = bind_isolate(state, line, i + 1, &cs); + if (i < 0) + return (xfree(seq), 1); + cmd = ft_strndup(line + cs, (size_t)(i - cs)); + if (seq && cmd) + bind_x_add(seq, cmd); + return (xfree(seq), xfree(cmd), 0); +} + +/* -X: the -x bindings, as bash prints them -- the command re-escaped so + the line can be read back. */ +int bind_x_list(void) +{ + t_zle_widget *a; + size_t i; + size_t j; + + a = (t_zle_widget *)zle_widgets()->ctx; + i = -1; + while (++i < zle_widgets()->len) + { + if (!a[i].bashx) + continue ; + ft_printf("\"%s\" \"", a[i].name + 8); + j = -1; + while (a[i].fn[++j]) + { + if (a[i].fn[j] == '"' || a[i].fn[j] == '\\') + ft_printf("\\"); + ft_printf("%c", a[i].fn[j]); + } + ft_printf("\"\n"); + } + return (0); +} diff --git a/src/builtins/builtin_declare.c b/src/builtins/builtin_declare.c index a52965bf..f985bc67 100644 --- a/src/builtins/builtin_declare.c +++ b/src/builtins/builtin_declare.c @@ -59,6 +59,8 @@ static int declare_print_one(t_shell *state, const char *name) return (ft_eprintf("%s: declare: %s: not found\n", state->ctx, name), 1); attr_letters(state, e, attrs); + if (!e->value) + return (ft_printf("declare -%s %s\n", attrs, e->key), 0); if (assoc_is(e->value)) fmt = assoc_format(e->value); else if (arr_is(e->value)) diff --git a/src/builtins/builtin_exec.c b/src/builtins/builtin_exec.c new file mode 100644 index 00000000..2a9f85c2 --- /dev/null +++ b/src/builtins/builtin_exec.c @@ -0,0 +1,110 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* builtin_exec.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "builtins_private.h" +#include "sh_error.h" +#include "sys.h" +#include "zle.h" +#include +#include +#include +#include + +void procsub_detach_all(t_shell *state); + +/* argv[first..] as execve's NULL-terminated array. The strings are not + copied: execve does not need them to outlive the image, and after a + failed exec they still belong to the command. */ +static char **exec_argv(t_vec argv, size_t first) +{ + char **out; + size_t i; + + out = ft_calloc(argv.len - first + 1, sizeof(char *)); + if (!out) + return (NULL); + i = first; + while (i < argv.len) + { + out[i - first] = ((char **)argv.ctx)[i]; + i++; + } + return (out); +} + +/* A failed exec ends a shell that is not interactive -- a script, -c, any + subshell -- with its status, the EXIT trap run first; an interactive + shell only returns the status (bash's execfail rule, less the option). */ +static int exec_failed(t_shell *state, int status) +{ + if (sh_interactive(state) && getpid() == state->shell_pid) + return (status); + exit_clean(state, status); + return (status); +} + +/* Replace the shell with `path`. Should execve return, say why the way + bash does -- a directory is EACCES to execve, "Is a directory" to the + user -- and give 127 for a file that is not there, 126 otherwise. */ +static int exec_run(t_shell *state, char *path, t_vec argv, size_t first) +{ + char **xargv; + char **envp; + struct stat st; + int err; + + xargv = exec_argv(argv, first); + envp = get_envp(state, path); + pal_editor_leave(); + execve(path, xargv, envp); + err = errno; + xfree(xargv); + free_tab(envp); + if (err == EACCES && stat(path, &st) == 0 && S_ISDIR(st.st_mode)) + err = EISDIR; + ft_eprintf("%s: %s: %s\n", state->ctx, path, strerror(err)); + if (err == ENOENT) + return (EXIT_CMD_NOT_FOUND); + return (EXIT_CMD_NOT_EXEC); +} + +/* exec [--] [command [args]]: replace the shell with command, or, with no + command, keep the redirections already applied. Run from a widget, a + failed execve has already handed the terminal over, and the editor + cannot have it back: that shell exits, as it always did. */ +int builtin_exec(t_shell *state, t_vec argv) +{ + char *path; + char *name; + size_t first; + int status; + bool widget; + + first = 1; + if (argv.len > 1 && ft_strcmp(((char **)argv.ctx)[1], "--") == 0) + first = 2; + if (argv.len <= first) + return (procsub_detach_all(state), 0); + name = ((char **)argv.ctx)[first]; + path = exec_lookup(state, name); + if (!path) + { + ft_eprintf("%s: exec: %s: not found\n", state->ctx, name); + return (exec_failed(state, EXIT_CMD_NOT_FOUND)); + } + widget = zle_active(); + status = exec_run(state, path, argv, first); + xfree(path); + if (widget) + exit_clean(state, status); + return (exec_failed(state, status)); +} diff --git a/src/builtins/builtin_exec_path.c b/src/builtins/builtin_exec_path.c new file mode 100644 index 00000000..6efa15d1 --- /dev/null +++ b/src/builtins/builtin_exec_path.c @@ -0,0 +1,75 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* builtin_exec_path.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "builtins_private.h" +#include "cmd_hash.h" +#include + +void path_cache_sync(t_shell *state); +char *exe_path_preferred(char **path_dirs, char *exe_name, bool posix); + +/* The file as bash's exec names it (full_pathname): a relative path joined + to the working directory, less one leading "./". execve gets the same + file either way; this is the name its diagnostic prints -- `exec + ./noexec` in /tmp/w says "/tmp/w/noexec: Permission denied". Frees path + when it returns a new string. */ +static char *exec_abspath(t_shell *state, char *path) +{ + char *cwd; + char *rel; + char *dir; + char *full; + + cwd = (char *)state->cwd.ctx; + if (!path || *path == '/' || !cwd || !*cwd) + return (path); + rel = path; + if (rel[0] == '.' && rel[1] == '/') + rel += 2; + if (cwd[ft_strlen(cwd) - 1] == '/') + dir = ft_strdup(cwd); + else + dir = ft_strjoin(cwd, "/"); + full = NULL; + if (dir) + full = ft_strjoin(dir, rel); + xfree(dir); + if (!full) + return (path); + xfree(path); + return (full); +} + +/* The file `exec NAME` runs, found as the executor finds a command: a + NAME with a '/' as it stands, else its remembered location, else the + first match on PATH -- a plain file without execute permission too, as + bash's exec picks it and then says "Permission denied". NULL when PATH + has nothing by that name. + This used to be find_cmd_path, which belongs to a forked child that + exits next: on a miss it prints its own diagnostic and frees the whole + shell. exec runs in the shell itself, so `exec nosuch` said "command + not found", then printed its own message through the freed state (a + use-after-free under ASan), and an interactive shell could not stay up + as bash's does. Nothing here prints or frees the shell. */ +char *exec_lookup(t_shell *state, char *name) +{ + char *hit; + + if (ft_strchr(name, '/')) + return (exec_abspath(state, ft_strdup(name))); + path_cache_sync(state); + hit = cmd_hash_lookup(&state->cmd_cache, name); + if (hit && access(hit, X_OK) == 0) + return (exec_abspath(state, ft_strdup(hit))); + return (exec_abspath(state, + exe_path_preferred(state->path_dirs, name, false))); +} diff --git a/src/builtins/builtin_fc.c b/src/builtins/builtin_fc.c index 71e7ddcb..51304d2f 100644 --- a/src/builtins/builtin_fc.c +++ b/src/builtins/builtin_fc.c @@ -17,87 +17,89 @@ #include #include "pal_wait.h" -/* Translate a history specifier to a 0-based array index. NULL means "most - recent". Positive numbers are 1-based history positions; negative numbers - are relative offsets from the end (e.g. -1 = last entry). Both are clamped - to the valid range rather than returning an error so fc is tolerant of - out-of-range values from scripts. */ -int fc_resolve_idx(t_shell *state, const char *s) +/* Resolve one history specifier against the `total` entries fc sees (a + 0-based index into *idx). A number: positive is an absolute history + number, negative counts back from the most recent (-1), 0 is the most + recent; out-of-range numbers are clamped, as POSIX lets fc do. Anything + else names the most recent command STARTING with it (`fc -l ech`), and + one that no command starts with is an error. NULL is the most recent. + Returns 0, or 1 after reporting. */ +int fc_resolve_idx(t_shell *state, const char *s, int total, int *idx) { - int n; - int total; + int n; + char **ents; - total = (int)state->hist.hist_cmds.len; + ents = (char **)state->hist.hist_cmds.ctx; + *idx = total - 1; if (!s) - return (total - 1); - n = ft_atoi(s); - if (n < 0) - n = total + n; - else - n = n - 1; - if (n < 0) - n = 0; - if (n >= total) - n = total - 1; - return (n); -} - -/* Print history entries first..last (inclusive, 0-based). Line numbers are - shown 1-based (matching `history` output) unless -n was passed. */ -static void fc_print_entries(t_shell *state, int first, int last, bool nonums) -{ - int i; - - i = first - 1; - while (++i <= last) + return (0); + if (*s == '-' || *s == '+' || ft_isdigit((unsigned char)*s)) { - if (!nonums) - ft_printf("%d\t", i + 1); - ft_printf("%s\n", ((char **)state->hist.hist_cmds.ctx)[i]); + n = ft_atoi(s) - 1; + if (n < 0) + n += total + 1; + *idx = n; + if (n < 0) + *idx = 0; + if (n >= total) + *idx = total - 1; + return (0); } + while (*idx >= 0 && ft_strncmp(ents[*idx], s, ft_strlen(s))) + (*idx)--; + if (*idx >= 0) + return (0); + return (ft_eprintf("%s: fc: no command found\n", state->ctx), 1); } -static bool fc_check_nonums(char **av, int ac) +/* One entry, bash's layout: `N cmd`, or ` cmd` with -n. */ +static void fc_print_entry(t_shell *state, int i, bool nonum) { - int i; - - i = -1; - while (++i < ac) - if (av[i] && ft_strcmp(av[i], "-n") == 0) - return (true); - return (false); + if (!nonum) + ft_printf("%d", i + 1); + ft_printf("\t %s\n", ((char **)state->hist.hist_cmds.ctx)[i]); } -/* fc -l [first [last]]: list history entries. With no range, default to the - last 16 entries. Reversed (-r) swaps first and last. -n suppresses line - numbers. The range arguments may be omitted individually — ac tracks which - were given. */ -int fc_list(t_shell *state, char **av, int ac, bool reverse) +/* Print first..last, whichever way round they are: first after last + lists backwards, and -r flips whichever direction that gave. */ +static void fc_print_range(t_shell *state, int first, int last, t_fcopt *o) { - int first; - int last; - int tmp; + int step; + int swap; - first = fc_resolve_idx(state, NULL); - if (ac > 0) - first = fc_resolve_idx(state, av[0]); - last = fc_resolve_idx(state, NULL); - if (ac > 1) - last = fc_resolve_idx(state, av[1]); - if (first == last && ac < 2) + if (o->rev) { - first = (int)state->hist.hist_cmds.len - 16; - if (first < 0) - first = 0; - last = (int)state->hist.hist_cmds.len - 1; + swap = first; + first = last; + last = swap; } - if (reverse && first < last) + step = 1 - 2 * (first > last); + while (first != last + step) { - tmp = first; - first = last; - last = tmp; + fc_print_entry(state, first, o->nonum); + first += step; } - fc_print_entries(state, first, last, fc_check_nonums(av, ac)); +} + +/* fc -l [-nr] [first [last]]. No operand: the last 16. One: from it to + the most recent. */ +int fc_list(t_shell *state, t_fcopt *o) +{ + int total; + int first; + int last; + + total = fc_total(state); + if (total <= 0) + return (0); + first = total - 16; + if (first < 0) + first = 0; + if (fc_resolve_idx(state, NULL, total, &last) + || (o->nops > 0 && fc_resolve_idx(state, o->ops[0], total, &first)) + || (o->nops > 1 && fc_resolve_idx(state, o->ops[1], total, &last))) + return (1); + fc_print_range(state, first, last, o); return (0); } diff --git a/src/builtins/builtin_fc3.c b/src/builtins/builtin_fc3.c new file mode 100644 index 00000000..75476bfb --- /dev/null +++ b/src/builtins/builtin_fc3.c @@ -0,0 +1,101 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* builtin_fc3.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "builtins_private.h" +#include "history.h" + +/* fc's command line and its view of the history. +** +** Options were matched as whole words -- `-l` and `-lr` only -- so every +** other spelling of a listing fell through to EDIT mode and launched +** $EDITOR on the history: `fc -ln -1`, `-nl`, `-lnr`, `-l -n`. The omz sudo +** widget asks `$(fc -ln -1)` for the previous command when ESC ESC hits an +** empty line (#137), which started vim inside a command substitution. +** Options now cluster the POSIX way, `--` ends them, and a `-` followed by +** a digit is the negative-number OPERAND it always is in fc (`fc -l -3`). */ + +static int fc_bad_opt(t_shell *state, char c) +{ + ft_eprintf("%s: fc: -%c: invalid option\n", state->ctx, c); + ft_eprintf("fc: usage: fc [-e ename] [-lnr] [first] [last]" + " or fc -s [pat=rep] [command]\n"); + return (2); +} + +/* One option word, `-lnr` or `-e` with its editor. Returns how many + argv words it used (1 or 2), or -1 after reporting an error. */ +static int fc_cluster(t_shell *state, char **av, int i, t_fcopt *o) +{ + const char *p; + + p = av[i]; + while (*++p && *p != 'e') + { + if (!ft_strchr("lnrs", *p)) + return (fc_bad_opt(state, *p), -1); + o->list |= (*p == 'l'); + o->nonum |= (*p == 'n'); + o->rev |= (*p == 'r'); + o->subst |= (*p == 's'); + } + if (*p != 'e') + return (1); + if (p[1]) + return (o->editor = p + 1, 1); + if (!av[i + 1]) + return (ft_eprintf("%s: fc: -e: option requires an argument\n", + state->ctx), -1); + return (o->editor = av[i + 1], 2); +} + +/* Fill `o` from argv. Returns 0, or 2 on a usage error (already reported). */ +int fc_parse(t_shell *state, char **av, int ac, t_fcopt *o) +{ + int i; + int used; + + *o = (t_fcopt){0}; + i = 1; + while (i < ac && av[i][0] == '-' && av[i][1] + && !ft_isdigit((unsigned char)av[i][1])) + { + if (ft_strcmp(av[i], "--") == 0) + { + i++; + break ; + } + used = fc_cluster(state, av, i, o); + if (used < 0) + return (2); + i += used; + } + if (o->editor && ft_strcmp(o->editor, "-") == 0) + o->subst = true; + o->ops = av + i; + o->nops = ac - i; + return (0); +} + +/* How many entries fc sees: the whole list, minus this very line when it + is already in it -- bash records a command before it runs, so `fc -l -1` + would otherwise name the fc command itself. Inside a zle widget the line + being edited is not recorded yet, and -1 is the previous command, which + is what zsh code means by it. */ +int fc_total(t_shell *state) +{ + int n; + + n = (int)state->hist.hist_cmds.len; + if (state->hist.recorded && n > 0) + n--; + return (n); +} diff --git a/src/builtins/builtin_proc.c b/src/builtins/builtin_proc.c index 01fc7bba..a0fae171 100644 --- a/src/builtins/builtin_proc.c +++ b/src/builtins/builtin_proc.c @@ -20,53 +20,6 @@ #include #include -void procsub_detach_all(t_shell *state); - -/* Build a NULL-terminated argv array for execve from argv[1..] (skipping - argv[0] which is the "exec" word itself). We calloc so the sentinel NULL - is already in place without an explicit assignment. The strings are NOT - duplicated — execve does not need them to outlive the process image. */ -static char **dup_exec_argv(t_vec argv) -{ - char **out; - size_t i; - - out = ft_calloc(argv.len, sizeof(char *)); - if (!out) - return (NULL); - i = 1; - while (i < argv.len) - { - out[i - 1] = ((char **)argv.ctx)[i]; - i++; - } - return (out); -} - -/* exec [command [args]]: replace the shell with command (or, with no command, - leave the already-applied redirections in place). */ -int builtin_exec(t_shell *state, t_vec argv) -{ - char *path; - char **xargv; - char **envp; - - if (argv.len < 2) - return (procsub_detach_all(state), 0); - if (find_cmd_path(state, ((char **)argv.ctx)[1], &path) != 0) - { - ft_eprintf("%s: exec: %s: not found\n", state->ctx, - ((char **)argv.ctx)[1]); - exit(127); - } - xargv = dup_exec_argv(argv); - envp = get_envp(state, path); - pal_editor_leave(); - execve(path, xargv, envp); - ft_eprintf("%s: exec: %s: %s\n", state->ctx, path, strerror(errno)); - exit(126); -} - /* waitpid() failed (ECHILD): the child was already reaped by reap_background_children's WNOHANG poll between list items, or by job_update_status's poll when `jobs` listed it. bash remembers a diff --git a/src/builtins/builtin_read2.c b/src/builtins/builtin_read2.c index 3bd523dd..cd0a2e20 100644 --- a/src/builtins/builtin_read2.c +++ b/src/builtins/builtin_read2.c @@ -77,17 +77,29 @@ char *last_field(char *p, const char *ifs, bool raw) } /* Write a variable into the environment. `value_owned` is transferred — - do not free it after calling this; env_create takes ownership. */ -void rd_set_var(t_shell *state, char *name, char *value_owned) + do not free it after calling this; env_create takes ownership. A + readonly variable is refused with bash's message and left as it was + (false); read goes on running, as bash's does -- it is not a special + builtin, so the error is not fatal. */ +bool rd_set_var(t_shell *state, char *name, char *value_owned) { + if (is_readonly_var(state, name)) + { + ft_eprintf("%s: %s: readonly variable\n", state->ctx, name); + xfree(value_owned); + return (false); + } env_set(&state->env, env_create(ft_strdup(name), value_owned, false)); + return (true); } /* Split `line` across the variable names in argv[o->first..end-1]. All but the last variable get one next_field() result; the last one gets last_field() which includes the rest of the line minus trailing IFS - whitespace. This matches the POSIX `read` field-splitting algorithm. */ -void assign_words(t_shell *state, char *line, t_vec argv, t_rdopt *o) + whitespace. This matches the POSIX `read` field-splitting algorithm. + A readonly name stops it the way it stops bash: before the last + variable, status 2 and nothing after it assigned; the last, status 1. */ +int assign_words(t_shell *state, char *line, t_vec argv, t_rdopt *o) { size_t i; char *p; @@ -98,10 +110,12 @@ void assign_words(t_shell *state, char *line, t_vec argv, t_rdopt *o) i = o->first; while (i + 1 < argv.len) { - rd_set_var(state, ((char **)argv.ctx)[i], - next_field(&p, o->ifs, o->raw)); + if (!rd_set_var(state, ((char **)argv.ctx)[i], + next_field(&p, o->ifs, o->raw))) + return (2); skip_delim(&p, o->ifs); i++; } - rd_set_var(state, ((char **)argv.ctx)[i], last_field(p, o->ifs, o->raw)); + return (!rd_set_var(state, ((char **)argv.ctx)[i], + last_field(p, o->ifs, o->raw))); } diff --git a/src/builtins/builtin_read3.c b/src/builtins/builtin_read3.c index 5be54b17..ee19e16c 100644 --- a/src/builtins/builtin_read3.c +++ b/src/builtins/builtin_read3.c @@ -61,21 +61,26 @@ int rd_wait_input(t_rdopt *o) /* Route the completed line to its destination: -a fills the named array, no variable names sends the whole line to $REPLY, otherwise the line is IFS-split across the named variables. rd_set_var takes ownership of the - line; the array/word paths copy fields out of it, so it is freed here. */ -static void rd_dispatch(t_shell *state, t_vec argv, char *line, t_rdopt *o) + line; the array/word paths copy fields out of it, so it is freed here. + Returns the status a readonly variable forces, 0 when every assignment + was made: bash's 2 for REPLY, 1 for an -a array (assign_words has the + named-variable rules). */ +static int rd_dispatch(t_shell *state, t_vec argv, char *line, t_rdopt *o) { - if (o->aname) - { + int st; + + if (o->first >= argv.len && !o->aname) + return (2 * !rd_set_var(state, "REPLY", line)); + st = 0; + if (o->aname && is_readonly_var(state, o->aname)) + st = (ft_eprintf("%s: %s: readonly variable\n", state->ctx, + o->aname), 1); + else if (o->aname) rd_assign_array(state, line, o); - xfree(line); - } - else if (o->first >= argv.len) - rd_set_var(state, "REPLY", line); else - { - assign_words(state, line, argv, o); - xfree(line); - } + st = assign_words(state, line, argv, o); + xfree(line); + return (st); } /* read [-r] [-n N] [-N N] [-d C] [-t S] [-p PROMPT] [-a NAME] [var ...]: @@ -84,6 +89,10 @@ static void rd_dispatch(t_shell *state, t_vec argv, char *line, t_rdopt *o) Returns 1 on EOF even if some data was read (mimics bash), so `while read line; do …; done` processes the last line before stopping even when the file lacks a trailing newline. + At EOF with nothing read the variables are still assigned, empty (an + -a array emptied), as bash does. They used to keep their old values, so + `while read -r l || [ -n "$l" ]` -- the loop that also handles a last + line with no newline -- never ended: $l still held that last line. -N reads a fixed byte count and must NOT field-split: an empty IFS is exactly that, so the ordinary assign_words path handles it with no second code path to keep in step. */ @@ -92,6 +101,7 @@ int builtin_read(t_shell *state, t_vec argv) char *line; t_rdopt o; int eof; + int st; o = (t_rdopt){.nchars = -1, .delim = '\n', .tmo_ms = -1}; o.first = parse_read_opts2(argv, &o); @@ -105,7 +115,9 @@ int builtin_read(t_shell *state, t_vec argv) return (xfree(o.ifs), 142); line = read_one_line(&o, &eof); if (!line) - return (xfree(o.ifs), 1); - rd_dispatch(state, argv, line, &o); + line = ft_strdup(""); + st = rd_dispatch(state, argv, line, &o); + if (st) + return (xfree(o.ifs), st); return (xfree(o.ifs), eof != 0); } diff --git a/src/builtins/builtin_set2.c b/src/builtins/builtin_set2.c index 6bd3559e..f6d72967 100644 --- a/src/builtins/builtin_set2.c +++ b/src/builtins/builtin_set2.c @@ -13,8 +13,9 @@ #include "builtins_private.h" #include "sh_input.h" -/* Print all environment entries in `name=value` form (or just `name` when a - variable has no value). This is the output of bare `set` — it differs from +/* Print all set variables in `name=value` form; a name exported before it + has a value is not set, and bash's `set` leaves it out too. This is the + output of bare `set` — it differs from `env` in that it includes all variables, not just the exported ones, and it does not quote the values. */ /* Arrays list in bash's display form, name=([0]="a" ...), so the @@ -38,17 +39,12 @@ void set_print_env(t_shell *state) while (i < state->env.len) { e = &((t_env *)state->env.ctx)[i]; - if (e->key) + if (e->key && e->value) { - if (e->value) - { - if (arr_is(e->value)) - print_env_array(e); - else - ft_printf("%s=%s\n", e->key, e->value); - } + if (arr_is(e->value)) + print_env_array(e); else - ft_printf("%s\n", e->key); + ft_printf("%s=%s\n", e->key, e->value); } i++; } diff --git a/src/builtins/builtin_zle.c b/src/builtins/builtin_zle.c index 871f13a2..7e4412ae 100644 --- a/src/builtins/builtin_zle.c +++ b/src/builtins/builtin_zle.c @@ -35,6 +35,11 @@ ** widget that appears to run and does nothing is indistinguishable from a ** working one until the user presses the key. ** +** Any other name readline has a command for -- `history-beginning-search- +** backward`, `beginning-of-line`, `kill-word` -- runs that command +** (zle_builtin_func), and the widget's BUFFER family is refreshed after it, +** or the dispatcher's write-back would restore the line it replaced. +** ** `reset-prompt` re-expands the prompt and redraws it, as zsh does -- the ** half of #77 item 5 that had to wait until the line was read in the shell ** process: a widget that moved the shell (dirhistory) now shows the new @@ -45,6 +50,8 @@ */ static int zle_builtin(t_shell *state, const char *name) { + t_zle_fn f; + if (!ft_strcmp(name, "reset-prompt") || !ft_strcmp(name, ".reset-prompt")) return (rl_prompt_repaint(state, true), zle_do_redisplay(), 0); if (!ft_strcmp(name, "redisplay") || !ft_strcmp(name, ".redisplay")) @@ -55,6 +62,9 @@ static int zle_builtin(t_shell *state, const char *name) return (zle_do_accept_line(), 0); if (zle_widget_get(name)) return (zle_run_widget(state, name)); + f = zle_builtin_func(name); + if (f) + return (f(1, 0), zle_publish(state), 0); return (ft_eprintf("%s: zle: %s: no such widget\n", state->ctx, name), 1); } diff --git a/src/builtins/builtins_private.h b/src/builtins/builtins_private.h index e913aba1..e34d8f1e 100644 --- a/src/builtins/builtins_private.h +++ b/src/builtins/builtins_private.h @@ -362,11 +362,26 @@ void skip_delim(char **pp, const char *ifs); char *last_field(char *p, const char *ifs, bool raw); size_t parse_read_opts2(t_vec argv, t_rdopt *o); void rd_assign_array(t_shell *state, char *line, t_rdopt *o); -void rd_set_var(t_shell *state, char *name, char *value_owned); -void assign_words(t_shell *state, char *line, t_vec argv, t_rdopt *o); +bool rd_set_var(t_shell *state, char *name, char *value_owned); +int assign_words(t_shell *state, char *line, t_vec argv, t_rdopt *o); long rd_secs_ms(const char *s); -int fc_resolve_idx(t_shell *state, const char *s); -int fc_list(t_shell *state, char **av, int ac, bool reverse); +/* fc's parsed command line (builtin_fc3.c): -e NAME, the -l -n -r -s + switches, and the first [last] operands that follow them. */ +typedef struct s_fcopt +{ + const char *editor; + char **ops; + int nops; + bool list; + bool nonum; + bool rev; + bool subst; +} t_fcopt; + +int fc_parse(t_shell *state, char **av, int ac, t_fcopt *o); +int fc_total(t_shell *state); +int fc_resolve_idx(t_shell *state, const char *s, int total, int *idx); +int fc_list(t_shell *state, t_fcopt *o); int fc_write_tmp(t_shell *state, char *tmpf, int first, int last); int fc_run_editor(t_shell *state, const char *editor, char *tmpf); int fc_edit_run(t_shell *state, const char *editor, int first, int last); @@ -451,4 +466,33 @@ int declare_names(t_shell *state, t_vec argv, size_t i); int list_all(t_shell *state); int list_named(t_shell *state, t_vec argv, size_t i); +/* The file `exec NAME` runs, by bash's full name, or NULL when PATH has + none (builtin_exec_path.c). Looks only: never prints, never exits. */ +char *exec_lookup(t_shell *state, char *name); + +/* `bind` (builtin_bind*.c, platform/posix/bind_rl*.c). */ +typedef struct s_bindopt +{ + const char *map; + const char *query; + const char *unbind; + const char *remove; + const char *file; + const char *unix_cmd; + char list[9]; + char bad; + char missing; +} t_bindopt; + +int bind_usage(t_shell *state, char bad, char missing); +int bind_list_all(t_shell *state, const char *map, const char *list); +int bind_file(t_shell *state, const char *map, const char *file); +int bind_unbind(t_shell *state, const char *map, const char *name); +int bind_rl_list(t_shell *state, const char *map, char opt); +int bind_rl_query(t_shell *state, const char *map, const char *name); +bool bind_rl_keymap_ok(const char *name); +bool bind_rl_fn_ok(const char *name); +int bind_unix(t_shell *state, const char *line); +int bind_x_list(void); + #endif diff --git a/src/builtins/collect_and_print_exported.c b/src/builtins/collect_and_print_exported.c index de387fff..5240cdb9 100644 --- a/src/builtins/collect_and_print_exported.c +++ b/src/builtins/collect_and_print_exported.c @@ -12,15 +12,29 @@ #include "builtins_private.h" -/* Walk the env table and format each exported entry as `export KEY="value"`. - Variables without a value (exported but never assigned) use an empty - string. All strings are heap-allocated — print_and_free_list owns them. */ +/* One exported entry as `export KEY="value"`. A variable exported but not + yet assigned is listed bare, `export KEY`, as bash lists it: printing + KEY="" would re-create it SET when the list is fed back to the shell. */ +static char *export_line(t_env *e) +{ + char *s; + char *tmp; + + if (!e->value) + return (ft_asprintf("export %s", e->key)); + tmp = dquote_str(e->value); + s = ft_asprintf("export %s=\"%s\"", e->key, tmp); + xfree(tmp); + return (s); +} + +/* Walk the env table and format each exported entry. All strings are + heap-allocated — print_and_free_list owns them. */ static void collect_exported_list(t_shell *st, t_vec *list) { size_t j; t_env *e; char *s; - char *tmp; vec_init(list); list->elem_size = sizeof(char *); @@ -30,9 +44,7 @@ static void collect_exported_list(t_shell *st, t_vec *list) e = &((t_env *)st->env.ctx)[j]; if (e->exported) { - tmp = dquote_str(e->value); - s = ft_asprintf("export %s=\"%s\"", e->key, tmp); - xfree(tmp); + s = export_line(e); vec_push(list, &s); } j++; diff --git a/src/builtins/declare_subscript.c b/src/builtins/declare_subscript.c index ea9becaa..dc679bd3 100644 --- a/src/builtins/declare_subscript.c +++ b/src/builtins/declare_subscript.c @@ -53,8 +53,10 @@ char *declare_assign_eq(const char *word) return (NULL); } -/* One NAME or NAME=VALUE operand: NAME=VALUE assigns (export flag from -x), - a bare NAME with no '=' is a no-op that just accepts the name. +/* One NAME or NAME=VALUE operand: NAME=VALUE assigns (export flag from -x). + A bare NAME with -x marks it exported, or, when it is not set, leaves a + record with no value that holds the attribute until it is, exactly as + `export NAME` does. Without -x a bare NAME just accepts the name. subscript_assign() rewrites key and value in place when the key carries a [subscript], so a plain name falls through it untouched and the two spellings cannot drift apart. */ @@ -68,6 +70,8 @@ void declare_assign(t_shell *state, const char *word, int exprt) { if (exprt && env_get(&state->env, (char *)word)) env_get(&state->env, (char *)word)->exported = true; + else if (exprt && ft_is_valid_ident((char *)word)) + env_set(&state->env, env_create(ft_strdup(word), NULL, true)); return ; } ev = env_create(ft_strndup(word, eq - word), ft_strdup(eq + 1), diff --git a/src/builtins/export_helpers2.c b/src/builtins/export_helpers2.c index 0ed5fda8..a50a03b8 100644 --- a/src/builtins/export_helpers2.c +++ b/src/builtins/export_helpers2.c @@ -97,20 +97,32 @@ static char *export_apply_append(t_shell *st, char *id, char *val) return (joined); } -int handle_identifier(t_shell *st, char *id, char *val, const char *argv0) +/* `export NAME` with no value. An existing variable is marked; a name + that is not set yet gets a record with no value, so the attribute + waits for it: `export X; X=1` exports X (POSIX), and until then X is + still unset -- ${X+set} is empty, no child sees it, and `export -p` + lists it bare, as bash does. */ +static void export_name_only(t_shell *st, char *id) { t_env *e; + e = env_get(&st->env, id); + if (e) + { + e->exported = true; + xfree(id); + return ; + } + env_set(&st->env, env_create(id, NULL, true)); +} + +int handle_identifier(t_shell *st, char *id, char *val, const char *argv0) +{ val = export_apply_append(st, id, val); if (ft_is_valid_ident(id)) { if (!val) - { - e = env_get(&st->env, id); - if (e) - e->exported = true; - xfree(id); - } + export_name_only(st, id); else env_set_shaped(st, (t_env){true, id, val}); return (0); diff --git a/src/builtins/hash_builtins_zsh.c b/src/builtins/hash_builtins_zsh.c index 11f73de0..9991ab2c 100644 --- a/src/builtins/hash_builtins_zsh.c +++ b/src/builtins/hash_builtins_zsh.c @@ -38,12 +38,13 @@ void fill_builtin_hash4(t_hash *h) fill_builtin_hash5(h); } -/* Programmable completion (#72 phase 4). Not zsh-only -- they live here - because fill_builtin_hash4 is where the registration chain currently - ends, and the norm caps a function at 25 lines. */ +/* Programmable completion (#72 phase 4) and `bind`. Not zsh-only -- they + live here because fill_builtin_hash4 is where the registration chain + currently ends, and the norm caps a function at 25 lines. */ void fill_builtin_hash5(t_hash *h) { hash_set(h, "compgen", (void *)builtin_compgen); hash_set(h, "complete", (void *)builtin_complete); hash_set(h, "compopt", (void *)builtin_compopt); + hash_set(h, "bind", (void *)builtin_bind); } diff --git a/src/builtins/help_data2.c b/src/builtins/help_data2.c index a5cdf078..11aee64f 100644 --- a/src/builtins/help_data2.c +++ b/src/builtins/help_data2.c @@ -28,6 +28,10 @@ static const t_help g_help_2[] = { {"complete", "shell", "complete [-abcdfkv] [-A action] [-W list] [-F fn] [-pr] name ...", "register what to offer when completing an argument of name"}, +{"bind", "shell", + "bind [-lpsvPSVX] [-m keymap] [-f file] [-q name] [-u name] [-r keyseq]" + " [-x keyseq:command] [keyseq:function | readline-line]", + "set key bindings and readline variables, or list them"}, {"compopt", "shell", "compopt [-o|+o option] [-DEI] [name ...]", "change a completion spec's options, or the running one's"}, diff --git a/src/completion/complete_commands.c b/src/completion/complete_commands.c index 6182fd9d..ff694b08 100644 --- a/src/completion/complete_commands.c +++ b/src/completion/complete_commands.c @@ -45,7 +45,7 @@ char *g_builtins[] = { "mapfile", "readarray", "declare", "typeset", "shopt", "pretty", "dirs", "setopt", "unsetopt", "emulate", "print", "autoload", "is-at-least", - "zmodload", "zstyle", "compdef", "zle", "bindkey", "add-zsh-hook", + "zmodload", "zstyle", "compdef", "zle", "bindkey", "bind", "add-zsh-hook", "compgen", "complete", "compopt", "colors", "vcs_info", NULL }; diff --git a/src/completion/complete_make.c b/src/completion/complete_make.c new file mode 100644 index 00000000..c9601da2 --- /dev/null +++ b/src/completion/complete_make.c @@ -0,0 +1,100 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* make : complete target names from the Makefile in the current +** directory, parsed in-process (no fork, make is never run). +** +** WHAT IS READ. The first regular file of GNUmakefile, makefile, Makefile +** (GNU make's order, symlinks followed). Anything else -- no file, a +** directory, an unreadable file -- is "not mine": NULL, so the word keeps +** filename completion. At most 4 MiB are read. +** +** WHAT COUNTS. `targets : ...` rule lines (`a b c:` gives three, `a::` is +** a rule). Not offered: variable assignments, directives, recipe lines +** (TAB), define blocks, names with % or $, names starting with '.', and +** names outside [A-Za-z0-9_./+-]. Prerequisites and `## docs` are ignored. +** A backslash-newline is joined with NOTHING in between (GNU make puts a +** space), which keeps `lo\ng:` a single name. +** +** INCLUDES. `include`, `-include` and `sinclude` with literal names are +** followed, relative to the cwd, ONE level deep: what an included file +** includes is not read (that also makes circular includes harmless). +** +** CACHE. One entry in the shell (t_make_cache), valid while the chosen +** file's name, device, inode, mtime and size are unchanged; one stat per +** TAB. LIMITATION: a change to an INCLUDED file alone does not invalidate +** it -- touch the Makefile. +** +** WHEN IT STAYS OUT. A word starting with '-' or holding '=' or '/' is not +** a target; and if an earlier argument is -C, --directory*, -f, --file* or +** --makefile*, the Makefile is not the one in the cwd, so filenames win. +** Short options are only recognised alone (`-C dir`, `-fx`), not clustered +** (`-kC dir`). Zero matching targets also gives NULL, so `make out.txt` +** still completes a file. +*/ + +#include "complete_make.h" +#include + +/* readline's generator protocol: state 0 starts over, then one match per + call. The iterator is the one documented function-local static, as in + pc_generator (progcomp.c). Strings go to readline via rl_dup. */ +static char *mk_generator(const char *text, int state) +{ + static size_t i; + t_vec *v; + char *s; + + v = &(*zle_state_cell())->make_cache.targets; + if (!state) + i = 0; + while (i < v->len) + { + s = ((char **)v->ctx)[i++]; + if (!ft_strncmp(s, text, ft_strlen(text))) + return (rl_dup(s)); + } + return (NULL); +} + +static int mk_pre(const char *w, size_t n, const char *lit) +{ + return (n >= ft_strlen(lit) && !ft_strncmp(w, lit, ft_strlen(lit))); +} + +/* Does an earlier argument point make somewhere else? */ +static int mk_diverts(t_pv_line *a) +{ + const char *w; + size_t n; + + n = prov_word(a, &w); + while (n) + { + if (n >= 2 && w[0] == '-' && (w[1] == 'C' || w[1] == 'f')) + return (1); + if (mk_pre(w, n, "--directory") || mk_pre(w, n, "--file") + || mk_pre(w, n, "--makefile")) + return (1); + n = prov_word(a, &w); + } + return (0); +} + +char **comp_make(t_shell *st, const char *text, t_pv_line *a) +{ + if (text[0] == '-' || ft_strchr(text, '=') || ft_strchr(text, '/')) + return (NULL); + if (mk_diverts(a) || !mk_load(st)) + return (NULL); + return (rl_completion_matches(text, mk_generator)); +} diff --git a/src/completion/complete_make.h b/src/completion/complete_make.h new file mode 100644 index 00000000..b4cf7865 --- /dev/null +++ b/src/completion/complete_make.h @@ -0,0 +1,83 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make.h :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Per-command completion providers and the Makefile target provider. + Included only by src/completion/complete_provider.c and + src/completion/complete_make*.c. */ + +#ifndef COMPLETE_MAKE_H +# define COMPLETE_MAKE_H + +# include "libft.h" +# include "shell.h" +# include "zle.h" +# include +# include + +/* The line under the cursor, as a cursor over the words that precede the + word being completed: [pos, end) is what has not been read yet. */ +typedef struct s_pv_line +{ + const char *l; + size_t pos; + size_t end; +} t_pv_line; + +typedef char **(*t_prov_fn)(t_shell *st, const char *text, t_pv_line *a); + +typedef struct s_provider +{ + const char *cmd; + t_prov_fn fn; +} t_provider; + +/* One Makefile scan: where targets go, how deep the include is (0 = the + Makefile itself, 1 = a file it includes), and the depth of define. */ +typedef struct s_mk_scan +{ + t_vec *out; + int level; + int def; +} t_mk_scan; + +# define MK_MAX 4194304 + +/* The nanosecond mtime field is st_mtim on POSIX 2008, st_mtimespec on + Darwin (same split as builtin_test_posix.c). */ +# ifdef __APPLE__ +# define ST_MTIM st_mtimespec +# else +# define ST_MTIM st_mtim +# endif + +size_t pc_cmd_off(int start); +size_t prov_word(t_pv_line *p, const char **w); +size_t prov_base(const char *w, size_t n); +char **comp_provider_try(const char *text, int start, int end); +char **comp_make(t_shell *st, const char *text, t_pv_line *a); +char *rl_dup(const char *s); +t_vec *mk_load(t_shell *st); +void mk_cache_free(t_shell *st); +char *mk_slurp(const char *path, size_t *len); +void mk_scan(t_mk_scan *m, const char *buf, size_t len); +void mk_line(t_mk_scan *m, char *l); +int mk_blank(char c); +int mk_word_is(const char *l, const char *w); +void mk_rule(t_mk_scan *m, char *l); +void mk_add(t_vec *v, const char *w, size_t n); +int mk_directive(t_mk_scan *m, char *l); +int mk_pick(struct stat *sb, const char **name); +void mk_clear(t_make_cache *c); +int mk_same(t_make_cache *c, const char *name, struct stat *sb); +void mk_stamp(t_make_cache *c, struct stat *sb); + +#endif diff --git a/src/completion/complete_make_cache.c b/src/completion/complete_make_cache.c new file mode 100644 index 00000000..26cc3dc7 --- /dev/null +++ b/src/completion/complete_make_cache.c @@ -0,0 +1,74 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_cache.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* The one-entry Makefile cache: pick the file, decide whether the parsed + targets are still good, refill them when not. See complete_make.c for + the rules and the include limitation. */ + +#include "complete_make.h" + +/* GNU make's default order; the first REGULAR file wins (stat follows + symlinks), a directory of that name is skipped like a missing file. */ +int mk_pick(struct stat *sb, const char **name) +{ + static const char *cand[] = {"GNUmakefile", "makefile", "Makefile", + NULL}; + int i; + + i = -1; + while (cand[++i]) + { + if (stat(cand[i], sb) == 0 && S_ISREG(sb->st_mode)) + { + *name = cand[i]; + return (1); + } + } + return (0); +} + +void mk_clear(t_make_cache *c) +{ + size_t i; + + i = 0; + while (i < c->targets.len) + xfree(((char **)c->targets.ctx)[i++]); + xfree(c->targets.ctx); + xfree(c->name); + ft_bzero(c, sizeof(*c)); +} + +/* Teardown, from free_all_state. */ +void mk_cache_free(t_shell *st) +{ + mk_clear(&st->make_cache); +} + +int mk_same(t_make_cache *c, const char *name, struct stat *sb) +{ + return (c->name && !ft_strcmp(c->name, name) + && c->dev == (long long)sb->st_dev + && c->ino == (long long)sb->st_ino + && c->size == (long long)sb->st_size + && c->mtime_s == (long long)sb->ST_MTIM.tv_sec + && c->mtime_ns == (long long)sb->ST_MTIM.tv_nsec); +} + +void mk_stamp(t_make_cache *c, struct stat *sb) +{ + c->dev = (long long)sb->st_dev; + c->ino = (long long)sb->st_ino; + c->size = (long long)sb->st_size; + c->mtime_s = (long long)sb->ST_MTIM.tv_sec; + c->mtime_ns = (long long)sb->ST_MTIM.tv_nsec; +} diff --git a/src/completion/complete_make_cache2.c b/src/completion/complete_make_cache2.c new file mode 100644 index 00000000..c8fd5029 --- /dev/null +++ b/src/completion/complete_make_cache2.c @@ -0,0 +1,51 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_cache2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Refilling the Makefile cache. See complete_make_cache.c. */ + +#include "complete_make.h" + +/* Reread `name` into the cache; the old entry goes first so a file that + became unreadable leaves nothing stale behind. */ +static t_vec *mk_fill(t_make_cache *c, const char *name, struct stat *sb) +{ + char *buf; + size_t len; + t_mk_scan m; + + buf = mk_slurp(name, &len); + mk_clear(c); + if (!buf) + return (NULL); + c->name = ft_strdup(name); + mk_stamp(c, sb); + c->targets.elem_size = sizeof(char *); + m.out = &c->targets; + m.level = 0; + m.def = 0; + mk_scan(&m, buf, len); + xfree(buf); + return (&c->targets); +} + +/* The targets of the cwd's Makefile, or NULL when there is none. */ +t_vec *mk_load(t_shell *st) +{ + struct stat sb; + const char *name; + + if (!mk_pick(&sb, &name)) + return (NULL); + if (mk_same(&st->make_cache, name, &sb)) + return (&st->make_cache.targets); + return (mk_fill(&st->make_cache, name, &sb)); +} diff --git a/src/completion/complete_make_inc.c b/src/completion/complete_make_inc.c new file mode 100644 index 00000000..271b9d9a --- /dev/null +++ b/src/completion/complete_make_inc.c @@ -0,0 +1,89 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_inc.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Directives and includes of a Makefile line. See complete_make.c. */ + +#include "complete_make.h" + +/* Read one included file by name (literal names only) and scan it as + level 1, so what it includes is never followed. */ +static void mk_inc_one(t_mk_scan *m, const char *name) +{ + t_mk_scan sub; + char *buf; + size_t len; + size_t i; + + i = 0; + while (name[i]) + { + if (ft_strchr("$*?[]%~", name[i++])) + return ; + } + buf = mk_slurp(name, &len); + if (!buf) + return ; + sub = *m; + sub.level = 1; + sub.def = 0; + mk_scan(&sub, buf, len); + xfree(buf); +} + +/* include FILES...: each name in turn, only from the Makefile itself. */ +static void mk_include(t_mk_scan *m, char *l) +{ + char *w; + + while (m->level == 0 && *l) + { + while (mk_blank(*l)) + l++; + w = l; + while (*l && !mk_blank(*l)) + l++; + if (*l) + *l++ = '\0'; + if (*w) + mk_inc_one(m, w); + } +} + +static int mk_in_list(const char *l, const char **list) +{ + int i; + + i = -1; + while (list[++i]) + { + if (mk_word_is(l, list[i])) + return (i); + } + return (-1); +} + +/* 1 when the line was a directive (and is consumed). */ +int mk_directive(t_mk_scan *m, char *l) +{ + static const char *inc[] = {"include", "-include", "sinclude", NULL}; + static const char *dir[] = {"ifeq", "ifneq", "ifdef", "ifndef", + "else", "endif", "export", "unexport", "override", "vpath", NULL}; + int i; + + i = mk_in_list(l, inc); + if (i >= 0) + { + mk_include(m, l + ft_strlen(inc[i])); + return (1); + } + return (mk_in_list(l, dir) >= 0); +} diff --git a/src/completion/complete_make_line.c b/src/completion/complete_make_line.c new file mode 100644 index 00000000..432561e9 --- /dev/null +++ b/src/completion/complete_make_line.c @@ -0,0 +1,82 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_line.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* What a logical Makefile line is: a recipe, a define block, a directive, + an include, or (the default) a candidate rule. See complete_make.c. */ + +#include "complete_make.h" + +int mk_blank(char c) +{ + return (c == ' ' || c == '\t'); +} + +/* `w` is the first word of `l`: ended by a blank, '(' (ifeq(...)) or NUL. */ +int mk_word_is(const char *l, const char *w) +{ + size_t n; + + n = ft_strlen(w); + return (!ft_strncmp(l, w, n) + && (!l[n] || mk_blank(l[n]) || l[n] == '(')); +} + +/* Cut a '#' comment (a backslash escapes it), then trim both ends. */ +static char *mk_clean(char *l) +{ + size_t i; + + i = 0; + while (l[i] && !(l[i] == '#' && (i == 0 || l[i - 1] != '\\'))) + i++; + l[i] = '\0'; + while (i > 0 && mk_blank(l[i - 1])) + l[--i] = '\0'; + while (mk_blank(*l)) + l++; + return (l); +} + +/* define NAME ... endef is skipped whole (they nest). `override define` + and `export define` open one as well. */ +static int mk_define(t_mk_scan *m, char *l) +{ + if (m->def) + { + if (mk_word_is(l, "endef")) + m->def--; + else if (mk_word_is(l, "define")) + m->def++; + return (1); + } + while (mk_word_is(l, "override") || mk_word_is(l, "export")) + { + while (*l && !mk_blank(*l)) + l++; + while (mk_blank(*l)) + l++; + } + if (!mk_word_is(l, "define")) + return (0); + m->def = 1; + return (1); +} + +void mk_line(t_mk_scan *m, char *l) +{ + if (!m->def && l[0] == '\t') + return ; + l = mk_clean(l); + if (mk_define(m, l) || mk_directive(m, l)) + return ; + mk_rule(m, l); +} diff --git a/src/completion/complete_make_read.c b/src/completion/complete_make_read.c new file mode 100644 index 00000000..8e22a1fc --- /dev/null +++ b/src/completion/complete_make_read.c @@ -0,0 +1,126 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_read.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Reading a Makefile and cutting it into LOGICAL lines (a backslash- + newline becomes one space, as in GNU make; CRLF undone), then + handing each to mk_line. */ + +#include "complete_make.h" +#include +#include +#include + +static size_t mk_fill_buf(int fd, char *buf, size_t cap) +{ + size_t got; + ssize_t r; + + got = 0; + r = 1; + while (r > 0 && got < cap) + { + r = read(fd, buf + got, cap - got); + if (r > 0) + got += (size_t)r; + } + return (got); +} + +/* The whole file (first MK_MAX bytes), NUL-terminated, xmalloc'd; NULL when + it is not a readable regular file. Plain open/read/close: nothing in + libft reads a whole file by name. */ +char *mk_slurp(const char *path, size_t *len) +{ + struct stat sb; + int fd; + char *buf; + size_t cap; + + if (stat(path, &sb) != 0 || !S_ISREG(sb.st_mode)) + return (NULL); + fd = open(path, O_RDONLY | O_CLOEXEC); + if (fd < 0) + return (NULL); + cap = (size_t)sb.st_size; + if (cap > MK_MAX) + cap = MK_MAX; + buf = xmalloc(cap + 1); + if (buf) + { + *len = mk_fill_buf(fd, buf, cap); + buf[*len] = '\0'; + } + close(fd); + return (buf); +} + +/* One physical line appended to out; 1 when a backslash continues it. The + backslash must be unescaped (odd run) and followed by a real newline. */ +static int mk_phys(const char *b, size_t len, size_t *pos, char *out) +{ + size_t s; + size_t e; + size_t bs; + int nl; + + s = *pos; + e = s; + while (e < len && b[e] != '\n') + e++; + nl = (e < len); + *pos = e + (size_t)nl; + if (e > s && b[e - 1] == '\r') + e--; + ft_memcpy(out, b + s, e - s); + bs = 0; + while (bs < e - s && b[e - 1 - bs] == '\\') + bs++; + out[e - s] = '\0'; + if ((bs & 1) && nl) + out[e - s - 1] = ' '; + return ((bs & 1) && nl); +} + +/* Join physical lines into the logical one in `out`; returns its length. */ +static size_t mk_logical(const char *b, size_t len, size_t *pos, char *out) +{ + size_t n; + int more; + + n = 0; + more = 1; + while (more) + { + more = mk_phys(b, len, pos, out + n); + n += ft_strlen(out + n); + } + return (n); +} + +/* Walk the buffer line by line. A logical line is never longer than the + buffer, so one scratch allocation serves the whole walk. */ +void mk_scan(t_mk_scan *m, const char *buf, size_t len) +{ + char *line; + size_t pos; + + line = xmalloc(len + 1); + if (!line) + return ; + pos = 0; + while (pos < len) + { + mk_logical(buf, len, &pos, line); + mk_line(m, line); + } + xfree(line); +} diff --git a/src/completion/complete_make_rule.c b/src/completion/complete_make_rule.c new file mode 100644 index 00000000..ed4eb90a --- /dev/null +++ b/src/completion/complete_make_rule.c @@ -0,0 +1,98 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_rule.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* The rule half of the Makefile scan: find the colon, split the target + list, keep the names worth offering. See complete_make.c. */ + +#include "complete_make.h" + +/* `$(...)` and `${...}` starting at l[i]: the index after it. */ +static size_t mk_skip_ref(const char *l, size_t i) +{ + char o; + char c; + int d; + + o = l[i + 1]; + if (o != '(' && o != '{') + return (i + 1); + c = ')'; + if (o == '{') + c = '}'; + d = 0; + i++; + while (l[i]) + { + if (l[i] == o) + d++; + else if (l[i] == c && --d == 0) + return (i + 1); + i++; + } + return (i); +} + +/* The first ':' is a rule's unless it opens := ::= :::= (an assignment). */ +static long mk_colon_kind(const char *l, size_t i) +{ + if (l[i + 1] == '=') + return (-1); + if (l[i + 1] == ':' && (l[i + 2] == '=' + || (l[i + 2] == ':' && l[i + 3] == '='))) + return (-1); + return ((long)i); +} + +/* Index of the rule colon, or -1 when the line is not a rule: an '=' before + the first colon makes it an assignment (V = a:b, V ?= c:d, V += e:f). */ +static long mk_colon(const char *l) +{ + size_t i; + + i = 0; + while (l[i]) + { + if (l[i] == '$') + { + i = mk_skip_ref(l, i); + continue ; + } + if (l[i] == '=') + return (-1); + if (l[i] == ':') + return (mk_colon_kind(l, i)); + i++; + } + return (-1); +} + +/* l is "targets : ..." with the colon found: take every target. */ +void mk_rule(t_mk_scan *m, char *l) +{ + long c; + size_t n; + + c = mk_colon(l); + if (c < 0) + return ; + l[c] = '\0'; + while (*l) + { + while (mk_blank(*l)) + l++; + n = 0; + while (l[n] && !mk_blank(l[n])) + n++; + mk_add(m->out, l, n); + l += n; + } +} diff --git a/src/completion/complete_make_tgt.c b/src/completion/complete_make_tgt.c new file mode 100644 index 00000000..4470a649 --- /dev/null +++ b/src/completion/complete_make_tgt.c @@ -0,0 +1,58 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_make_tgt.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Which target names are worth offering, and the list they go in. See + complete_make.c. */ + +#include "complete_make.h" + +/* [A-Za-z0-9_./+-]+, not starting with '.', hence no %, no $, no .PHONY. */ +static int mk_ok(const char *w, size_t n) +{ + size_t i; + + if (n == 0 || w[0] == '.') + return (0); + i = 0; + while (i < n) + { + if (!ft_isalnum(w[i]) && !ft_strchr("_./+-", w[i])) + return (0); + i++; + } + return (1); +} + +/* Push a copy of w unless the name is already there (first seen wins). */ +static void mk_push(t_vec *v, const char *w, size_t n) +{ + char *dup; + size_t i; + + i = 0; + while (i < v->len) + { + dup = ((char **)v->ctx)[i++]; + if (ft_strlen(dup) == n && !ft_strncmp(dup, w, n)) + return ; + } + dup = ft_strndup((char *)w, n); + if (dup) + vec_push(v, &dup); +} + +/* Offer w[0..n) if it passes the name filter. */ +void mk_add(t_vec *v, const char *w, size_t n) +{ + if (mk_ok(w, n)) + mk_push(v, w, n); +} diff --git a/src/completion/complete_provider.c b/src/completion/complete_provider.c new file mode 100644 index 00000000..270b7e89 --- /dev/null +++ b/src/completion/complete_provider.c @@ -0,0 +1,99 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_provider.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Per-command argument completion providers. cmd_completion calls + comp_provider_try for an ARGUMENT word that no `complete` spec claimed; + the table in comp_provider_try maps a command name to the function that + knows its arguments, so a new provider is one row there and one function + elsewhere. A provider answers NULL for "not mine", which leaves + readline's filename completion in charge exactly as before. */ + +#include "complete_make.h" +#include + +/* Words that only introduce the real command (`sudo make`, `X=1 make`, + `env -i make`) are skipped so the provider sees the command that owns the + argument. An option after a wrapper is skipped too, but not its value: + `sudo -u root make` is not understood and keeps filename completion. */ +static int prov_skip(const char *w, size_t n, int first) +{ + static const char *wrap[] = {"sudo", "env", "command", "exec", + "nohup", "time", "nice", "doas", NULL}; + int i; + + if (!first && w[0] == '-') + return (1); + if ((ft_isalpha(w[0]) || w[0] == '_') && ft_memchr(w, '=', n)) + return (1); + i = -1; + while (wrap[++i]) + { + if (ft_strlen(wrap[i]) == n && !ft_strncmp(w, wrap[i], n)) + return (1); + } + return (0); +} + +/* The command word owning the argument being completed, past wrappers. */ +static size_t prov_cmd(t_pv_line *p, const char **w) +{ + size_t n; + int first; + + first = 1; + n = prov_word(p, w); + while (n && prov_skip(*w, n, first)) + { + first = 0; + n = prov_word(p, w); + } + return (n); +} + +/* The provider for the command named by w[0..n), or NULL. */ +static t_prov_fn prov_find(const char *w, size_t n) +{ + static const t_provider tab[] = {{"make", comp_make}, + {"gmake", comp_make}, {NULL, NULL}}; + int i; + + i = -1; + while (n && tab[++i].cmd) + { + if (ft_strlen(tab[i].cmd) == n && !ft_strncmp(w, tab[i].cmd, n)) + return (tab[i].fn); + } + return (NULL); +} + +/* Dispatch an argument word to the provider of its command, or NULL. */ +char **comp_provider_try(const char *text, int start, int end) +{ + t_shell *st; + t_pv_line p; + const char *w; + size_t n; + t_prov_fn fn; + + (void)end; + st = *zle_state_cell(); + if (!st || !rl_line_buffer || start <= 0) + return (NULL); + p.l = rl_line_buffer; + p.pos = pc_cmd_off(start); + p.end = (size_t)start; + n = prov_cmd(&p, &w); + fn = prov_find(w + prov_base(w, n), n - prov_base(w, n)); + if (!fn) + return (NULL); + return (fn(st, text, &p)); +} diff --git a/src/completion/complete_provider2.c b/src/completion/complete_provider2.c new file mode 100644 index 00000000..87e051c8 --- /dev/null +++ b/src/completion/complete_provider2.c @@ -0,0 +1,41 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* complete_provider2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/16 18:30:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/16 18:30:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* Word-walking helpers shared by the completion providers. */ + +#include "complete_make.h" + +/* The next blank-separated word before the cursor word, 0 at the end. */ +size_t prov_word(t_pv_line *p, const char **w) +{ + size_t n; + + while (p->pos < p->end && mk_blank(p->l[p->pos])) + p->pos++; + *w = p->l + p->pos; + n = 0; + while (p->pos + n < p->end && !mk_blank(p->l[p->pos + n])) + n++; + p->pos += n; + return (n); +} + +/* Offset of the basename in w[0..n), so /usr/bin/make is make. */ +size_t prov_base(const char *w, size_t n) +{ + size_t i; + + i = n; + while (i > 0 && w[i - 1] != '/') + i--; + return (i); +} diff --git a/src/completion/completion.c b/src/completion/completion.c index 12914abb..f9025ba4 100644 --- a/src/completion/completion.c +++ b/src/completion/completion.c @@ -33,6 +33,8 @@ char **complete_exec_files(const char *text, int start, int end); /* complete_cmdpos.c -- "is this word a command name?", which is not the same question as "is it at column 0". */ int is_cmd_word(int start); +/* complete_provider.c -- per-command argument providers (make, ...). */ +char **comp_provider_try(const char *text, int start, int end); /* readline OWNS every string a generator hands back, and releases it with libc free(). Our xmalloc family compiles to ft_malloc on a SAFE=0 build, @@ -95,7 +97,11 @@ char *rl_dup_dollar(const char *name, size_t len) alone when there is none, so every command without a registration keeps the filename completion it has always had -- which is the case that would have been broken silently, and everywhere, by a dispatcher that claimed - arguments unconditionally. */ + arguments unconditionally. + + Only when no spec exists does a built-in provider get a turn (make + targets, complete_make.c), so a user's `complete` always wins; a provider + that has nothing to offer answers NULL and the filenames come back. */ static char **cmd_completion(const char *text, int start, int end) { char **spec; @@ -110,7 +116,10 @@ static char **cmd_completion(const char *text, int start, int end) spec = progcomp_try(text, start, end); if (rl_attempted_completion_over) return (spec); - return (NULL); + spec = comp_provider_try(text, start, end); + if (spec) + rl_attempted_completion_over = 1; + return (spec); } rl_attempted_completion_over = 1; if (ft_strchr(text, '/')) diff --git a/src/completion/progcomp2.c b/src/completion/progcomp2.c index a506896b..3a7743b3 100644 --- a/src/completion/progcomp2.c +++ b/src/completion/progcomp2.c @@ -37,7 +37,7 @@ static bool pc_blank(char c) separator set is the one is_cmd_word uses: POSIX XCU 2.9 makes the word after ; | & ( { ` or a newline a new command, so `ls | grep ` must look up grep and not ls. */ -static size_t pc_cmd_off(int start) +size_t pc_cmd_off(int start) { int i; diff --git a/src/completion/progcomp_private.h b/src/completion/progcomp_private.h index c115878b..4ed0e7bc 100644 --- a/src/completion/progcomp_private.h +++ b/src/completion/progcomp_private.h @@ -31,6 +31,7 @@ void pc_push(const char *s, int n); /* What the line under the cursor says. `start` is readline's offset of the word being completed. */ +size_t pc_cmd_off(int start); char *pc_cmd_word(int start); char *pc_prev_word(int start); diff --git a/src/editing/bind_lines.c b/src/editing/bind_lines.c new file mode 100644 index 00000000..208ae325 --- /dev/null +++ b/src/editing/bind_lines.c @@ -0,0 +1,72 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* bind_lines.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "shell.h" +#include "libft.h" +#include "zle.h" + +/* What `bind` asked readline to do, kept until readline can take it. +** +** The same timing problem `bindkey` has (zle_bind.c): an rc file runs +** before readline is initialised and before the editing mode installs the +** keymap a binding would land in, which replaces whatever was there. So +** each request is recorded here and replayed, in order, right before a +** line is read (rl_preinit) -- and replayed again from the start whenever +** the editing mode changes, since the new keymap is empty of it. +** +** One entry per request: `op` is 0 for a readline line +** (`"\e[A": history-search-backward`, `set completion-ignore-case on`, +** `$include FILE` for -f), 'u' to unbind a function, 'r' to unbind a key +** sequence. `map` is -m's keymap, NULL for the current one. */ + +t_vec *bind_lines(void) +{ + static t_vec v; + + if (!v.elem_size) + { + vec_init(&v); + v.elem_size = sizeof(t_bind_line); + } + return (&v); +} + +void bind_line_add(char op, const char *map, const char *line) +{ + t_bind_line b; + + b.op = op; + b.map = NULL; + if (map) + b.map = ft_strdup(map); + b.line = ft_strdup(line); + if (!b.line) + return (xfree(b.map)); + vec_push(bind_lines(), &b); +} + +void bind_lines_free(void) +{ + t_bind_line *a; + size_t i; + + a = (t_bind_line *)bind_lines()->ctx; + i = 0; + while (i < bind_lines()->len) + { + xfree(a[i].map); + xfree(a[i].line); + i++; + } + xfree(bind_lines()->ctx); + *bind_lines() = (t_vec){.elem_size = sizeof(t_bind_line)}; +} diff --git a/src/editing/zle_bind.c b/src/editing/zle_bind.c index b0767062..17809c17 100644 --- a/src/editing/zle_bind.c +++ b/src/editing/zle_bind.c @@ -41,29 +41,32 @@ t_vec *zle_binds(void) return (&v); } -/* Record one binding. A repeat of the same sequence REPLACES: a plugin that - binds the same key in three keymaps (emacs, vicmd, viins -- which is what - oh-my-zsh's sudo does) must not leave three entries fighting over it, and - readline's keymaps are not per-editing-mode here. */ +/* Record one binding, the sequence in readline's spelling (zle_keyseq). + A repeat of the same sequence REPLACES: a plugin that binds the same key + in three keymaps (emacs, vicmd, viins -- which is what oh-my-zsh's sudo + does) must not leave three entries fighting over it, and readline's + keymaps are not per-editing-mode here. */ void zle_bind_add(const char *seq, const char *widget) { t_zle_bind b; t_zle_bind *a; size_t i; + b.seq = zle_keyseq(seq); + if (!b.seq) + return ; a = (t_zle_bind *)zle_binds()->ctx; - i = 0; - while (i < zle_binds()->len) + i = -1; + while (++i < zle_binds()->len) { - if (!ft_strcmp(a[i].seq, seq)) + if (!ft_strcmp(a[i].seq, b.seq)) { + xfree(b.seq); xfree(a[i].widget); a[i].widget = ft_strdup(widget); return ; } - i++; } - b.seq = ft_strdup(seq); b.raw = NULL; b.widget = ft_strdup(widget); vec_push(zle_binds(), &b); diff --git a/src/editing/zle_keyseq.c b/src/editing/zle_keyseq.c new file mode 100644 index 00000000..fe6de937 --- /dev/null +++ b/src/editing/zle_keyseq.c @@ -0,0 +1,74 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* zle_keyseq.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "shell.h" +#include "libft.h" +#include "zle.h" + +/* zsh's spelling of a key sequence, in readline's. +** +** The two share their backslash escapes (\e \C-x \M-x \n \t \NNN ...). +** What only zsh has is the caret -- `^X` a control character, `^[` the +** escape, `^?` DEL -- and \E. Handed to readline as written, +** +** bindkey '^[[A' history-beginning-search-backward +** +** bound the four characters ^ [ [ A: the arrow never reached it, and +** neither did any other key a zsh config spells that way (#136). */ + +/* One two-byte token: a backslash escape passes through whole (so the + `\C` of `\C-x` stays with its backslash), a caret becomes readline's + control or escape. */ +static void zle_keyseq_pair(t_string *out, char a, char b) +{ + if (a == '\\' && b == 'E') + vec_push_str(out, "\\e"); + else if (a == '\\') + { + vec_push_char(out, a); + vec_push_char(out, b); + } + else if (b == '[') + vec_push_str(out, "\\e"); + else if (b == '?') + vec_push_str(out, "\\C-?"); + else + { + vec_push_str(out, "\\C-"); + vec_push_char(out, (char)ft_tolower(b)); + } +} + +/* Owned; NULL only when out of memory. A caret or backslash with nothing + after it is itself. */ +char *zle_keyseq(const char *s) +{ + t_string out; + size_t i; + + vec_init(&out); + out.elem_size = 1; + i = 0; + while (s[i]) + { + if (s[i + 1] && (s[i] == '\\' || s[i] == '^')) + { + zle_keyseq_pair(&out, s[i], s[i + 1]); + i += 2; + } + else + vec_push_nstr(&out, s + i++, 1); + } + if (!vec_push_nstr(&out, "", 0)) + return (xfree(out.ctx), NULL); + return ((char *)out.ctx); +} diff --git a/src/editing/zle_params.c b/src/editing/zle_params.c index d66a5648..0b9244de 100644 --- a/src/editing/zle_params.c +++ b/src/editing/zle_params.c @@ -27,7 +27,7 @@ static const char *zle_param_name(int i) { static const char *names[] = {"BUFFER", "LBUFFER", "RBUFFER", - "CURSOR"}; + "CURSOR", "POSTDISPLAY"}; return (names[i]); } diff --git a/src/editing/zle_widget.c b/src/editing/zle_widget.c index baa01d91..1751a3ec 100644 --- a/src/editing/zle_widget.c +++ b/src/editing/zle_widget.c @@ -87,6 +87,7 @@ void zle_widget_add(const char *name, const char *fn) } w.name = ft_strdup(name); w.fn = ft_strdup(fn); + w.bashx = false; vec_push(zle_widgets(), &w); } diff --git a/src/environment/README.md b/src/environment/README.md index 9180677b..3a1aab89 100644 --- a/src/environment/README.md +++ b/src/environment/README.md @@ -127,7 +127,10 @@ Resolution order: `expand_special_1` (`$-` from `build_flagstr`, `$0` inside a zsh function via `zsh_arg_zero`, `$#` from `pos.cnt_str`); then `expand_special_dyn` (`expand2.c`): `$LINENO`, `$RANDOM` (session PRNG - masked to 15 bits), `$SECONDS`, `$EPOCHSECONDS`, all formatted into + masked to 15 bits), `$SECONDS`, `$EPOCHSECONDS`, and after them + `expand_special_dyn_time` (`expand_dyn_time.c`): `$EPOCHREALTIME` + (seconds.microseconds) and `$SRANDOM` (32 bits from `getentropy`, never + from the seedable `$RANDOM` generator), all formatted into `state->linebuf`. A user assignment to these names does not shadow them (accepted divergence). 2. a nameref (`attr_target`) -> recurse on the target; diff --git a/src/environment/expand2.c b/src/environment/expand2.c index e02c9107..6a33c3e4 100644 --- a/src/environment/expand2.c +++ b/src/environment/expand2.c @@ -19,25 +19,32 @@ #include "helpers.h" #include "sh_input.h" #include "ft_stdlib.h" +#include "sh_alias.h" #include bool is_readonly_var(t_shell *state, const char *key); void exit_clean(t_shell *state, int code); int shell_fatal_status(t_shell *state); char *lineno_str(t_shell *state); +char *expand_special_dyn_time(t_shell *state, char *key, int len); int tok_lineno(t_shell *state); /* Dynamic special variables computed at expansion time, second tier of expand_special: $LINENO, plus the bash-isms $RANDOM (0..32767 from the session PRNG), $SECONDS (whole seconds since shell start) and - $EPOCHSECONDS. All borrow state->linebuf like lineno_str. A user + $EPOCHSECONDS; $EPOCHREALTIME and $SRANDOM are the next tier + (expand_dyn_time.c). All borrow state->linebuf like lineno_str. A user assignment to these names does NOT shadow them (bash lets SECONDS=N - re-base the counter; accepted divergence). NULL = not one of ours. */ + re-base the counter; accepted divergence). NULL = not one of ours. + In the zsh dialect `aliases` is the alias table (alias_view.c). */ char *expand_special_dyn(t_shell *state, char *key, int len) { if (len == 6 && ft_strncmp(key, "LINENO", 6) == 0) return (lineno_str(state)); + if (len == 7 && key[0] == 'a' && zsh_mode(state) + && ft_strncmp(key, "aliases", 7) == 0) + return (zsh_aliases_view(state)); if (len == 6 && ft_strncmp(key, "RANDOM", 6) == 0) { snprintf(state->linebuf, sizeof(state->linebuf), "%u", @@ -56,7 +63,7 @@ char *expand_special_dyn(t_shell *state, char *key, int len) (long long)time(NULL)); return (state->linebuf); } - return (NULL); + return (expand_special_dyn_time(state, key, len)); } /* $LINENO: the line number currently being read/executed (input-line diff --git a/src/environment/expand_dyn_time.c b/src/environment/expand_dyn_time.c new file mode 100644 index 00000000..bcbd67d8 --- /dev/null +++ b/src/environment/expand_dyn_time.c @@ -0,0 +1,54 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* expand_dyn_time.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: marvin +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/26 00:10:00 by marvin #+# #+# */ +/* Updated: 2026/09/26 00:10:00 by marvin ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "env.h" +#include "shell.h" +#include "ft_stdlib.h" +#include +#include +#include +#include + +/* Third tier of the dynamic variables, after expand_special_dyn. + + $EPOCHREALTIME is the time since the epoch with microseconds, as bash + prints it: seconds, a '.', six digits. It is what a prompt timer reads + without forking `date +%s%N`; $EPOCHSECONDS has one-second resolution. + + $SRANDOM is 32 random bits per read, from the kernel (getentropy), not + from the $RANDOM generator: bash promises that it cannot be seeded, so + RANDOM=42 must not make it predictable. getentropy only fails on a + kernel without the call; then the session PRNG, all 32 bits, is the + fallback, as bash falls back to its own generator. + + Both borrow state->linebuf like $SECONDS. NULL = not one of ours. */ +char *expand_special_dyn_time(t_shell *state, char *key, int len) +{ + struct timespec ts; + uint32_t r; + + if (len == 13 && ft_strncmp(key, "EPOCHREALTIME", 13) == 0) + { + clock_gettime(CLOCK_REALTIME, &ts); + snprintf(state->linebuf, sizeof(state->linebuf), "%lld.%06ld", + (long long)ts.tv_sec, ts.tv_nsec / 1000); + return (state->linebuf); + } + if (len == 7 && ft_strncmp(key, "SRANDOM", 7) == 0) + { + if (getentropy(&r, sizeof(r)) != 0) + r = random_uint32(&state->prng); + snprintf(state->linebuf, sizeof(state->linebuf), "%u", r); + return (state->linebuf); + } + return (NULL); +} diff --git a/src/environment/utils.c b/src/environment/utils.c index 1ec262dd..53ca94e2 100644 --- a/src/environment/utils.c +++ b/src/environment/utils.c @@ -107,8 +107,9 @@ char *env_to_str(t_env *e) /* Build the NULL-terminated envp[] for execve, including only entries where exported==true. Shell-local variables (exported=false) are - intentionally hidden from child processes. Caller owns the array and - each element. */ + intentionally hidden from child processes, and so is a name exported + before it has a value (`export X` alone): bash passes nothing for it, + not "X=". Caller owns the array and each element. */ char **get_envp(t_shell *state, char *exe_path) { char **ret; @@ -123,7 +124,7 @@ char **get_envp(t_shell *state, char *exe_path) while (++i < state->env.len) { e = &((t_env *)state->env.ctx)[i]; - if (e->exported && !arr_is(e->value)) + if (e->exported && e->value && !arr_is(e->value)) ret[j++] = env_to_str(e); } return (ret); diff --git a/src/environment/utils2.c b/src/environment/utils2.c index fb8f2a50..3578b994 100644 --- a/src/environment/utils2.c +++ b/src/environment/utils2.c @@ -25,6 +25,9 @@ void update_pwd_vars(t_shell *state) pwd = env_get(&state->env, PWD_NAME); if (pwd == NULL) try_unset(state, OLDPWD_NAME); + else if (!pwd->value) + env_set(&state->env, env_create(ft_strdup(OLDPWD_NAME), NULL, + pwd->exported)); else env_set(&state->env, env_create(ft_strdup(OLDPWD_NAME), ft_strdup(pwd->value), pwd->exported)); diff --git a/src/execution/call_frames.c b/src/execution/call_frames.c index 2f47e34b..fda70ba0 100644 --- a/src/execution/call_frames.c +++ b/src/execution/call_frames.c @@ -46,8 +46,10 @@ ** as FUNCNAME plus a "main" entry. Here there is exactly one entry per live ** frame, innermost first. The element that matters -- [0] -- agrees with ** bash in every case the idiom above cares about. BASH_LINENO is not -** provided: tok_lineno() cannot resolve lines inside sourced text today -** (see src/execution/exec_lineno.c:97), so it would be a confident lie. */ +** provided: tok_lineno() resolves lines in the script and in sourced files +** (exec_where.c) but not inside a function body, whose tokens are copies +** with no position -- the one place BASH_LINENO is read -- so it would be +** a confident lie. */ /* Collect one field of the stack, innermost first, into a fresh array value. An empty stack yields an EMPTY array rather than nothing: bash leaves diff --git a/src/execution/exe_error.c b/src/execution/exe_error.c index 59d9d0f8..94e4d720 100644 --- a/src/execution/exe_error.c +++ b/src/execution/exe_error.c @@ -57,20 +57,27 @@ int no_such_file_or_dir(t_shell *state, return (COMMAND_NOT_FOUND); } -static int errex_code(t_shell *state, - char *cmd_name, - char **path_of_exe, - int ex) +/* Report a direct path's failure with the errno the caller left -- the one + that says WHY -- and let it decide the status as bash does: 127 when the + file is not there, 126 for everything else. It used to force ENOENT, so + a file that exists but is not executable, a directory, `./file/x` and a + symlink loop all read "No such file or directory" (#133: `./life` was + there), and the last two also came back 127 instead of 126. */ +static int errex_code(t_shell *state, char *cmd_name, char **path_of_exe) { - errno = ENOENT; + int ex; + + ex = EXE_PERM_DENIED; + if (errno == ENOENT) + ex = COMMAND_NOT_FOUND; err_1_errno(state, cmd_name); xfree(*path_of_exe); *path_of_exe = NULL; return (ex); } -/* Validate a command given as a direct path (contains '/'). A missing - file -> COMMAND_NOT_FOUND; a directory -> EXE_PERM_DENIED. Shell +/* Validate a command given as a direct path (contains '/'): stat's own + errno, EISDIR, or access's EACCES, each reported for what it is. Shell script extensions (.sh/.hell/.hellish) only need to be readable, not executable (the shell will interpret them directly), so we skip the X_OK check for those. Everything else requires X_OK (POSIX). */ @@ -81,18 +88,15 @@ int handle_direct_path_error(t_shell *state, char *cmd_name, size_t len; if (stat(*path_of_exe, &st) == -1) - return (errex_code(state, cmd_name, path_of_exe, COMMAND_NOT_FOUND)); + return (errex_code(state, cmd_name, path_of_exe)); if (S_ISDIR(st.st_mode)) - return (errex_code(state, cmd_name, path_of_exe, EXE_PERM_DENIED)); + return (errno = EISDIR, errex_code(state, cmd_name, path_of_exe)); len = ft_strlen(*path_of_exe); if ((len >= 3 && ft_strcmp(*path_of_exe + len - 3, ".sh") == 0) || (len >= 5 && ft_strcmp(*path_of_exe + len - 5, ".hell") == 0) || (len >= 8 && ft_strcmp(*path_of_exe + len - 8, ".hellish") == 0)) return (0); if (access(*path_of_exe, X_OK) != 0) - { - return (errex_code(state, cmd_name, path_of_exe, - EXE_PERM_DENIED)); - } + return (errex_code(state, cmd_name, path_of_exe)); return (0); } diff --git a/src/execution/exec_ctx.c b/src/execution/exec_ctx.c new file mode 100644 index 00000000..1bb54771 --- /dev/null +++ b/src/execution/exec_ctx.c @@ -0,0 +1,87 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* exec_ctx.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 19:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 19:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "execution_private.h" + +/* ctx, the "script: line N" every runtime error starts with, kept pointing + at the executing command (exec_where.c decides where that is). It is + rebuilt only when the file changes; a new line in the same file is + written over the old number in place. */ + +/* Write `line` over the digits at the end of ctx. ctx_rebuild left room + for any int, so a loop body moving between its lines costs this and no + allocation -- ctx changes on every command there. */ +static void ctx_renumber(t_shell *state, int line) +{ + char digits[16]; + int n; + long v; + + v = line; + if (v < 0) + v = 0; + n = 0; + while (n == 0 || v) + { + digits[n++] = (char)('0' + v % 10); + v /= 10; + } + v = (long)state->ctx_num; + while (n > 0) + state->ctx[v++] = digits[--n]; + state->ctx[v] = '\0'; +} + +/* bash's prefixes: "FILE: line N" from a script or -c, "bash: FILE: line + N" at a prompt; "NAME: line N" for the script or -c text itself, and + the bare name at a prompt, which has no line to give. */ +static void ctx_rebuild(t_shell *state, const char *src) +{ + char *pre; + size_t n; + + if (src && sh_interactive(state)) + pre = ft_asprintf("%s: %s: line ", state->dft_ctx, src); + else if (src) + pre = ft_asprintf("%s: line ", src); + else if (state->rl.should_update_ctx) + pre = ft_asprintf("%s: line ", state->dft_ctx); + else + pre = ft_strdup(state->dft_ctx); + if (!pre) + return ; + xfree(state->ctx); + state->ctx_num = 0; + state->ctx = pre; + if (!src && !state->rl.should_update_ctx) + return ; + n = ft_strlen(pre); + state->ctx = xmalloc(n + 16); + if (!state->ctx) + return ((void)(state->ctx = pre)); + ft_memcpy(state->ctx, pre, n); + state->ctx_num = n; + xfree(pre); +} + +/* Point ctx at `line` of `src` (NULL: the script or -c text itself). */ +void ctx_follow(t_shell *state, const char *src, int line) +{ + if (src == state->ctx_src && line == state->ctx_line) + return ; + if (src != state->ctx_src || !state->ctx_num) + ctx_rebuild(state, src); + if (state->ctx_num) + ctx_renumber(state, line); + state->ctx_src = src; + state->ctx_line = line; +} diff --git a/src/execution/exec_diag.c b/src/execution/exec_diag.c new file mode 100644 index 00000000..6b8d4cc9 --- /dev/null +++ b/src/execution/exec_diag.c @@ -0,0 +1,134 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* exec_diag.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 18:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 18:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "execution_private.h" +#include "sys.h" + +/* What execve left behind, said the way bash's shell_execve says it (#133). + Only EACCES used to be reported: a program whose loader or `#!` + interpreter is missing -- execve's ENOENT on a file that IS there -- + printed nothing at all and exited 127, and a binary for another machine + was handed to /bin/sh, which printed its own confusion about line 1. + All of this runs in the forked child, after execve has already failed. */ + +/* The first line of a sample, the first two when it opens with "#!", holds + a NUL: bash's check_binary_file. An ELF header is binary outright. */ +static bool sample_is_binary(const unsigned char *s, ssize_t n) +{ + ssize_t i; + int lines; + + if (n >= 4 && s[0] == 0x7f && s[1] == 'E' && s[2] == 'L' && s[3] == 'F') + return (true); + lines = 1; + if (n >= 2 && s[0] == '#' && s[1] == '!') + lines = 2; + i = -1; + while (++i < n) + { + if (s[i] == '\n' && --lines == 0) + return (false); + if (s[i] == '\0') + return (true); + } + return (false); +} + +/* The 80 bytes bash judges a file by. Unreadable or empty is not binary: + an empty executable is a script that does nothing, status 0. */ +bool exec_file_is_binary(const char *path) +{ + unsigned char buf[EXEC_SAMPLE_LEN]; + ssize_t n; + int fd; + + fd = open(path, O_RDONLY); + if (fd < 0) + return (false); + n = read(fd, buf, sizeof(buf)); + close(fd); + return (n > 0 && sample_is_binary(buf, n)); +} + +/* The interpreter a NUL-terminated "#!" sample names, cut out in place: + bash's getinterp, a trailing CR spelled ^M -- the classic CRLF script. + The caller's NUL sits at least one byte past the name, room for the M. */ +static char *sample_interp(char *buf) +{ + int i; + int start; + + i = 2; + while (buf[i] == ' ' || buf[i] == '\t') + i++; + start = i; + while (buf[i] && buf[i] != '\n' && buf[i] != ' ' && buf[i] != '\t') + i++; + if (i > start && buf[i - 1] == '\r') + { + buf[i - 1] = '^'; + buf[i++] = 'M'; + } + buf[i] = '\0'; + return (buf + start); +} + +/* A "#!" script the kernel would not start: bash names the interpreter. + The sample's last byte is dropped as bash drops it. bash says this one + through sys_error, which never adds "line N:" -- hence the bare dft_ctx + where every other message here has ctx. */ +static bool bad_interpreter(t_shell *state, char *path, int err) +{ + char buf[EXEC_SAMPLE_LEN + 1]; + ssize_t n; + int fd; + + fd = open(path, O_RDONLY); + if (fd < 0) + return (false); + n = read(fd, buf, EXEC_SAMPLE_LEN); + close(fd); + if (n <= 2 || buf[0] != '#' || buf[1] != '!') + return (false); + buf[n - 1] = '\0'; + ft_eprintf("%s: %s: %s: bad interpreter: %s\n", state->dft_ctx, path, + sample_interp(buf), strerror(err)); + return (true); +} + +/* bash 5.3's order, and its statuses -- the pinned oracle's, which moved + the `#!` check ahead of ENOENT: a script whose interpreter is missing + is a "bad interpreter" (126), and only a file with no `#!` whose loader + is missing is "required file not found" (127, the one ENOENT status). + err == EXEC_ERR_BINARY is our own: ENOEXEC on a file the sample says + is binary, which must never reach the /bin/sh fallback. */ +int exec_failure_report(t_shell *state, char *path, int err) +{ + struct stat st; + + if (err == EXEC_ERR_BINARY) + return (ft_eprintf("%s: %s: cannot execute binary file: %s\n", + state->ctx, path, strerror(ENOEXEC)), EXIT_CMD_NOT_EXEC); + if (stat(path, &st) == 0 && S_ISDIR(st.st_mode)) + err_2(state, path, strerror(EISDIR)); + else if (access(path, X_OK) != 0 || err == E2BIG || err == ENOMEM) + err_2(state, path, strerror(err)); + else if (bad_interpreter(state, path, err)) + return (EXIT_CMD_NOT_EXEC); + else if (err == ENOENT) + err_2(state, path, "cannot execute: required file not found"); + else + err_2(state, path, strerror(err)); + if (err == ENOENT) + return (EXIT_CMD_NOT_FOUND); + return (EXIT_CMD_NOT_EXEC); +} diff --git a/src/execution/exec_lineno.c b/src/execution/exec_lineno.c index fc47a624..0f139773 100644 --- a/src/execution/exec_lineno.c +++ b/src/execution/exec_lineno.c @@ -39,76 +39,84 @@ static const char *first_tok_start(t_ast_node *node) /* With batched input delivery, rl.line points at the END of the batch, so $LINENO can no longer be read off the line counter. Instead we remember the executing command's first source token here (a borrowed pointer into - alias_exp); lineno_str() lazily turns that into a line number by counting - newlines up to its offset — only when a $LINENO expansion actually asks. - Tokens that don't point into the cycle buffer (heap-copied function - bodies, eval/cmdsub re-lexes) are skipped, so those keep the previous - command's line — the call site — which is what bash reports for eval. */ + the text it was lexed from), and where_follow moves the error context + there: a batch's line from the cycle buffer, a sourced file's from its + chunk. Tokens that point into neither (heap-copied function bodies, + eval/cmdsub re-lexes) are skipped, so those keep the previous command's + line -- the call site -- which is what bash reports for eval. */ void note_cmd_lineno(t_shell *state, t_ast_node *node) { const char *start; start = first_tok_start(node); - if (start) - state->rl.ln_tok = start; + if (!start) + return ; + state->rl.ln_tok = start; + where_follow(state, start); } -/* Line number of a token inside hd_stripped (the cycle buffer with heredoc - body lines removed). Before the first heredoc, stripped text matches the - source byte-for-byte, so the newline count is exact. Past the end of the - first heredoc's operator line, add back every removed body line (packed - in hd_src) — exact for the common one-heredoc cycle, an approximation - when several heredocs interleave with commands. */ -static int hd_stripped_line(t_shell *state, const char *tok) +/* The text this cycle's tokens slice, when it holds `tok`: alias_exp + normally, hd_stripped when the cycle held heredocs -- whose bodies left + an empty line apiece behind, so both count lines as the source does. */ +static const char *cycle_text(t_shell *state, const char *tok) { - const char *s; - size_t off; - size_t cut; - int line; + const char *base; - s = state->hd_stripped; - off = tok - s; - line = state->rl.cycle_line0 + nl_count(s, off); - cut = 0; - while (s[cut] && !(s[cut] == '<' && s[cut + 1] == '<')) - cut++; - while (s[cut] && s[cut] != '\n') - cut++; - if (off > cut && state->hd_src) - line += nl_count(state->hd_src, ft_strlen(state->hd_src)); - return (line); + base = (const char *)state->alias_exp.ctx; + if (base && tok >= base && tok < base + state->alias_exp.len) + return (base); + base = state->hd_stripped; + if (base && tok >= base && tok < base + state->hd_stripped_len) + return (base); + return (NULL); } -/* Resolve the executing command's line number under batched input - delivery: rl.line points at the END of the delivered batch, so the real - line is recovered from the command's first token offset into the cycle - buffer (cycle start line + newlines before the token). The (ln_ptr, - ln_val) pair memoises the scan so a loop body re-reading $LINENO costs a - pointer compare, not a rescan. Tokens live in alias_exp normally, or in - hd_stripped when the cycle contained heredocs; anything else (function - bodies, eval re-lexes) keeps the last resolved line — the call site. */ -int tok_lineno(t_shell *state) +/* The line of `tok` in this cycle's text, or -1 when it is not there. + The (ln_ptr, ln_val) pair memoises the last answer, and a token further + on (or back) in the same text resolves from it, so a batch run top to + bottom is scanned once and a loop body re-reading $LINENO costs a scan + of that body, not of everything before it. */ +int cycle_lineno(t_shell *state, const char *tok) { const char *base; + const char *m; + + m = state->rl.ln_ptr; + if (m && tok == m) + return (state->rl.ln_val); + base = cycle_text(state, tok); + if (!base) + return (-1); + if (m && cycle_text(state, m) == base) + state->rl.ln_val += nl_between(m, tok); + else + state->rl.ln_val = state->rl.cycle_line0 + nl_count(base, + (size_t)(tok - base)); + state->rl.ln_ptr = tok; + return (state->rl.ln_val); +} + +/* $LINENO: the executing command's line -- in the sourced file when it + comes from one, else in the batch (the reader's counter at a prompt, + whose line counter keeps its per-entry semantics). Anything else -- + function bodies, eval re-lexes -- keeps the last resolved line: the + call site. */ +int tok_lineno(t_shell *state) +{ const char *tok; int line; + tok = state->rl.ln_tok; + if (tok && srcpos_has(&state->err_pos, tok)) + return (srcpos_line(&state->err_pos, state->err_line, tok)); if (!state->rl.tok_line) return (state->rl.line); - tok = state->rl.ln_tok; - if (tok && tok == state->rl.ln_ptr) + line = -1; + if (tok) + line = cycle_lineno(state, tok); + if (line >= 0) + return (line); + if (state->rl.ln_ptr) return (state->rl.ln_val); - base = (const char *)state->alias_exp.ctx; - if (tok && base && tok >= base && tok < base + state->alias_exp.len) - line = state->rl.cycle_line0 + nl_count(base, tok - base); - else if (tok && state->hd_stripped && tok >= state->hd_stripped - && tok < state->hd_stripped + ft_strlen(state->hd_stripped)) - line = hd_stripped_line(state, tok); - else if (state->rl.ln_ptr) - return (state->rl.ln_val); - else - return (state->rl.line); - state->rl.ln_ptr = tok; - state->rl.ln_val = line; - return (line); + return (state->rl.line); } diff --git a/src/execution/exec_string.c b/src/execution/exec_string.c index 8e53e3dc..2fecbfd6 100644 --- a/src/execution/exec_string.c +++ b/src/execution/exec_string.c @@ -45,12 +45,25 @@ void skip_delimiters(t_deque_tok *tt) as-is, under whatever hd_src the caller already installed. Stripping happens on the RAW text now that alias splicing is per-chunk (#105): bodies are removed before any splice, so alias words inside a heredoc - body are never expanded -- which is also what bash does. */ + body are never expanded -- which is also what bash does. + A string whose heredoc found no body in its own text gets an EMPTY + stream, so the heredoc ends at once with the "delimited by end-of-file" + warning. The caller's stream was never this string's to read: `eval + 'cat <hd_from_caller; + state->hd_from_caller = false; stripped = NULL; if (ft_strnstr(str, "<<", ft_strlen(str)) && split_heredocs(str, &stripped, bodies)) @@ -59,10 +72,15 @@ static int exec_split_heredocs(t_shell *state, char *str, char **bodies) state->hd_pos = 0; status = exec_chunks(state, stripped); xfree(stripped); + return (status); } - else - status = exec_chunks(state, str); - return (status); + if (!from_caller) + { + none[0] = '\0'; + state->hd_src = none; + state->hd_pos = 0; + } + return (exec_chunks(state, str)); } /* Execute the string, routing any heredoc bodies aside first. Alias @@ -80,9 +98,9 @@ static int exec_split_heredocs(t_shell *state, char *str, char **bodies) fresh-install pty test catches under ASan. The old single-pass code was immune only by accident: its up-front alias splice WAS the copy. - The two entry points below differ only in state->err_src, the file a - parse error is reported against (error_where.c). It is saved and - restored around every run so an eval inside a sourced file reports + The two entry points below differ only in state->err_src, the file an + error is reported against (error_where.c, exec_where.c). It is saved + and restored around every run so an eval inside a sourced file reports against ITS text rather than the file's line count. */ static int exec_string_own(t_shell *state, char *str) { @@ -120,19 +138,14 @@ int exec_string(t_shell *state, char *str) return (status); } -/* `source FILE` and the rc loader: parse errors name FILE and the line. */ +/* `source FILE` and the rc loader: errors name FILE and the line. */ int exec_file_string(t_shell *state, char *str, const char *src) { - const char *saved_src; - int saved_line; - int status; + t_where saved; + int status; - saved_src = state->err_src; - saved_line = state->err_line; - state->err_src = src; - state->err_line = 1; + where_push(state, &saved, src); status = exec_string_own(state, str); - state->err_src = saved_src; - state->err_line = saved_line; + where_pop(state, &saved); return (status); } diff --git a/src/execution/exec_string3.c b/src/execution/exec_string3.c index 94c1b845..d35c3ac9 100644 --- a/src/execution/exec_string3.c +++ b/src/execution/exec_string3.c @@ -45,58 +45,7 @@ hidden from the byte scan (a function body calling shopt, invoked in the same chunk that then uses extglob) still lexes ahead. */ -/* Which bytes end a span. A chunk's FIRST span (open=false) clips at - every input hazard, so a top-level alias/shopt/source line always gets - its own chunk and executes before later text lexes. GROWTH spans - (open=true) are different: the parser has already demanded more, so we - are inside an open construct -- and a hazard line INSIDE a construct - cannot take effect before the whole construct executes in bash either - (bash parses the compound first, then runs its body). Clipping there - bought no correctness and cost a full re-splice/re-lex/re-parse of the - accumulated chunk per interior hazard word: the 2244-line theme rc, - one giant if/else with ~60 alias/source words inside, re-parsed ~80KB - dozens of times and `source ~/.hellishrc` visibly lagged bash. Only - `<<` still clips growth (heredoc consumption depends on delivery). - The overshoot a growth span can add past the construct's close is the - one divergence: a def-then-use pair landing in that tail lexes - together -- accepted, and the statement replay still bounds errors. */ -static bool span_hazard(const char *s, size_t i, size_t n, bool open) -{ - if (!open) - return (input_hazard_at(s, i, n)); - return (s[i] == '<' && i + 1 < n && s[i + 1] == '<'); -} - -/* Longest hazard-free run of COMPLETE lines starting at off; if a hazard - sits in the very first line, that one LOGICAL line -- backslash-newline - joins included. Ending a chunk between a trailing `\` and its - continuation would hand the parser a truncated command (the tokenizer - strips the join, so `echo a \` looks complete and the continuation - line becomes a separate statement). The retreat loop cannot land on a - joined newline: a backslash-newline is itself a hazard, so the scan - already stopped at the first one. */ -static size_t str_span(const char *s, size_t off, size_t n, bool open) -{ - size_t clip; - - clip = off; - while (clip < n && !span_hazard(s, clip, n, open)) - clip++; - while (clip > off && s[clip - 1] != '\n') - clip--; - if (clip > off) - return (clip - off); - clip = off; - while (clip < n && s[clip] != '\n') - clip++; - while (clip < n && clip > off && s[clip - 1] == '\\') - { - clip++; - while (clip < n && s[clip] != '\n') - clip++; - } - return (clip - off + (clip < n)); -} +/* Where each chunk ends is chunk_span's business (exec_string5.c). */ /* Parse every statement of the tokenized chunk WITHOUT executing any. Stops at the first non-OK result; only the tail statement can come @@ -156,6 +105,6 @@ static void chunk_open(t_shell *state, const char *at, size_t len, void chunk_grow(t_shell *state, const char *s, size_t n, t_chunkctx *c) { chunk_close(c); - c->end += str_span(s, c->end, n, c->end > c->start); + c->end += chunk_span(s, c->end, n, c->end > c->start); chunk_open(state, s + c->start, c->end - c->start, c); } diff --git a/src/execution/exec_string4.c b/src/execution/exec_string4.c index 726da5ab..446532c1 100644 --- a/src/execution/exec_string4.c +++ b/src/execution/exec_string4.c @@ -34,13 +34,15 @@ void chunk_close(t_chunkctx *c) /* Execute the fully parsed statements of a chunk, in order, stopping on exit/return/errexit/unwind. Freeing is chunk_close's job so even the - never-reached tail is reclaimed. */ + never-reached tail is reclaimed. -1 when the chunk held no statement at + all -- blank lines, comments -- which must not stand in for the status + of the last command that did run. */ static int run_stmt_list(t_shell *state, t_chunkctx *c, bool *stop) { size_t i; int status; - status = 0; + status = -1; i = 0; while (i < c->asts.len && !*stop) { @@ -79,7 +81,8 @@ static int replay_chunk(t_shell *state, t_chunkctx *c, bool *stop) /* One chunk, start to finish: open at the next hazard boundary, grow while the construct is incomplete and text remains (nothing executes during growth), then run it -- whole-chunk on success, replay on - failure. Advances *off past the chunk either way. */ + failure. Advances *off past the chunk either way. Returns the status + of its last statement, -1 when it had none. */ static int run_chunk(t_shell *state, const char *s, size_t *off, bool *stop) { @@ -94,14 +97,12 @@ static int run_chunk(t_shell *state, const char *s, size_t *off, chunk_grow(state, s, n, &c); while (c.parser.res == RES_GETMOREINPUT && c.end < n) chunk_grow(state, s, n, &c); + where_chunk(state, c.spliced); if (c.parser.res == RES_OK) status = run_stmt_list(state, &c, stop); else - { - if (state->err_src) - state->err_line = 1 + nl_count(s, c.start); status = replay_chunk(state, &c, stop); - } + where_chunk(state, NULL); chunk_close(&c); *off = c.end; return (status); @@ -109,20 +110,31 @@ static int run_chunk(t_shell *state, const char *s, size_t *off, /* Drive the whole string chunk by chunk. Clearing func_return at the end keeps a `return` inside eval from leaking into the caller's frame, - same contract as the old single-pass loop. */ + same contract as the old single-pass loop. The string's status is the + last command's: a trailing chunk of blank lines or comments -- what a + hazard line such as `. file` or a heredoc leaves behind it -- ran + nothing, and used to make `eval $'. f; false\n# note'` succeed. A + sourced file's running line (err_line) advances chunk by chunk. */ int exec_chunks(t_shell *state, const char *str) { + size_t at; size_t off; - size_t n; int status; + int ran; bool stop; off = 0; - n = ft_strlen(str); status = 0; stop = false; - while (off < n && !stop) - status = run_chunk(state, str, &off, &stop); + while (str[off] && !stop) + { + at = off; + ran = run_chunk(state, str, &off, &stop); + if (ran >= 0) + status = ran; + if (state->err_src) + state->err_line += nl_count(str + at, off - at); + } state->func_return = 0; return (status); } diff --git a/src/execution/exec_string5.c b/src/execution/exec_string5.c new file mode 100644 index 00000000..c5d31718 --- /dev/null +++ b/src/execution/exec_string5.c @@ -0,0 +1,81 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* exec_string5.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "execution_private.h" +#include "sh_input.h" + +/* Where the chunker (exec_string3.c) cuts eval and source text. */ + +/* Which bytes end a span. A chunk's FIRST span (open=false) clips at + every input hazard, so a top-level alias/shopt/source line always gets + its own chunk and executes before later text lexes. GROWTH spans + (open=true) are different: the parser has already demanded more, so we + are inside an open construct -- and a hazard line INSIDE a construct + cannot take effect before the whole construct executes in bash either + (bash parses the compound first, then runs its body). Clipping there + bought no correctness and cost a full re-splice/re-lex/re-parse of the + accumulated chunk per interior hazard word: the 2244-line theme rc, + one giant if/else with ~60 alias/source words inside, re-parsed ~80KB + dozens of times and `source ~/.hellishrc` visibly lagged bash. Only + `<<` still clips growth (heredoc consumption depends on delivery). + The overshoot a growth span can add past the construct's close is the + one divergence: a def-then-use pair landing in that tail lexes + together -- accepted, and the statement replay still bounds errors. */ +static bool span_hazard(const char *s, size_t i, size_t n, bool open) +{ + if (!open) + return (input_hazard_at(s, i, n)); + return (s[i] == '<' && i + 1 < n && s[i + 1] == '<'); +} + +/* Push clip past every line that a trailing backslash joins to the one + before it. A chunk may not end between a `\` and its continuation: the + tokenizer strips the join, so `echo a \` looks complete, the + continuation becomes a separate statement, and `{ cmd \` inside an + open group parses as an ERROR rather than as "needs more" -- the chunk + was then replayed instead of grown. A growth span could stop there: + there, a backslash-newline is no hazard, only `<<` is, so the retreat + to a line start landed on the join (#139: nvm's `}' \| grep \ + | sort; } << EOF`). A `\` that only looks like a join (quoted, or in a + comment) makes the span longer, which a growth span can always afford. */ +static size_t str_join_end(const char *s, size_t clip, size_t n) +{ + while (clip < n && clip > 1 && s[clip - 1] == '\n' + && s[clip - 2] == '\\') + { + while (clip < n && s[clip] != '\n') + clip++; + clip += (clip < n); + } + return (clip); +} + +/* Longest hazard-free run of COMPLETE lines starting at off; if a hazard + sits in the very first line, that one line. Either way the span is + stretched to the end of its LOGICAL line, joins included. */ +size_t chunk_span(const char *s, size_t off, size_t n, bool open) +{ + size_t clip; + + clip = off; + while (clip < n && !span_hazard(s, clip, n, open)) + clip++; + while (clip > off && s[clip - 1] != '\n') + clip--; + if (clip == off) + { + while (clip < n && s[clip] != '\n') + clip++; + clip += (clip < n); + } + return (str_join_end(s, clip, n) - off); +} diff --git a/src/execution/exec_where.c b/src/execution/exec_where.c new file mode 100644 index 00000000..a633fef8 --- /dev/null +++ b/src/execution/exec_where.c @@ -0,0 +1,87 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* exec_where.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 19:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 19:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "execution_private.h" + +/* Where the executing command is: the "FILE: line N" a runtime error + starts with, and $LINENO inside a sourced file. + + ctx was only ever set by the READER, from the line it had read up to. + Input arrives in batches -- a whole small script at once -- so by the + time a command ran, the reader stood past the end of the batch, and + every error in `printf 'true\n./nope\n' > t.sh; hellish t.sh` said + "t.sh: line 3" where bash says line 2. Inside a sourced file it was + worse: the error named the file that did the sourcing, at a line of + its own, and $LINENO was that call site's too. + + Now each simple command, as it starts, moves ctx to where its first + token sits -- in the sourced chunk it came from (err_pos) when there is + one, else in the top-level cycle (tok_lineno). ctx is rebuilt only when + that place changes. A token that lies in neither -- a function body, + an eval string -- leaves ctx at the last place that could be named, + the call site, which is also what $LINENO reports there. */ + +/* Newlines between two points of one text, signed. Resolving each line + from the previous one keeps a file run top to bottom at one scan in + total, and a loop at a rescan of its own body per iteration. */ +int nl_between(const char *from, const char *to) +{ + if (to >= from) + return (nl_count(from, (size_t)(to - from))); + return (-nl_count(to, (size_t)(from - to))); +} + +/* The file line of `tok`, a token of the chunk whose first line is line0. */ +int srcpos_line(t_srcpos *p, int line0, const char *tok) +{ + if (p->memo) + p->memo_line += nl_between(p->memo, tok); + else + p->memo_line = line0 + nl_count(p->base, (size_t)(tok - p->base)); + p->memo = tok; + return (p->memo_line); +} + +/* Called as each simple command starts, with its first token: in the + sourced chunk, the file's line; in the top-level text, the script's + (just the name at a prompt, which has no line to give); anywhere else + -- a function body, an eval string -- ctx stays at the call site. */ +void where_follow(t_shell *state, const char *tok) +{ + int line; + + if (srcpos_has(&state->err_pos, tok)) + { + line = srcpos_line(&state->err_pos, state->err_line, tok); + ctx_follow(state, state->err_src, line); + return ; + } + line = cycle_lineno(state, tok); + if (line < 0) + return ; + if (!state->rl.should_update_ctx) + line = 0; + ctx_follow(state, NULL, line); +} + +/* A sourced file's chunk is about to run (text), or has run (NULL). eval + and trap text is not a file: its chunks leave the frame alone. */ +void where_chunk(t_shell *state, const char *text) +{ + if (!state->err_src) + return ; + state->err_pos = (t_srcpos){0}; + if (!text) + return ; + state->err_pos.base = text; + state->err_pos.len = ft_strlen(text); +} diff --git a/src/execution/exec_where2.c b/src/execution/exec_where2.c new file mode 100644 index 00000000..49713caf --- /dev/null +++ b/src/execution/exec_where2.c @@ -0,0 +1,45 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* exec_where2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 19:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 19:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "execution_private.h" + +/* Enter a sourced file: its errors name `src` from line 1. The caller's + frame, ctx string included, is parked in `saved`. */ +void where_push(t_shell *state, t_where *saved, const char *src) +{ + saved->src = state->err_src; + saved->line = state->err_line; + saved->pos = state->err_pos; + saved->ctx = ft_strdup(state->ctx); + saved->ctx_src = state->ctx_src; + saved->ctx_line = state->ctx_line; + state->err_src = src; + state->err_line = 1; + state->err_pos = (t_srcpos){0}; +} + +/* Leave it: the caller's frame comes back as it was, so an error reported + right after the `source` -- by the next command, or by the dot builtin + itself -- names the caller's line, not the file's last one. */ +void where_pop(t_shell *state, t_where *saved) +{ + state->err_src = saved->src; + state->err_line = saved->line; + state->err_pos = saved->pos; + if (!saved->ctx) + return ; + xfree(state->ctx); + state->ctx = saved->ctx; + state->ctx_src = saved->ctx_src; + state->ctx_line = saved->ctx_line; + state->ctx_num = 0; +} diff --git a/src/execution/execution_private.h b/src/execution/execution_private.h index 98a15fe6..f7e23400 100644 --- a/src/execution/execution_private.h +++ b/src/execution/execution_private.h @@ -45,6 +45,12 @@ invalid redirect index %d\n" # define NAME "shell" +/* exec_diag.c: bash reads this much of a file that execve refused to + decide whether it is binary, and try_exec_with_fallback returns + EXEC_ERR_BINARY -- no errno is negative -- when it was. */ +# define EXEC_SAMPLE_LEN 80 +# define EXEC_ERR_BINARY -1 + /* Iteration context for execute_pipeline_children. Bundles everything the loop body needs across prepare_child_exec and finalize_child_parent so we do not have to pass eight separate arguments to each helper. @@ -182,11 +188,12 @@ int handle_direct_path_error(t_shell *state, char *cmd_name, int run_builtin_or_continue(t_shell *state, t_vec *args); int find_exe_path_wrapper(t_shell *state, char *cmd0, char **out_path); -void try_exec_with_fallback(char *path_of_exe, +int try_exec_with_fallback(char *path_of_exe, t_vec *args, char **envp); void cleanup_after_exec_failure(t_vec *args, char *path_of_exe, char **envp); -int map_errno_to_exit(void); +bool exec_file_is_binary(const char *path); +int exec_failure_report(t_shell *state, char *path, int err); void set_for_var(t_shell *state, char *name, char *val); void restore_one(t_shell *state, t_scope_save *s); void restore_temp_assigns(t_shell *state, t_vec *saves); @@ -246,6 +253,8 @@ typedef struct s_chunkctx } t_chunkctx; void chunk_close(t_chunkctx *c); +size_t chunk_span(const char *s, size_t off, size_t n, + bool open); void chunk_grow(t_shell *state, const char *s, size_t n, t_chunkctx *c); int exec_chunks(t_shell *state, const char *str); @@ -255,6 +264,36 @@ int run_one_stmt(t_shell *state, t_deque_tok *tt, int run_parsed(t_shell *state, t_ast_node *ast); bool must_stop(t_shell *state); +/* exec_where.c: where the executing command is. `source` saves the whole + frame -- the file, its chunk, and the ctx string naming the caller's + place -- and puts it back exactly, so the next error after the source + names the caller again. */ +typedef struct s_where +{ + const char *src; + int line; + t_srcpos pos; + char *ctx; + const char *ctx_src; + int ctx_line; +} t_where; + +int nl_between(const char *from, const char *to); +int srcpos_line(t_srcpos *p, int line0, const char *tok); +int cycle_lineno(t_shell *state, const char *tok); +int tok_lineno(t_shell *state); +void ctx_follow(t_shell *state, const char *src, int line); +void where_follow(t_shell *state, const char *tok); +void where_chunk(t_shell *state, const char *text); +void where_push(t_shell *state, t_where *saved, + const char *src); +void where_pop(t_shell *state, t_where *saved); + +static inline bool srcpos_has(t_srcpos *p, const char *tok) +{ + return (p->base && tok >= p->base && tok < p->base + p->len); +} + /* select's loop state (execute_select.c): the expanded words, the body wrapped once, the loop variable, whether the next round reprints the menu, and the status the loop ends with. */ diff --git a/src/execution/run.c b/src/execution/run.c index 5efb8ece..a1393b9e 100644 --- a/src/execution/run.c +++ b/src/execution/run.c @@ -13,27 +13,11 @@ #include "execution_private.h" #include "sys.h" -/* Print the EACCES diagnostic and clean up after an execve failure. We - save and restore errno around the cleanup calls because free functions - (xfree, free_tab) can internally call libc routines that clobber errno - before map_errno_to_exit gets a chance to read it. */ -static void preserve_errno_exec_failed(t_shell *state, char **path_of_exe, - t_vec *args, char **envp) -{ - int saved_errno; - - saved_errno = errno; - if (saved_errno == EACCES) - ft_eprintf("%s: %s: %s\n", state->ctx, - *path_of_exe, strerror(saved_errno)); - cleanup_after_exec_failure(args, *path_of_exe, envp); - errno = saved_errno; -} - /* Resolve argv[0] on PATH and execve it, consulting NEITHER the function table nor the builtin table. Publishes the resolved path as ULTIMATE_ARG ("_"), builds envp, then execve -- which only returns on failure, so - everything after try_exec_with_fallback is cleanup + status mapping. + everything after try_exec_with_fallback is the diagnostic, cleanup and + the status. Runs in a forked child; it must never return to the parent's call stack. Split out of actually_run for `command`: skipping the function lookup is the entire point of that builtin, and it needs this half alone. */ @@ -42,6 +26,7 @@ int exec_external_argv(t_shell *state, t_vec *args) char *path_of_exe; char **envp; int status; + int err; path_of_exe = NULL; ft_assert(args->len >= 1); @@ -55,9 +40,10 @@ int exec_external_argv(t_shell *state, t_vec *args) env_set(&state->env, env_create(ft_strdup(ULTIMATE_ARG), ft_strdup(path_of_exe), true)); envp = get_envp(state, path_of_exe); - try_exec_with_fallback(path_of_exe, args, envp); - preserve_errno_exec_failed(state, &path_of_exe, args, envp); - return (map_errno_to_exit()); + err = try_exec_with_fallback(path_of_exe, args, envp); + status = exec_failure_report(state, path_of_exe, err); + cleanup_after_exec_failure(args, path_of_exe, envp); + return (status); } /* The child side of executing an external command. Builtins and functions diff --git a/src/execution/run_utils.c b/src/execution/run_utils.c index 2b730c6d..b9015db4 100644 --- a/src/execution/run_utils.c +++ b/src/execution/run_utils.c @@ -71,11 +71,12 @@ static void exec_fallback_shell(char *path, t_vec *args, char **envp) /* Attempt execve; on ENOEXEC (file is executable but not in a format the kernel can load -- e.g. a shell script without a shebang), retry with - a shell interpreter prepended. The NULL sentinel is pushed onto args - so that the execve call receives a proper argv terminator. On any - other execve failure this function returns and actually_run maps the - errno to an exit code. */ -void try_exec_with_fallback(char *path_of_exe, + a shell interpreter prepended -- unless the file is binary, which bash + refuses rather than feed to a shell (#133). The NULL sentinel is pushed + onto args so that execve receives a proper argv terminator. Returns + only on failure, with the errno to report (EXEC_ERR_BINARY for the + refused binary), which exec_failure_report turns into bash's words. */ +int try_exec_with_fallback(char *path_of_exe, t_vec *args, char **envp) { @@ -84,6 +85,10 @@ void try_exec_with_fallback(char *path_of_exe, null_ptr = NULL; vec_push(args, &null_ptr); execve(path_of_exe, (char **)(args->ctx), envp); - if (errno == ENOEXEC) - exec_fallback_shell(path_of_exe, args, envp); + if (errno != ENOEXEC) + return (errno); + if (exec_file_is_binary(path_of_exe)) + return (EXEC_ERR_BINARY); + exec_fallback_shell(path_of_exe, args, envp); + return (errno); } diff --git a/src/execution/run_utils2.c b/src/execution/run_utils2.c index f212f68b..7aa65799 100644 --- a/src/execution/run_utils2.c +++ b/src/execution/run_utils2.c @@ -76,21 +76,3 @@ void cleanup_after_exec_failure(t_vec *args, xfree(path_of_exe); free_tab(envp); } - -/* Map errno to POSIX shell exit codes after execve failure. */ -int map_errno_to_exit(void) -{ - if (errno == EACCES) - return (PERMISSION_DENIED); - if (errno == ENOENT) - return (NO_SUCH_FILE_OR_DIR); - if (errno == ENOEXEC) - return (EXIT_CMD_NOT_EXEC); - if (errno == ENOTDIR) - return (NO_SUCH_DIR); - if (errno == ENOMEM) - return (OUT_OF_MEM); - if (errno == EISDIR) - return (IS_A_DIR); - return (EXIT_GENERAL_ERR); -} diff --git a/src/expander/expand_cmd_simple_word.c b/src/expander/expand_cmd_simple_word.c index 64dd748f..8d6c3960 100644 --- a/src/expander/expand_cmd_simple_word.c +++ b/src/expander/expand_cmd_simple_word.c @@ -15,16 +15,22 @@ /* Expand one AST_WORD child of a simple command into argv. The word slab is activated (word_slab_push(1)) for the duration so that argv strings use the per-command slab allocator instead of plain malloc — this makes - bulk-freeing the argv after execution much cheaper. The `export` flag is - set if the first word is literally "export", which later suppresses globbing - on any following assignment words (export PATH=$PATH is not globbed). */ + bulk-freeing the argv after execution much cheaper. The `decl` flag is + set when the command is a declaration utility (decl_word_kind), directly + or through a bare `command`: its NAME=value arguments then expand as one + field each (`local v=$1`, `export PATH=$PATH`). */ int expand_simple_cmd_word(t_shell *state, t_expander_simple_cmd *exp, t_executable_cmd *ret) { int o; + int kind; - if (!exp->found_first && is_export(*exp->curr)) - exp->export = true; + if (!exp->found_first || exp->decl_next) + { + kind = decl_word_kind(*exp->curr); + exp->decl = (kind == DECL_UTILITY); + exp->decl_next = (!exp->found_first && kind == DECL_COMMAND); + } o = word_slab_push(1); expand_word_ro(state, exp->curr, &ret->argv, false); word_slab_push(o); diff --git a/src/expander/expand_decl_word.c b/src/expander/expand_decl_word.c new file mode 100644 index 00000000..6cec064b --- /dev/null +++ b/src/expander/expand_decl_word.c @@ -0,0 +1,46 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* expand_decl_word.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "expander_private.h" + +/* Is the word, a single unquoted TT_WORD, exactly `name`? */ +static bool word_is(t_ast_node word, const char *name) +{ + t_ast_node c; + + if (word.children.len != 1) + return (false); + c = ((t_ast_node *)word.children.ctx)[0]; + return (c.token.tt == TT_WORD && (size_t)c.token.len == ft_strlen(name) + && ft_strncmp(c.token.start, name, c.token.len) == 0); +} + +/* DECL_UTILITY when the word names a declaration utility, DECL_COMMAND + for a bare `command` (the NEXT word then decides), else 0. + A declaration utility's NAME=value arguments are not field-split and + not globbed: `local new=$2` with $2 = "sudo -e" is one assignment in + bash and zsh alike. Only `export` was recognised, so local, declare, + typeset and readonly split theirs -- `local v=$1` kept the first word + of $1, and the omz sudo widget's `local old=$1 new=$2 space=${2:+ }` + turned `vim f` into `sudo/f` on ESC ESC (#137). bash applies the rule + to the literal names, and through a bare `command`, but not through + `builtin` or `command -p`; so does this. */ +int decl_word_kind(t_ast_node word) +{ + if (word_is(word, "export") || word_is(word, "readonly") + || word_is(word, "declare") || word_is(word, "typeset") + || word_is(word, "local")) + return (DECL_UTILITY); + if (word_is(word, "command")) + return (DECL_COMMAND); + return (0); +} diff --git a/src/expander/expand_param_xform.c b/src/expander/expand_param_xform.c index 7a4b8e4c..50e34c46 100644 --- a/src/expander/expand_param_xform.c +++ b/src/expander/expand_param_xform.c @@ -13,12 +13,15 @@ #include "expander_private.h" #include "env.h" #include "helpers.h" +#include "prompt.h" /* ${var@OP} parameter transformations (bash 5): @Q single-quote the value so it can be re-read by the shell @U uppercase @L lowercase @u uppercase-first @A an assignment statement that would recreate the variable - The rarer @E/@P/@a/@K are documented v1 scope-outs. + @P the value rendered as a prompt string, by the PS1 renderer + (prompt_expand_p) -- so ${PS1@P} is the prompt itself + The rarer @E/@a/@K are documented v1 scope-outs. @Q is sq_quote (src/helpers/sq_quote.c), shared with the completion dispatcher, which quotes the word under the cursor for the same reason: @@ -68,6 +71,8 @@ char *expand_xform(t_shell *state, const char *s, int name_len, char op) return (xform_case(val, op)); if (op == 'A') return (xform_assign(s, name_len, val)); + if (op == 'P') + return (prompt_expand_p(state, val)); return (ft_strdup(val)); } diff --git a/src/expander/expand_simple_cmd_assignment.c b/src/expander/expand_simple_cmd_assignment.c index 0fac7608..3ab38a62 100644 --- a/src/expander/expand_simple_cmd_assignment.c +++ b/src/expander/expand_simple_cmd_assignment.c @@ -19,14 +19,16 @@ a private clone so the caller's node stays intact (loops reuse it). Returns 1 on unwind request, 0 otherwise. - `export` gets EW_KEEP_AS_ONE because an assignment word does not field - split -- `export PATH=$PATH` is one operand however many spaces $PATH - holds. It used to also get the RAW SOURCE TEXT restored over any argv - entry ending in `=`, which paired with a second quote-strip-and-expand - inside the export builtin: two wrongs that cancelled on `export s=$UNSET` - and destroyed everything else (`export s=''` stored two literal quotes, + A declaration utility (export, readonly, declare, typeset, local) gets + EW_KEEP_AS_ONE because an assignment word does not field split -- + `export PATH=$PATH` is one operand however many spaces $PATH holds, and + so is `local new=$2` (#137). `export` used to also get the RAW SOURCE + TEXT restored over any argv entry ending in `=`, which paired with a + second quote-strip-and-expand inside the export builtin: two wrongs + that cancelled on `export s=$UNSET` and destroyed everything else + (`export s=''` stored two literal quotes, `export s='x$y'` stored `x`). Both halves are gone; the ordinary - expansion is the answer, as it already was for declare/readonly/local. */ + expansion is the answer. */ static int expand_assignment_word_and_fixup(t_shell *state, t_expander_simple_cmd *exp, t_executable_cmd *ret) { @@ -38,7 +40,7 @@ static int expand_assignment_word_and_fixup(t_shell *state, scratch = clone_ast(exp->curr); assignment_word_to_word(&scratch); flags = EW_NO_GLOB; - if (exp->export) + if (exp->decl) flags |= EW_KEEP_AS_ONE; expand_word_glob_ctl(state, &scratch, &ret->argv, flags); if (exec_aborting(state)) diff --git a/src/expander/expand_zsh_bare.c b/src/expander/expand_zsh_bare.c index 51f9976c..74d458ff 100644 --- a/src/expander/expand_zsh_bare.c +++ b/src/expander/expand_zsh_bare.c @@ -11,6 +11,7 @@ /* ************************************************************************** */ #include "expander_private.h" +#include "parena.h" /* `${${(s: :)$(git version)}[3]}` -- a nested flagged expansion with NO ** flags of its own on the outside. @@ -46,3 +47,39 @@ bool zsh_bare_nested(t_shell *state, t_token *tt, bool split_ctx) zf_emit(state, &f, tt, &l); return (true); } + +/* The same nested flagged expansion with an OPERATOR after it, the +** subscript optional: `${${(Az)aliases[$cmd]}[1]:-$cmd}` -- the omz sudo +** widget's "what command does this alias run", evaluated on ESC ESC. +** +** zsh_bare_nested stops at a bare subscript, so this shape fell one layer +** down to the scalar engine, which has neither the token nor a subscript: +** a bad substitution on every ESC ESC (#137). Here the inner value comes +** from zf_nested WITH the token, so the quoting decides, as it does for +** the bare shape, whether a flagged inner value is still an array -- the +** `[1]` then picks a word (`ls`) and not a character (`l`) -- and the +** outer subscript and operator are applied by zsh_nested_apply, which +** binds the value and hands `scratch[1]:-$cmd` to the ordinary expander. +** An unflagged inner (`${${x}:-d}`) has no array-ness to lose, and keeps +** going the scalar way. */ +bool zsh_nested_op(t_shell *state, t_token *tt) +{ + char *v; + int n; + + n = zsh_nested_len(tt->start, tt->len); + if (n < 5 || n >= tt->len || tt->start[2] != '(' + || !zf_is_nested(tt->start, n)) + return (false); + v = zf_nested(state, tt, tt->start, n); + if (!v) + return (false); + v = zsh_nested_apply(state, v, tt->start + n, tt->len - n); + if (!v) + return (false); + tt->start = v; + tt->len = (int)ft_strlen(v); + tt->allocated = true; + parena_note_attach(); + return (true); +} diff --git a/src/expander/expand_zsh_disp.c b/src/expander/expand_zsh_disp.c index dcbfd7b2..4a648cb0 100644 --- a/src/expander/expand_zsh_disp.c +++ b/src/expander/expand_zsh_disp.c @@ -41,7 +41,7 @@ of the nested part including both braces, or 0. Only a body that STARTS with `${` qualifies; a `${` later in the body belongs to an operator's word and is expanded by that operator, not here. */ -static int zd_nested_len(const char *s, int slen) +int zsh_nested_len(const char *s, int slen) { int depth; int i; @@ -61,8 +61,8 @@ static int zd_nested_len(const char *s, int slen) return (0); } -/* Evaluate the nested part, BIND it to a scratch parameter, and read the -** outer operator against that parameter. +/* BIND an evaluated nested value to a scratch parameter, and read the +** outer operator `rest` against that parameter. Owns `inner`. ** ** THE VALUE, NOT ITS TEXT. Splicing the value in as text was the first ** attempt and it is wrong in a way that reads as working: with A=1, @@ -76,34 +76,45 @@ static int zd_nested_len(const char *s, int slen) ** substitution, the modifiers, :#, a subscript -- applies to the value ** through the code that already implements it. There is no second ** implementation of any of them, which is why this is a few lines rather -** than a parallel expander. +** than a parallel expander. The scratch body goes through the token-level +** expander (zsh_body_eval): a subscript with an operator, `[1]:-d`, lives +** there and not in the scalar engine (#137). ** ** The scratch parameter is removed afterwards, and its name is depth-keyed -** so a doubly-nested body cannot clobber the level still reading it. */ -static char *zd_nested(t_shell *state, const char *s, int slen, int n) +** so a doubly-nested body cannot clobber the level still reading it. +** Two callers: zd_nested below, and zsh_nested_op one layer up, which +** knows the quoting and so can evaluate the inner value as an array. */ +char *zsh_nested_apply(t_shell *state, char *inner, const char *rest, + int rlen) { static int depth = 0; - char *inner; char *body; char *name; char *out; - inner = zf_inner_text(state, s, n); name = zd_bind_name(depth++); if (!inner || !name) return (depth--, xfree(inner), xfree(name), NULL); env_set(&state->env, env_create(ft_strdup(name), inner, false)); - body = zd_splice(name, s + n, slen - n); + body = zd_splice(name, rest, rlen); out = NULL; if (body) - out = expand_param_format(state, body, (int)ft_strlen(body), false); - if (body && !out) - out = zd_plain(state, name, (int)ft_strlen(name)); + out = zsh_body_eval(state, body, (int)ft_strlen(body)); zd_unbind(state, name); depth--; return (xfree(body), xfree(name), out); } +/* A nested operand reached WITHOUT its token -- inside another operator's + word, or a flag's operand -- so without the quoting that would decide + whether a flagged inner value stays an array; it is read as text. The + common shapes never get here: zsh_nested_op handles them with the token. */ +static char *zd_nested(t_shell *state, const char *s, int slen, int n) +{ + return (zsh_nested_apply(state, zf_inner_text(state, s, n), + s + n, slen - n)); +} + char *zsh_dispatch(t_shell *state, const char *s, int slen, bool arr) { char *out; @@ -111,7 +122,7 @@ char *zsh_dispatch(t_shell *state, const char *s, int slen, bool arr) if (!zsh_mode(state) || slen <= 0) return (NULL); - n = zd_nested_len(s, slen); + n = zsh_nested_len(s, slen); if (n > 0) return (zd_nested(state, s, slen, n)); (void)arr; diff --git a/src/expander/expand_zsh_disp3.c b/src/expander/expand_zsh_disp3.c index fec5f3aa..fd0a8fc1 100644 --- a/src/expander/expand_zsh_disp3.c +++ b/src/expander/expand_zsh_disp3.c @@ -40,3 +40,37 @@ char *zsh_token_text(t_shell *state, const char *s, int slen) return ((char *)tok.start); return (ft_strndup(tok.start, (size_t)tok.len)); } + +/* A ${...} body, given WITHOUT its braces, through the token-level +** expander -- the entry an ordinary ${...} word takes. The two zsh +** operands that are only a slice of an outer body -- what a nested +** expansion's outer operator reads (`${${x}[1]:-d}`), and what a flag +** list applies to (`${(U)e[2]}`) -- went to expand_param_format, the +** scalar engine, which has no subscript: `name[sub]` there, alone or +** before an operator, is expand_array_elem_op's form, one layer up. Both +** came back as bad substitutions (issue #137: the omz sudo widget's +** `${${(Az)aliases[$cmd]}[1]:-$cmd}`, on ESC ESC). Owned result, "" for +** an unset parameter. */ +char *zsh_body_eval(t_shell *state, const char *s, int slen) +{ + t_token tok; + + tok = (t_token){.tt = TT_ENVVAR, .start = (char *)s, .len = slen}; + expand_token(state, &tok, false); + if (tok.allocated) + return ((char *)tok.start); + if (!tok.start) + return (ft_strdup("")); + return (ft_strndup(tok.start, (size_t)tok.len)); +} + +/* Does the operand start with `name[` -- a subscripted parameter? */ +bool zsh_subscripted(const char *s, int slen) +{ + int i; + + i = 0; + while (i < slen && (ft_isalnum(s[i]) || s[i] == '_')) + i++; + return (i > 0 && i < slen && s[i] == '[' && !ft_isdigit(s[0])); +} diff --git a/src/expander/expand_zsh_flags.c b/src/expander/expand_zsh_flags.c index 6ae3ee95..8c05bb18 100644 --- a/src/expander/expand_zsh_flags.c +++ b/src/expander/expand_zsh_flags.c @@ -125,7 +125,8 @@ bool expand_zsh_flags(t_shell *state, t_token *tt, bool split_ctx) if (!zsh_mode(state) || tt->len < 2) return (false); if (tt->start[0] != '(') - return (zsh_bare_nested(state, tt, split_ctx)); + return (zsh_bare_nested(state, tt, split_ctx) + || zsh_nested_op(state, tt)); f = (t_zflags){0}; f.split = split_ctx; end = zf_parse(tt->start, tt->len, &f); diff --git a/src/expander/expand_zsh_flags2.c b/src/expander/expand_zsh_flags2.c index f993ff12..9d39e15f 100644 --- a/src/expander/expand_zsh_flags2.c +++ b/src/expander/expand_zsh_flags2.c @@ -48,7 +48,9 @@ void zf_free(t_zflags *f) which is what makes ${(f)$(cmd)} the idiom it is. The second case runs the ordinary word pipeline (tilde, command substitution, parameters; no split, no glob); the first re-enters expand_param_format so ${(U)x:-y} - keeps working, and falls back to a plain lookup. + keeps working, and falls back to a plain lookup. A SUBSCRIPTED name -- + ${(U)e[2]}, ${(z)aliases[ll]} -- goes through the token-level expander + instead, which is where subscripts live (zsh_body_eval, #137). env_expand_n's result is BORROWED from the environment, so it is copied: everything downstream of here owns what it holds. */ char *zf_inner(t_shell *state, t_token *tt, const char *s, int slen) @@ -67,6 +69,8 @@ char *zf_inner(t_shell *state, t_token *tt, const char *s, int slen) } if (s[0] == '$' || s[0] == '"' || s[0] == '\'' || s[0] == '`') return (pf_word_pipeline(state, s, slen, false)); + if (zsh_subscripted(s, slen)) + return (zsh_body_eval(state, s, slen)); v = expand_param_format(state, s, slen, false); if (v) return (v); diff --git a/src/expander/expander_private.h b/src/expander/expander_private.h index b92656c6..4c531aa7 100644 --- a/src/expander/expander_private.h +++ b/src/expander/expander_private.h @@ -170,6 +170,7 @@ int net_redir_open(char *fname, t_redir *ret); bool create_redir_4(t_tt tt, char *fname, t_redir *ret, int src_fd); bool redir_park_fd(t_redir *ret); bool dup_target_is_fd(const char *fname); +bool redir_src_fd_ok(t_shell *state, t_redir *r); bool dup_of_pending(t_shell *state, t_tt tt, char *fname, t_redir *ret); int parse_src_fd(t_tt tt, t_token op_tok); @@ -266,7 +267,7 @@ void print_redir_err(t_shell *state, int redirect_from_ast_redir(t_shell *state, t_ast_node *curr, int *redir_idx); -bool is_export(t_ast_node word); +int decl_word_kind(t_ast_node word); int expand_simple_cmd_assignment(t_shell *state, t_expander_simple_cmd *exp, t_executable_cmd *ret); int expand_simple_cmd_redir(t_shell *state, @@ -305,7 +306,6 @@ bool token_starts_with(t_token t, char *str); t_token_old get_old_token(t_ast_node word); int expand_simple_cmd_redir(t_shell *state, t_expander_simple_cmd *exp, t_vec_int *redirects); -bool is_export(t_ast_node word); bool is_empty_command(const char *cmd); void expand_cmd_substitutions(t_shell *state, t_ast_node *node); void expand_node_glob(t_ast_node *node, t_vec *args, bool keep_as_one, @@ -333,6 +333,7 @@ void ft_reset(void *ptr, size_t size, void free_children(void *p); void push_and_reinit_curr_node(t_vec_nd *ret, t_ast_node *curr_node); void push_new_env_child(t_ast_node *curr_node, char *new_start); +void push_new_dq_child(t_ast_node *curr_node, char *new_start); /* indexed arrays (expand_array*.c, split_array.c) */ bool expand_array_token(t_shell *state, t_token *tt, bool split_ctx); @@ -342,12 +343,19 @@ void arr_marks_clear(t_shell *state); bool arr_keys_defer(t_shell *state, t_token *tt); void emit_keys_fields(t_shell *state, const char *name, t_ast_node *curr_node, t_vec_nd *ret); +void emit_keys_at(t_shell *state, const char *name, + t_ast_node *curr_node, t_vec_nd *ret); void emit_array_at(t_shell *state, const char *name, t_ast_node *curr_node, t_vec_nd *ret); void emit_array_split(t_shell *state, const char *name, t_ast_node *curr_node, t_vec_nd *ret); +/* emit_assoc_fields mode bits: the keys rather than the values, and + verbatim (quoted) fields rather than glob-eligible ones. */ +# define EMIT_KEYS 1 +# define EMIT_QUOTED 2 + void emit_assoc_fields(char *val, t_ast_node *curr_node, t_vec_nd *ret, - int want_keys); + int mode); typedef struct s_slice_ctx { char *val; @@ -449,6 +457,12 @@ char *zsh_param(t_shell *state, const char *s, int slen); char *zsh_dispatch(t_shell *state, const char *s, int slen, bool arr); char *zf_inner_text(t_shell *state, const char *s, int n); char *zsh_token_text(t_shell *state, const char *s, int slen); +char *zsh_body_eval(t_shell *state, const char *s, int slen); +bool zsh_subscripted(const char *s, int slen); +int zsh_nested_len(const char *s, int slen); +char *zsh_nested_apply(t_shell *state, char *inner, const char *rest, + int rlen); +bool zsh_nested_op(t_shell *state, t_token *tt); char *zd_plain(t_shell *state, const char *name, int len); char *zd_splice(const char *val, const char *rest, int rlen); char *zd_bind_name(int depth); diff --git a/src/expander/expander_split2.c b/src/expander/expander_split2.c index c0876838..3c1ba3ad 100644 --- a/src/expander/expander_split2.c +++ b/src/expander/expander_split2.c @@ -37,6 +37,8 @@ static bool try_array_child(t_shell *state, t_token *curr_t, return (false); if (name[0] == ARR_MAGIC) emit_val_at(name, curr_node, ret); + else if (name[0] == '!' && curr_t->tt == TT_DQENVVAR) + emit_keys_at(state, name, curr_node, ret); else if (name[0] == '!') emit_keys_fields(state, name, curr_node, ret); else if (curr_t->tt == TT_DQENVVAR) diff --git a/src/expander/redir_fd.c b/src/expander/redir_fd.c index cb9ae2d9..8cad4ec0 100644 --- a/src/expander/redir_fd.c +++ b/src/expander/redir_fd.c @@ -6,12 +6,14 @@ /* By: marvin +#+ +:+ +#+ */ /* +#+#+#+#+#+ +#+ */ /* Created: 2026/08/19 20:40:00 by marvin #+# #+# */ -/* Updated: 2026/08/19 20:40:00 by marvin ### ########.fr */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ /* */ /* ************************************************************************** */ #include "expander_private.h" +#include #include +#include #include /* Park a freshly acquired redirection fd out of the low, user-addressable @@ -69,3 +71,32 @@ bool dup_target_is_fd(const char *fname) } return (true); } + +/* A redirection onto a descriptor at or above the limit cannot be made: + dup2 fails with EBADF there, and nowhere else for a valid source. bash + finds out as it applies it -- after opening the file, which is created + all the same -- and says "N: Bad file descriptor"; the command does not + run and the status is 1. This shell applies a command's redirections + only once all are resolved, past any way back, so the same test is made + as each one is resolved (`1234567>f` reaches here now that the lexer + takes every IO_NUMBER whole). A dup is named by its target, as in bash; + closing a descriptor that cannot be open (`N>&-`) quietly does nothing + there too. What this resolution opened is closed again on failure, since + a redirection that failed is never handed to the command's teardown. */ +bool redir_src_fd_ok(t_shell *state, t_redir *r) +{ + long max; + + max = sysconf(_SC_OPEN_MAX); + if (r->close_fd || max < 0 || r->src_fd < max) + return (true); + if (r->is_dup) + ft_eprintf("%s: %d: %s\n", state->ctx, r->fd, strerror(EBADF)); + else + ft_eprintf("%s: %d: %s\n", state->ctx, r->src_fd, strerror(EBADF)); + if (!r->is_dup && r->fd > STDERR_FILENO) + close(r->fd); + if (!r->is_dup) + r->fd = -1; + return (false); +} diff --git a/src/expander/redirect_from_ast_redir2.c b/src/expander/redirect_from_ast_redir2.c index 55accbff..bf44df20 100644 --- a/src/expander/redirect_from_ast_redir2.c +++ b/src/expander/redirect_from_ast_redir2.c @@ -108,7 +108,8 @@ static int heredoc_redir(t_shell *state, t_ast_node *curr, t_token op_tok, handled specially: if the heredoc body is already materialised (has_redirect) we just return its cached index; if not yet materialised, materialize_heredoc is called. All other redirect types go through try_create_redir which opens - the file and registers the fd in state->redirects. */ + the file and registers the fd in state->redirects. Either way the fd it + acts on must be one a redirection can reach (redir_src_fd_ok). */ int redirect_from_ast_redir(t_shell *state, t_ast_node *curr, int *redir_idx) { t_token op_tok; @@ -119,10 +120,16 @@ int redirect_from_ast_redir(t_shell *state, t_ast_node *curr, int *redir_idx) op_tok = ((t_ast_node *)curr->children.ctx)[0].token; tt = op_tok.tt; if (tt == TT_HEREDOC) - return (heredoc_redir(state, curr, op_tok, redir_idx)); - src_fd = parse_src_fd(tt, op_tok); - if (try_create_redir(state, curr, tt, src_fd) < 0) + heredoc_redir(state, curr, op_tok, redir_idx); + else + { + src_fd = parse_src_fd(tt, op_tok); + if (try_create_redir(state, curr, tt, src_fd) < 0) + return (-1); + *redir_idx = curr->redir_idx; + } + if (!redir_src_fd_ok(state, + vec_idx(&state->redirects, (size_t)(*redir_idx)))) return (-1); - *redir_idx = curr->redir_idx; return (0); } diff --git a/src/expander/split_array.c b/src/expander/split_array.c index 42fd83d2..42756068 100644 --- a/src/expander/split_array.c +++ b/src/expander/split_array.c @@ -35,13 +35,15 @@ void emit_array_at(t_shell *state, const char *name, t_ast_node *curr_node, emit_val_at(val, curr_node, ret); } -/* One field per associative element: values (want_keys 0) or keys - (want_keys 1). Shared by "${h[@]}" and "${!h[@]}" emission. */ +/* One field per associative element: the values, or the keys with + EMIT_KEYS. Shared by "${h[@]}", ${h[@]} and both keys forms; EMIT_QUOTED + is the quoted ones, whose fields are never pathname-expanded. */ void emit_assoc_fields(char *val, t_ast_node *curr_node, t_vec_nd *ret, - int want_keys) + int mode) { t_assoc_it it; int nth; + char *s; assoc_it_init(&it, val); nth = 0; @@ -49,10 +51,14 @@ void emit_assoc_fields(char *val, t_ast_node *curr_node, t_vec_nd *ret, { if (nth++ > 0) push_and_reinit_curr_node(ret, curr_node); - if (want_keys) - push_new_env_child(curr_node, ft_strndup(it.k, it.kl)); + if (mode & EMIT_KEYS) + s = ft_strndup(it.k, it.kl); else - push_new_env_child(curr_node, ft_strndup(it.v, it.vl)); + s = ft_strndup(it.v, it.vl); + if (mode & EMIT_QUOTED) + push_new_dq_child(curr_node, s); + else + push_new_env_child(curr_node, s); } } @@ -112,7 +118,7 @@ void emit_keys_fields(t_shell *state, const char *name, if (!val) return ; if (assoc_is(val)) - return (emit_assoc_fields(val, curr_node, ret, 1)); + return (emit_assoc_fields(val, curr_node, ret, EMIT_KEYS)); cur = ""; if (arr_is(val)) cur = val + 1; diff --git a/src/expander/split_array2.c b/src/expander/split_array2.c index 131499bf..04923e98 100644 --- a/src/expander/split_array2.c +++ b/src/expander/split_array2.c @@ -21,7 +21,8 @@ A value that is not an array is one field, and an unset one is none -- the same two edge cases "${x[@]}" has always had, now shared rather than - restated. */ + restated. Every field is a quoted one (push_new_dq_child): nothing in it + is a glob. */ void emit_val_at(const char *val, t_ast_node *curr_node, t_vec_nd *ret) { const char *cur; @@ -32,15 +33,15 @@ void emit_val_at(const char *val, t_ast_node *curr_node, t_vec_nd *ret) if (!val) return ; if (assoc_is(val)) - return (emit_assoc_fields((char *)val, curr_node, ret, 0)); + return (emit_assoc_fields((char *)val, curr_node, ret, EMIT_QUOTED)); if (!arr_is(val)) - return (push_new_env_child(curr_node, ft_strdup(val))); + return (push_new_dq_child(curr_node, ft_strdup(val))); cur = val + 1; nth[0] = 0; while (arr_next(&cur, &idx, &v, &nth[1])) { if (nth[0]++ > 0) push_and_reinit_curr_node(ret, curr_node); - push_new_env_child(curr_node, ft_strndup(v, nth[1])); + push_new_dq_child(curr_node, ft_strndup(v, nth[1])); } } diff --git a/src/expander/split_envvar.c b/src/expander/split_envvar.c index e6c0ebda..bca9ec43 100644 --- a/src/expander/split_envvar.c +++ b/src/expander/split_envvar.c @@ -74,8 +74,9 @@ void emit_positional_split(t_shell *state, t_ast_node *curr_node, } /* Expand a quoted "$@": one field per positional parameter (NOT IFS-split, so - params containing spaces stay intact). First/last fields join adjacent - literal text, mirroring POSIX behaviour for pre"$@"post. */ + params containing spaces stay intact, and not pathname-expanded, so a `*` + stays a `*`). First/last fields join adjacent literal text, mirroring + POSIX behaviour for pre"$@"post. */ void emit_positional_at(t_shell *state, t_ast_node *curr_node, t_vec_nd *ret) { char *cnt; @@ -98,8 +99,8 @@ void emit_positional_at(t_shell *state, t_ast_node *curr_node, t_vec_nd *ret) if (i > 1) push_and_reinit_curr_node(ret, curr_node); if (v) - push_new_env_child(curr_node, ft_strdup(v)); + push_new_dq_child(curr_node, ft_strdup(v)); else - push_new_env_child(curr_node, ft_strdup("")); + push_new_dq_child(curr_node, ft_strdup("")); } } diff --git a/src/expander/split_quoted.c b/src/expander/split_quoted.c new file mode 100644 index 00000000..1cd4040d --- /dev/null +++ b/src/expander/split_quoted.c @@ -0,0 +1,59 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* split_quoted.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 17:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 17:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "expander_private.h" +#include "env.h" + +/* Append one element of a QUOTED list expansion -- "$@", "${a[@]}", + "${!a[@]}" -- to the field being built. + The token is TT_DQENVVAR, which is what tells the glob stage the text + was quoted (star_expandable). These fields used to be built with + push_new_env_child, whose TT_ENVVAR means UNQUOTED, so every element was + pathname-expanded: `f() { rm -f -- "$@"; }; f '*'` removed every file in + the directory. */ +void push_new_dq_child(t_ast_node *curr_node, char *new_start) +{ + t_ast_node tmp; + + tmp = new_env_node(new_start); + tmp.token.tt = TT_DQENVVAR; + vec_push(&curr_node->children, &tmp); +} + +/* Quoted "${!name[@]}": one verbatim field per index or key. The unquoted + form is emit_keys_fields; `name` still has its leading '!'. */ +void emit_keys_at(t_shell *state, const char *name, + t_ast_node *curr_node, t_vec_nd *ret) +{ + char *val; + const char *cur; + const char *v; + long idx; + int nth[2]; + + val = env_expand(state, (char *)name + 1); + if (!val) + return ; + if (assoc_is(val)) + return (emit_assoc_fields(val, curr_node, ret, + EMIT_KEYS | EMIT_QUOTED)); + cur = ""; + if (arr_is(val)) + cur = val + 1; + nth[0] = 0; + while (arr_next(&cur, &idx, &v, &nth[1])) + { + if (nth[0]++ > 0) + push_and_reinit_curr_node(ret, curr_node); + push_new_dq_child(curr_node, ft_itoa((int)idx)); + } +} diff --git a/src/expander/utils.c b/src/expander/utils.c index 73f678cb..85f698b3 100644 --- a/src/expander/utils.c +++ b/src/expander/utils.c @@ -28,23 +28,6 @@ bool is_empty_command(const char *cmd) return (true); } -/* True when `word` is a single TT_WORD token whose text is exactly "export". - Used to detect the command name so assignment words that follow it are - expanded with EW_KEEP_AS_ONE rather than field-splitting. */ -bool is_export(t_ast_node word) -{ - t_ast_node c; - - if (word.children.len != 1) - return (false); - c = ((t_ast_node *)word.children.ctx)[0]; - if (c.token.tt != TT_WORD) - return (false); - if (ft_strncmp(c.token.start, "export", c.token.len)) - return (false); - return (true); -} - /* Expand and register a redirect node from a simple command's child list. On error, clears should_unwind (the error is handled by returning the AMBIGUOUS_REDIRECT sentinel, not by propagating a signal-style unwind) diff --git a/src/helpers/casescan.c b/src/helpers/casescan.c index dfffc513..f4ef4853 100644 --- a/src/helpers/casescan.c +++ b/src/helpers/casescan.c @@ -26,6 +26,8 @@ void casescan_init(t_casescan *cs, int len) cs->len = len; cs->ncase = 0; cs->cmdpos = true; + cs->nhd = 0; + cs->arith = -1; } /* End of the alnum/underscore run starting at i — the word the keyword @@ -80,16 +82,14 @@ int casescan_step(t_casescan *cs, const char *s, int *i, int depth) { char c; + if (cs->arith >= 0 && depth <= cs->arith) + cs->arith = -1; + if (casescan_comment(cs, s, i) || casescan_bodies(cs, s, i) + || (cs->arith < 0 && casescan_heredoc_op(cs, s, i))) + return (0); c = s[*i]; - if (c == '(') - return (cs->cmdpos = true, (*i)++, 1); - if (c == ')') - { - (*i)++; - if (cs->ncase > 0 && cs->at[cs->ncase - 1] == depth) - return (cs->cmdpos = true, 0); - return (cs->cmdpos = false, -1); - } + if (c == '(' || c == ')') + return (casescan_paren(cs, s, i, depth)); if (c == ';' || c == '|' || c == '&' || c == '\n') return (cs->cmdpos = true, (*i)++, 0); if (c == ' ' || c == '\t') diff --git a/src/helpers/casescan2.c b/src/helpers/casescan2.c new file mode 100644 index 00000000..753ab22f --- /dev/null +++ b/src/helpers/casescan2.c @@ -0,0 +1,103 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* casescan2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "casescan.h" +#include "libft.h" + +/* Comments and heredoc operators inside a $( ) span (issue #139). The + bodies those operators owe are skipped by casescan3.c. */ + +bool casescan_in(const t_casescan *cs, const char *s, int j) +{ + return ((cs->len < 0 || j < cs->len) && s[j] != '\0'); +} + +/* Does s[i] begin a word? Only there does `#` open a comment: after an + unescaped blank, a newline, `;` `&` `|`, or the `(` of the `$(` itself. + Not after any other `(` (zsh's `(#i)` glob flag, `${(#)x}`), nor after + `)` (`$(a)#b` is one word). The backslash count keeps `\ #x` a word. */ +static bool cs_word_start(const char *s, int i) +{ + char p; + int n; + + p = s[i - 1]; + if (p == '\n' || p == ';' || p == '&' || p == '|') + return (true); + if (p == '(') + return (s[i - 2] == '$'); + if (p != ' ' && p != '\t') + return (false); + n = 0; + while (s[i - 2 - n] == '\\') + n++; + return (n % 2 == 0); +} + +/* A comment runs to the newline, which is left for the caller: it still + ends the command line (and may owe heredoc bodies). */ +bool casescan_comment(t_casescan *cs, const char *s, int *i) +{ + if (s[*i] != '#' || !cs_word_start(s, *i)) + return (false); + while (casescan_in(cs, s, *i) && s[*i] != '\n') + (*i)++; + return (true); +} + +/* End of the delimiter word at s[j]: a quoted span belongs to it + (`<<'E O'`), a backslash takes the next byte, and an unquoted blank + or operator byte ends it. */ +static int cs_hd_word_end(const t_casescan *cs, const char *s, int j) +{ + char q; + + while (casescan_in(cs, s, j) && !ft_strchr(" \t\n;&|<>()", s[j])) + { + if (s[j] == '\'' || s[j] == '"') + { + q = s[j++]; + while (casescan_in(cs, s, j) && s[j] != q) + j++; + } + else if (s[j] == '\\') + j++; + if (casescan_in(cs, s, j)) + j++; + } + return (j); +} + +/* `<cmdpos = false; + if (casescan_in(cs, s, j) && (s[j] == '<' || s[j] == '=')) + return (*i = j + 1, true); + dash = (casescan_in(cs, s, j) && s[j] == '-'); + j += dash; + while (casescan_in(cs, s, j) && (s[j] == ' ' || s[j] == '\t')) + j++; + *i = cs_hd_word_end(cs, s, j); + if (*i > j && cs->nhd < CASESCAN_HD_MAX) + cs->hd[cs->nhd++] = (t_cshd){.word = s + j, .wlen = *i - j, + .dash = dash}; + return (true); +} diff --git a/src/helpers/casescan3.c b/src/helpers/casescan3.c new file mode 100644 index 00000000..6fe11586 --- /dev/null +++ b/src/helpers/casescan3.c @@ -0,0 +1,99 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* casescan3.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "casescan.h" +#include "libft.h" + +/* Heredoc bodies inside a $( ) span, and the parens (issue #139). */ + +/* Is [line, line + n) the delimiter line of h? The word is compared with + its quotes dropped and one backslash level undone -- the rule hd_delim + applies when the body is read, so both agree on where a body ends. */ +static bool cs_delim_is(const t_cshd *h, const char *line, int n) +{ + int a; + int b; + char c; + + a = 0; + b = 0; + while (h->dash && b < n && line[b] == '\t') + b++; + while (a < h->wlen) + { + c = h->word[a++]; + if (c == '\\' && a < h->wlen) + c = h->word[a++]; + else if (c == '\'' || c == '"') + c = '\0'; + if (c && (b >= n || line[b++] != c)) + return (false); + } + return (b == n); +} + +/* Past one body: every line up to and including the delimiter line, or + to the end of the span when that line never comes. */ +static void cs_skip_body(t_casescan *cs, const char *s, int *i, + const t_cshd *h) +{ + int ls; + + while (casescan_in(cs, s, *i)) + { + ls = *i; + while (casescan_in(cs, s, *i) && s[*i] != '\n') + (*i)++; + if (cs_delim_is(h, s + ls, *i - ls)) + { + *i += casescan_in(cs, s, *i); + return ; + } + *i += casescan_in(cs, s, *i); + } +} + +/* The newline that ends a command line: every body owed on it follows, + in operator order, and none of their bytes is shell text. */ +bool casescan_bodies(t_casescan *cs, const char *s, int *i) +{ + int k; + + if (s[*i] != '\n' || cs->nhd == 0) + return (false); + (*i)++; + k = 0; + while (k < cs->nhd) + cs_skip_body(cs, s, i, &cs->hd[k++]); + cs->nhd = 0; + cs->cmdpos = true; + return (true); +} + +/* `(` and `)`: the depth delta, and the case-pattern exception -- a `)` + at the depth where the innermost open `case` started ends a pattern, + not the span. A `(` straight after a `(` opens (( )) or $(( )), whose + `<<` is a shift: the heredoc scan stays off until the depth drops back + below where that span began. */ +int casescan_paren(t_casescan *cs, const char *s, int *i, int depth) +{ + if (s[*i] == '(') + { + if (cs->arith < 0 && s[*i - 1] == '(') + cs->arith = depth - 1; + return (cs->cmdpos = true, (*i)++, 1); + } + (*i)++; + if (cs->ncase > 0 && cs->at[cs->ncase - 1] == depth) + return (cs->cmdpos = true, 0); + return (cs->cmdpos = false, -1); +} diff --git a/src/helpers/free_utils.c b/src/helpers/free_utils.c index 9cd3b287..8953f77a 100644 --- a/src/helpers/free_utils.c +++ b/src/helpers/free_utils.c @@ -153,9 +153,12 @@ static void free_session_data(t_shell *state) free_hist(state); alias_table_free(&state->aliases); + xfree(state->aliases_view[0]); + xfree(state->aliases_view[1]); cmd_hash_free(&state->cmd_cache); zle_widgets_free(); zle_binds_free(); + bind_lines_free(); xfree(state->cwd.ctx); pos_free(&state->pos); free_argv_pool(state); @@ -176,7 +179,9 @@ static void free_session_data(t_shell *state) /* The single canonical shutdown path. Order is deliberate: functions first (they may reference variables), then the owned strings and caches, then per-command scratch (redirects, proc subs, AST), then session data - (history, cwd, positional args, the argv pool, traps). env_index_free() + (history, cwd, positional args, the argv pool, traps), then the `make + ` target cache (a t_make_cache, plain xmalloc'd strings that nothing + above refers to). env_index_free() must come after all env lookups; word_slab_teardown() is last so any stray word_free() calls during the earlier steps still work. alloc_live_report() is the ft_malloc leak oracle -- it's a no-op on libc @@ -191,6 +196,7 @@ void free_all_state(t_shell *state) arr_marks_clear(state); attr_clear(state); free_session_data(state); + mk_cache_free(state); env_index_free(); parena_destroy(); word_slab_teardown(); diff --git a/src/heredoc/collect2.c b/src/heredoc/collect2.c index 9651c066..212460d5 100644 --- a/src/heredoc/collect2.c +++ b/src/heredoc/collect2.c @@ -20,7 +20,12 @@ un-read remainder of rl.buff, capture the RAW body onto the node (so materialize_heredoc expands it at execution time, after any preceding same-line assignment), then advance rl.cursor past what we consumed so the - REPL does not re-read the body lines as commands. */ + REPL does not re-read the body lines as commands -- counting them, as + reading them would have: the next cycle starts on the line after the + delimiter, and $LINENO and every error after it said a line from before. + rl.buff is a length-counted vector, not a C string, and the body scan + stops at a NUL: without the terminator written past len here, a body + whose delimiter never came ran off the end of the buffer (#139). */ bool capture_heredoc_from_buff(t_shell *state, t_ast_node *node) { char *saved_src; @@ -29,13 +34,16 @@ bool capture_heredoc_from_buff(t_shell *state, t_ast_node *node) if (state->metinp != INP_ARG && state->metinp != INP_FILE) return (false); - if (!state->rl.buff.ctx || state->rl.cursor >= state->rl.buff.len) + if (!state->rl.buff.ctx || state->rl.cursor >= state->rl.buff.len + || !vec_ensure_space_n(&state->rl.buff, 1)) return (false); + ((char *)state->rl.buff.ctx)[state->rl.buff.len] = '\0'; saved_src = state->hd_src; saved_pos = state->hd_pos; state->hd_src = (char *)state->rl.buff.ctx + state->rl.cursor; state->hd_pos = 0; ok = capture_heredoc_to_node(state, node); + state->rl.line += nl_count(state->hd_src, state->hd_pos); state->rl.cursor += state->hd_pos; state->hd_src = saved_src; state->hd_pos = saved_pos; diff --git a/src/heredoc/collect4.c b/src/heredoc/collect4.c new file mode 100644 index 00000000..7d098da5 --- /dev/null +++ b/src/heredoc/collect4.c @@ -0,0 +1,91 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* collect4.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "heredoc_private.h" +#include "parena.h" + +/* A heredoc inside a function or loop body is re-read at every call, so + its raw body has to live on the node. When it was not pre-extracted -- + a script's `f() { cat <remove_tabs && i < len && line[i] == '\t') + i++; + if (len > i && line[len - 1] == '\n') + len--; + sl = ft_strlen(req->sep); + return (len - i == sl && ft_strncmp(line + i, req->sep, sl) == 0); +} + +/* Read one raw line onto the body, newline-terminated; true when it was + the delimiter line. End of input sets req->finished (and warns, once, + here at definition time) and returns false. */ +static bool live_line(t_shell *state, t_hdoc *req, t_string *body) +{ + t_string line; + bool delim; + + if (get_line_heredoc(state, req, &line)) + return (xfree(line.ctx), false); + if (line.ctx) + vec_push_nstr(body, (char *)line.ctx, line.len); + if (!line.len || ((char *)line.ctx)[line.len - 1] != '\n') + vec_push_char(body, '\n'); + delim = live_is_delim(req, (char *)line.ctx, line.len); + req->finished = delim; + xfree(line.ctx); + return (delim); +} + +/* The body always ends on a delimiter line, the one read or (input ran + out) one added: materialize_heredoc reads it back through the same + reader at every call, and a body ending on end-of-file would repeat + the warning each time. */ +bool capture_heredoc_live(t_shell *state, t_ast_node *node) +{ + t_string sep; + t_hdoc req; + t_string body; + bool seen; + + sep = word_to_hrdoc_string(((t_ast_node *)node->children.ctx)[1]); + if (!sep.ctx || !vec_ensure_space_n(&sep, 1)) + return (xfree(sep.ctx), false); + ((char *)sep.ctx)[sep.len] = '\0'; + req = (t_hdoc){.sep = (char *)sep.ctx, .remove_tabs = heredoc_op_strips( + ((t_ast_node *)node->children.ctx)[0].token.start)}; + vec_init(&body); + body.elem_size = 1; + seen = false; + state->rl.line_exact = true; + while (!req.finished) + seen = live_line(state, &req, &body); + state->rl.line_exact = false; + if (!seen) + (vec_push_str(&body, (char *)sep.ctx), vec_push_char(&body, '\n')); + vec_push_char(&body, '\0'); + xfree(sep.ctx); + node->heredoc_body = (char *)body.ctx; + parena_note_attach(); + return (true); +} diff --git a/src/heredoc/extract2.c b/src/heredoc/extract2.c index c7e4bb69..b9baf7a7 100644 --- a/src/heredoc/extract2.c +++ b/src/heredoc/extract2.c @@ -50,11 +50,20 @@ static bool delim_present(const char *p, t_hd *s) /* Collect every heredoc body whose operator sits on the current line. ln pins the operator line so multiple specs sharing it (`cat <si < c->n && c->sp[c->si].line < ln) @@ -62,7 +71,13 @@ static void advance_hd(const char **p, size_t *cur, while (c->si < c->n && c->sp[c->si].line == ln) { if (delim_present(*p, &c->sp[c->si])) + { + from = *p; c->got += (collect_body(p, cur, &out[1], &c->sp[c->si]), 1); + while (from < *p) + if (*from++ == '\n') + vec_push_nstr(&out[0], "\n", 1); + } c->si++; } } @@ -70,9 +85,10 @@ static void advance_hd(const char **p, size_t *cur, /* Walk the source line by line: emit each line into out[0] (the stripped source for the parser), and after each line call advance_hd to pull any heredoc bodies that start on that line into out[1]. The result is a - parser-safe source string (no heredoc bodies) plus a packed body stream - in source order. Returns the count of heredoc bodies actually extracted - (some operators may not have their delimiter present in the string). */ + parser-safe source string (no heredoc bodies, an empty line for each of + their lines) plus a packed body stream in source order. Returns the + count of heredoc bodies actually extracted (some operators may not have + their delimiter present in the string). */ static int walk_and_strip(const char *str, t_hd *sp, int n, t_string *out) { const char *p; diff --git a/src/heredoc/extract3.c b/src/heredoc/extract3.c index de732a35..7b5cdfa6 100644 --- a/src/heredoc/extract3.c +++ b/src/heredoc/extract3.c @@ -52,39 +52,51 @@ static void push_spec(t_vec *v, t_deque_tok *tt, size_t i, size_t line) vec_push(v, &sp); } -/* Tokenise a single command line (copied so its NUL terminates the lexer) and - append a spec for every `<deqtok.len) + { + if (((t_ltoken *)deque_idx(&tt->deqtok, i))->tt == TT_HEREDOC) + (push_spec(v, tt, i, line), found++); + i++; + } + return (found); +} + +/* Tokenise the command line at *p -- grown across physical lines while a + quote or substitution is open (extract5.c) -- append a spec for every + `<tt == TT_HEREDOC) - (push_spec(v, &tt, i, line), found++); - i++; - } + if (copy) + found = push_specs(&tt, *line, v); + *p = end; return (xfree(tt.deqtok.buff), xfree(copy), found); } -/* Walk the source line by line, collecting heredoc specs and skipping each - heredoc's body so its bytes are never lexed. *out receives the spec array. */ +/* Walk the source command line by command line, collecting heredoc specs and + skipping each heredoc's body so its bytes are never lexed. *out receives + the spec array. */ int collect_specs(const char *str, t_hd **out) { t_vec v; const char *p; - const char *ls; size_t line; int k; @@ -94,11 +106,8 @@ int collect_specs(const char *str, t_hd **out) line = 0; while (*p) { - ls = p; - while (*p && *p != '\n') - p++; - k = specs_on_line(ls, p - ls + (*p == '\n'), line++, &v); - p += (*p == '\n'); + k = specs_on_line(&p, &line, &v); + line++; while (k-- > 0 && *p) skip_one_body(&p, &line, &((t_hd *)v.ctx)[v.len - 1 - k]); } diff --git a/src/heredoc/extract4.c b/src/heredoc/extract4.c index 4d2bc3f1..3e99f29e 100644 --- a/src/heredoc/extract4.c +++ b/src/heredoc/extract4.c @@ -74,7 +74,7 @@ bool heredoc_incomplete(const char *str) { t_vec v; const char *p; - const char *ls; + size_t line; int k; if (!str || !ft_strnstr(str, "<<", ft_strlen(str))) @@ -82,13 +82,10 @@ bool heredoc_incomplete(const char *str) vec_init(&v); v.elem_size = sizeof(t_hd); p = str; + line = 0; while (*p) { - ls = p; - while (*p && *p != '\n') - p++; - k = specs_on_line(ls, p - ls + (*p == '\n'), v.len, &v); - p += (*p == '\n'); + k = specs_on_line(&p, &line, &v); if (k > 0 && !line_bodies_ok(&p, &v, k)) return (free_hd_vec(&v), true); } diff --git a/src/heredoc/extract5.c b/src/heredoc/extract5.c new file mode 100644 index 00000000..8e14d12c --- /dev/null +++ b/src/heredoc/extract5.c @@ -0,0 +1,77 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* extract5.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/24 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/24 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +/* The command line a `<<` sits on is a LOGICAL line. A heredoc body starts + after the newline TOKEN that ends its command, and a newline inside a + quote, `...`, $( ) or ${ } is word text, not that token: bash reads the + body of `cat <b'` after the `b'` line. Lexing one + physical line alone (the old way) read `b' | cat <looking_for = 0; + open = tokenizer(copy, tt); + if (!open || !**end) + return (copy); + xfree(copy); + hd_join(end, line, tt->looking_for); + } +} diff --git a/src/heredoc/helpers1.c b/src/heredoc/helpers1.c index 529a6f38..581a00d7 100644 --- a/src/heredoc/helpers1.c +++ b/src/heredoc/helpers1.c @@ -15,7 +15,10 @@ /* Try to defer this heredoc's materialisation to execution time: from a stored body (functions / pre-extracted hd_src) or, for a same-line -c/script heredoc, straight from the unread remainder of rl.buff. Returns true if the - body was captured onto the node (caller skips eager temp-file writing). */ + body was captured onto the node (caller skips eager temp-file writing). + Inside a function or loop body the capture MUST happen: an eager write + lands in this command's redirect slot, which is gone by the first call, + so with nothing pre-extracted the body is read from the live input. */ static bool try_defer_heredoc(t_shell *state, t_ast_node *curr, bool is_pipeline) { @@ -25,6 +28,8 @@ static bool try_defer_heredoc(t_shell *state, t_ast_node *curr, if (!is_pipeline && !state->gather_in_func && !state->hd_src && capture_heredoc_from_buff(state, curr)) return (true); + if (state->gather_in_func && !state->hd_src) + return (capture_heredoc_live(state, curr)); return (false); } diff --git a/src/heredoc/heredoc_private.h b/src/heredoc/heredoc_private.h index c0f1c785..549618e8 100644 --- a/src/heredoc/heredoc_private.h +++ b/src/heredoc/heredoc_private.h @@ -19,6 +19,7 @@ # include "redir.h" # include "env.h" # include "expander.h" +# include "lexer.h" # include # include "helpers.h" # include "pal_wait.h" @@ -68,8 +69,11 @@ void hdoc_attach_backing(t_shell *state, int idx, t_string *body); bool contains_quotes(t_ast_node node); bool is_delim_line(const char *line, size_t len, t_hd *s); bool split_heredocs(const char *str, char **stripped, char **bodies); -int specs_on_line(const char *ls, size_t len, size_t line, t_vec *v); +int specs_on_line(const char **p, size_t *line, t_vec *v); +char *hd_lex_line(const char *ls, const char **end, size_t *line, + t_deque_tok *tt); bool heredoc_incomplete(const char *str); bool heredoc_op_strips(const char *op); +bool capture_heredoc_live(t_shell *state, t_ast_node *node); #endif \ No newline at end of file diff --git a/src/heredoc/reader.c b/src/heredoc/reader.c index fcbf5316..d703e5aa 100644 --- a/src/heredoc/reader.c +++ b/src/heredoc/reader.c @@ -32,7 +32,9 @@ static int read_hd_src_line(t_shell *state, t_string *out) /* Read one line of heredoc body. Source priority: if hd_src is set (we are executing a string -- eval/source/cmdsub) we serve from the - pre-extracted body stream; otherwise we call the readline layer. EOF + pre-extracted body stream; otherwise we call the readline layer, with + PS2 as the prompt when one is set (bash's here-document prompt) and + "heredoc> " when not. EOF (stat==0) or Ctrl-D (stat==2) trigger the POSIX "delimited by EOF" warning and mark req->finished so the caller loop terminates. A stat of 4 from read_hd_src_line means "got a line". */ @@ -42,25 +44,25 @@ bool get_line_heredoc(t_shell *state, int stat; char *prompt; - if (req->is_pipe_heredoc) - prompt = "pipe heredoc> "; - else - prompt = "heredoc> "; vec_init(alloc_line); if (state->hd_src) stat = read_hd_src_line(state, alloc_line); else + { + prompt = "heredoc> "; + if (req->is_pipe_heredoc) + prompt = "pipe heredoc> "; + prompt = prompt_ps2(state, prompt); stat = buff_readline(state, alloc_line, prompt); + xfree(prompt); + } state->rl.has_finished = false; if (stat == 0) ft_eprintf("%s: warning: here-document at" " line %i delimited by end-of-file (wanted `%s')\n", state->ctx, state->rl.line, req->sep); if (stat == 0 || stat == 2) - { - req->finished = true; - return (true); - } + return (req->finished = true, true); return (false); } diff --git a/src/infrastructure/README.md b/src/infrastructure/README.md index 29e02364..22ac9e3d 100644 --- a/src/infrastructure/README.md +++ b/src/infrastructure/README.md @@ -308,6 +308,18 @@ commands from the newline that terminates a command record. The history subsystem therefore maintains a **vector of strings in memory** and an **append‑only file** on disk. +The append descriptor (`history_fd.c`) is the shell's own, not the user's: + +- `hist_open_append` opens it `O_CLOEXEC` and moves it to fd ≥ 10 (as + `save_fd` does), so no program the shell runs inherits the history file + and it stays out of the 3..9 range scripts use; +- it also records the file's device and inode. `hist_append_fd` checks them + before each write. If the user has since redirected that number + (`exec 10>f`) or closed it, the number is theirs: it is not written to or + closed, and the history file is opened again; +- `hist_close_append` closes the descriptor only while it is still the + history file (`hist_rehome` uses it). + ### 5.4 Initialization and teardown - `init_history` sets default flags and reads existing history from disk. diff --git a/src/infrastructure/error_where.c b/src/infrastructure/error_where.c index a384d9e5..d0f314fe 100644 --- a/src/infrastructure/error_where.c +++ b/src/infrastructure/error_where.c @@ -14,6 +14,7 @@ #include "lexer.h" #include "sh_error.h" #include "prompt.h" +#include "sh_input.h" /* Where a parse error is, the way bash says it for a sourced file. ** @@ -38,6 +39,13 @@ ** plus the newlines before the token inside the chunk. */ +/* At a prompt -- a terminal session, or -i -- bash leads a file's + "FILE: line N" with its own name; from a script or -c it does not. */ +bool sh_interactive(t_shell *state) +{ + return (state->opt_interactive || state->metinp == INP_RL); +} + /* Owned: "ctx", or "hellish: FILE: line N" when a file is being sourced. `at` points into tt->base, the text the tokens index. The shell's own name leads only when interactive -- bash prints "bash: FILE: line N" at @@ -49,7 +57,7 @@ char *parse_err_ctx(t_shell *state, t_deque_tok *tt, const char *at) if (!state->err_src || !tt || !tt->base || !at || at < tt->base) return (ft_strdup(state->ctx)); line = state->err_line + nl_count(tt->base, (size_t)(at - tt->base)); - if (!state->opt_interactive) + if (!sh_interactive(state)) return (ft_asprintf("%s: line %d", state->err_src, line)); return (ft_asprintf("%s: %s: line %d", state->dft_ctx, state->err_src, line)); diff --git a/src/infrastructure/history2.c b/src/infrastructure/history2.c index 30a307ee..56bd6348 100644 --- a/src/infrastructure/history2.c +++ b/src/infrastructure/history2.c @@ -89,7 +89,7 @@ void parse_history_file(t_shell *state) xfree(hist.ctx); trimmed = cap_history(state, &state->hist.hist_cmds); load_and_persist(state, path, trimmed); - state->hist.append_fd = open(path, O_CREAT | O_WRONLY | O_APPEND, 0600); + hist_open_append(state, path); if (state->hist.append_fd < 0) warning_error("Can't open the history file for writing"); xfree(state->hist.file); diff --git a/src/infrastructure/history_fd.c b/src/infrastructure/history_fd.c new file mode 100644 index 00000000..fd23096b --- /dev/null +++ b/src/infrastructure/history_fd.c @@ -0,0 +1,91 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* history_fd.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "history_private.h" +#include + +/* The descriptor this session streams its history into. +** +** It shares the fd table with the user's own descriptors, and it used to +** be opened the plain way: the lowest free number, 3, inherited by every +** program the shell ran (`ls -l /proc/self/fd` listed the history file), +** and sitting in the 3..9 range scripts claim for themselves. After +** `exec 3>f` the history went on into f -- the user's file -- and after +** `exec 3>&-` it went nowhere for the rest of the session. +** +** So it is opened close-on-exec, moved out of that range the way save_fd +** moves the executor's copies, and remembered by identity (device and +** inode) as well as by number: a number the user has since redirected or +** closed is theirs, never written to or closed by the shell, and the +** history file is simply opened again. */ + +/* Open path for appending, as the session's history descriptor. On any + failure append_fd is -1 and the session records in memory only. */ +void hist_open_append(t_shell *state, const char *path) +{ + int fd; + int high; + struct stat st; + + state->hist.append_fd = -1; + fd = open(path, O_CREAT | O_WRONLY | O_APPEND | O_CLOEXEC, 0600); + if (fd < 0) + return ; + high = fcntl(fd, F_DUPFD_CLOEXEC, 10); + if (high >= 0) + { + close(fd); + fd = high; + } + if (fstat(fd, &st) < 0) + { + close(fd); + return ; + } + state->hist.append_fd = fd; + state->hist.append_dev = st.st_dev; + state->hist.append_ino = st.st_ino; +} + +/* True while append_fd still refers to the file it was opened on. */ +static bool hist_fd_ours(t_history *h) +{ + struct stat st; + + if (h->append_fd < 0 || fstat(h->append_fd, &st) < 0) + return (false); + return (st.st_dev == h->append_dev && st.st_ino == h->append_ino); +} + +/* The descriptor to append the next entry to, or -1: append_fd while it + is still the history file, else a fresh one on the same path. One + fstat per recorded command. */ +int hist_append_fd(t_shell *state) +{ + if (state->hist.append_fd < 0 || hist_fd_ours(&state->hist)) + return (state->hist.append_fd); + if (!state->hist.file) + { + state->hist.append_fd = -1; + return (-1); + } + hist_open_append(state, state->hist.file); + return (state->hist.append_fd); +} + +/* Stop streaming: close append_fd only if the number is still ours. */ +void hist_close_append(t_shell *state) +{ + if (hist_fd_ours(&state->hist)) + close(state->hist.append_fd); + state->hist.append_fd = -1; +} diff --git a/src/infrastructure/history_file.c b/src/infrastructure/history_file.c index e43c6e4e..9671e304 100644 --- a/src/infrastructure/history_file.c +++ b/src/infrastructure/history_file.c @@ -45,9 +45,7 @@ void hist_rehome(t_shell *state) if (!ft_strcmp(want, state->hist.file)) return (xfree(want)); xfree(want); - if (state->hist.append_fd >= 0) - close(state->hist.append_fd); - state->hist.append_fd = -1; + hist_close_append(state); free_hist(state); hist_rl_clear(); parse_history_file(state); diff --git a/src/infrastructure/history_private.h b/src/infrastructure/history_private.h index 8eda5b72..2b95b2a1 100644 --- a/src/infrastructure/history_private.h +++ b/src/infrastructure/history_private.h @@ -103,4 +103,11 @@ void hist_trim_to_limit(t_shell *state); void append_hist_entry(t_shell *state, char *hist_entry); void history_record(t_shell *state, bool early); +/* The session's history descriptor (history_fd.c): close-on-exec, out of + the 3..9 range, and never written to once the user has taken its number + over -- hist_append_fd opens the file again instead. */ +void hist_open_append(t_shell *state, const char *path); +int hist_append_fd(t_shell *state); +void hist_close_append(t_shell *state); + #endif \ No newline at end of file diff --git a/src/infrastructure/history_record.c b/src/infrastructure/history_record.c index e5e85338..1f0e4acc 100644 --- a/src/infrastructure/history_record.c +++ b/src/infrastructure/history_record.c @@ -32,21 +32,24 @@ /* Push one entry into readline's list, our vector and the file, then apply the HISTCONTROL and HISTSIZE rules that can retire older entries. The file write happens before those, because erasedups and the size cap may - free the very string being written. */ + free the very string being written. The descriptor comes from + hist_append_fd, never straight from append_fd: that number may be the + user's by now. */ void append_hist_entry(t_shell *state, char *hist_entry) { char *enc; + int fd; add_history(hist_entry); vec_push(&state->hist.hist_cmds, &hist_entry); - if (state->hist.append_fd >= 0) + fd = hist_append_fd(state); + if (fd >= 0) { enc = (char *)encode_cmd_hist(hist_entry).ctx; - if (write_to_file(enc, state->hist.append_fd)) + if (write_to_file(enc, fd)) { warning_error("Failed to write to the history file"); - close(state->hist.append_fd); - state->hist.append_fd = -1; + hist_close_append(state); } xfree(enc); } diff --git a/src/infrastructure/input.c b/src/infrastructure/input.c index f0a5ad71..797ea012 100644 --- a/src/infrastructure/input.c +++ b/src/infrastructure/input.c @@ -126,10 +126,10 @@ static void update_prompt(t_shell *state, char **prompt, t_deque_tok *tt) else *prompt = tokenizer((char *)state->alias_exp.ctx, tt); if (*prompt) - *prompt = ft_strdup(*prompt); + *prompt = prompt_ps2(state, *prompt); else if (state->gathering_compound && state->input.ctx && heredoc_incomplete((char *)state->input.ctx)) - *prompt = ft_strdup("> "); + *prompt = prompt_ps2(state, "> "); } /* Keep reading lines until the tokenizer stops asking for more (prompt becomes diff --git a/src/infrastructure/input_utils3.c b/src/infrastructure/input_utils3.c index bc23566c..ac6b4f0d 100644 --- a/src/infrastructure/input_utils3.c +++ b/src/infrastructure/input_utils3.c @@ -42,8 +42,9 @@ bool ends_with_bs_nl(t_string s) } /* Strip trailing \ continuations by popping the last two bytes (the - backslash and the newline) and reading another line with a ">" prompt. This - repeats until no continuation remains, so "echo hello \\" works. */ + backslash and the newline) and reading another line with PS2 (">" when it + is unset). This repeats until no continuation remains, so + "echo hello \\" works. */ void extend_bs(t_shell *state) { char *prompt; @@ -52,7 +53,7 @@ void extend_bs(t_shell *state) { vec_pop(&state->input); vec_pop(&state->input); - prompt = ft_strdup("> "); + prompt = prompt_ps2(state, "> "); if (readline_cmd(state, &prompt)) return ; } diff --git a/src/infrastructure/input_utils4.c b/src/infrastructure/input_utils4.c index fd4ffbd7..1d9c51dc 100644 --- a/src/infrastructure/input_utils4.c +++ b/src/infrastructure/input_utils4.c @@ -48,6 +48,7 @@ static void extract_input_heredocs(t_shell *state, t_deque_tok *tt) state->hd_pos = 0; xfree(state->hd_stripped); state->hd_stripped = NULL; + state->hd_stripped_len = 0; in = (char *)state->alias_exp.ctx; state->cycle_has_hd = (in && ft_strnstr(in, "<<", state->alias_exp.len) != NULL); @@ -57,6 +58,7 @@ static void extract_input_heredocs(t_shell *state, t_deque_tok *tt) return ; state->hd_src = bodies; state->hd_stripped = stripped; + state->hd_stripped_len = ft_strlen(stripped); tokenizer(stripped, tt); } @@ -106,7 +108,7 @@ bool try_parse_tokens(t_shell *state, t_parser *parser, if (parser->res == RES_OK) return (true); else if (parser->res == RES_GETMOREINPUT) - *prompt = (char *)prompt_more_input(state, parser).ctx; + *prompt = prompt_more_input(state, parser); else if (parser->res == RES_ERR) abort_on_syntax_error(state); return (free_ast(&state->tree), true); diff --git a/src/infrastructure/mascot_redraw.c b/src/infrastructure/mascot_redraw.c index e0ab1812..52cc71ef 100644 --- a/src/infrastructure/mascot_redraw.c +++ b/src/infrastructure/mascot_redraw.c @@ -94,6 +94,6 @@ void redraw_mascot(t_string *r) while (s + i < last) rd_char(&f, s[i++]); vec_push_str(&f, "\0338\033[?25h\033[?2026l"); - tty_write_all(fileno(rl_outstream), f.ctx, f.len); + rl_out_write(f.ctx, f.len); xfree(f.ctx); } diff --git a/src/infrastructure/prompt.c b/src/infrastructure/prompt.c index 99c631df..7ce55a13 100644 --- a/src/infrastructure/prompt.c +++ b/src/infrastructure/prompt.c @@ -40,23 +40,20 @@ static const char *prompt_label(t_tt curr) return (NULL); } -/* Build the continuation prompt. A user-set PS2 (rc file or export) wins - and renders with the full bash escape set; otherwise the built-in - open-construct stack view: "if > ", "if while > ", etc. — the last - space is replaced by '>' to give it the feel of a depth indicator. An - empty stack (shouldn't happen here, but guarded) just produces "> ". */ -t_string prompt_more_input(t_shell *state, t_parser *parser) +/* Build the continuation prompt for an unfinished compound command. The + built-in label is the open-construct stack view: "if> ", "if while> ", + etc. -- the last space is replaced by '>' to give it the feel of a + depth indicator; an empty stack (shouldn't happen here, but guarded) + just produces "> ". prompt_ps2 decides whether a user-set PS2 replaces + it, the same way for every continuation line. */ +char *prompt_more_input(t_shell *state, t_parser *parser) { t_string ret; t_tt curr; size_t i; const char *label; + char *prompt; - rprompt_clear(state); - state->rl.ps1_read = false; - label = env_expand(state, "PS2"); - if (label && *label) - return (ps1_render(state, label)); i = -1; vec_init(&ret); ret.elem_size = 1; @@ -71,7 +68,10 @@ t_string prompt_more_input(t_shell *state, t_parser *parser) } if (ret.len > 0) ((char *)ret.ctx)[ret.len - 1] = '>'; - return (vec_push_str(&ret, " "), ret); + vec_push_str(&ret, " "); + prompt = prompt_ps2(state, (char *)ret.ctx); + xfree(ret.ctx); + return (prompt); } /* Build the whole prompt for a given animation frame and last exit status: the diff --git a/src/infrastructure/prompt_expand.c b/src/infrastructure/prompt_expand.c new file mode 100644 index 00000000..49532e31 --- /dev/null +++ b/src/infrastructure/prompt_expand.c @@ -0,0 +1,61 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* prompt_expand.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: marvin +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/26 00:30:00 by marvin #+# #+# */ +/* Updated: 2026/09/26 00:30:00 by marvin ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "prompt_private.h" +#include "sh_input.h" +#include + +/* ${var@P}: the value of var rendered as a prompt string, bash's way to + ask the shell what a prompt looks like (issue #134, section 1). + + It is the PS1 rule and nothing else: the one prompt_normal applies to + PS1 -- zsh_to_ps1 in the mode the dialect bit selects, then + ps1_render. So `${PS1@P}` is what the prompt shows, including what it + does NOT do: a variable's value is not scanned again for escapes, in + either language (V='100%done' stays 100%done, V='\w' stays \w), which + is also bash's order. print -P is zsh's and does scan again, as zsh's + does; a preview of a PS1 theme built on it drew prompts that never + appear. + + ps1_render, not ps1_animated: the animation cells belong to the live + prompt, and an @P inside a PS1 or a precmd hook must not rebuild them + from another string. + + \[ and \] become readline's zero-width markers, \001 and \002. Without + a line editor bash drops them, so they are stripped here unless the + read is interactive -- or HELLISH_DBG_PROMPT_MARKS asks for them, the + window print -P gives the width tests too. Returns a new string. */ +char *prompt_expand_p(t_shell *state, const char *fmt) +{ + t_string conv; + t_string out; + char *s; + size_t i; + size_t j; + + conv = zsh_to_ps1(state, fmt, zsh_mode(state)); + out = ps1_render(state, (char *)conv.ctx); + xfree(conv.ctx); + s = (char *)out.ctx; + if (state->metinp == INP_RL || getenv("HELLISH_DBG_PROMPT_MARKS")) + return (s); + i = 0; + j = 0; + while (s[i]) + { + if (s[i] != '\001' && s[i] != '\002') + s[j++] = s[i]; + i++; + } + s[j] = '\0'; + return (s); +} diff --git a/src/infrastructure/prompt_private.h b/src/infrastructure/prompt_private.h index 672ad621..c2429cb0 100644 --- a/src/infrastructure/prompt_private.h +++ b/src/infrastructure/prompt_private.h @@ -152,6 +152,7 @@ typedef struct s_dcache void vec_push_ansi(t_string *v, const char *seq); void tty_write_all(int fd, const char *buf, size_t len); +void rl_out_write(const char *buf, size_t len); int anim_style(t_shell *state); /* Seconds the pending-update badge trusts its cached read of the update @@ -188,7 +189,7 @@ void push_user_seg(t_string *ret, t_prompt *p); void push_cwd_seg(t_string *ret, t_prompt *p); char *get_venv_name(void); void get_timebuf(char *buf, size_t buflen); -t_string prompt_more_input(t_shell *state, t_parser *parser); +char *prompt_more_input(t_shell *state, t_parser *parser); t_string prompt_normal(t_shell *state); t_string ps1_render(t_shell *state, const char *fmt); void ps1_host(t_string *out, char kind); diff --git a/src/infrastructure/prompt_ps2.c b/src/infrastructure/prompt_ps2.c new file mode 100644 index 00000000..57983981 --- /dev/null +++ b/src/infrastructure/prompt_ps2.c @@ -0,0 +1,47 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* prompt_ps2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: marvin +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 19:00:00 by marvin #+# #+# */ +/* Updated: 2026/09/25 19:00:00 by marvin ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "prompt_private.h" +#include "sh_input.h" +#include "env.h" + +/* The prompt for a continuation line, every kind of it: an open quote, a + backslash-newline, a heredoc body, an unfinished compound command. In + bash PS2 is the one continuation prompt, so a user-set PS2 wins + everywhere and renders with the full escape set; unset, `fallback` is + the built-in label naming what is still open ("dquote> ", "if> "). + It used to be read only for an unfinished compound: the lexer and the + heredoc reader handed readline a fixed label, so PS2 looked ignored. + + Only an interactive read shows a prompt, so only it renders PS2: a + script pays nothing for it, and a PS2 with a side effect ($((n+=1))) + runs no more often than in bash, which never expands it there. + + A continuation row is not the primary prompt. The right prompt is + dropped (zsh paints none there) and ps1_read goes false, which keeps + the idle repaint -- it rewrites the PS1 rows above the cursor -- off + this row. The lexer paths skipped both, so RPROMPT was painted after + "dquote> ". Returns a new string for readline_cmd to free. */ +char *prompt_ps2(t_shell *state, const char *fallback) +{ + const char *ps2; + + rprompt_clear(state); + state->rl.ps1_read = false; + if (state->metinp == INP_RL) + { + ps2 = env_expand(state, "PS2"); + if (ps2 && *ps2) + return ((char *)ps1_render(state, ps2).ctx); + } + return (ft_strdup(fallback)); +} diff --git a/src/infrastructure/prompt_rprompt.c b/src/infrastructure/prompt_rprompt.c index ef58b2a9..7a3a0324 100644 --- a/src/infrastructure/prompt_rprompt.c +++ b/src/infrastructure/prompt_rprompt.c @@ -86,8 +86,9 @@ static void rp_strip(t_string *txt) } /* Forget the rendered right prompt: nothing gets painted until the next - primary prompt renders one. prompt_more_input calls this too -- zsh - shows no right prompt on a continuation row, and neither do we. */ + primary prompt renders one. prompt_ps2 calls this too, for every kind + of continuation row -- zsh shows no right prompt there, and neither + do we. */ void rprompt_clear(t_shell *state) { if (state->rl.rp_txt.ctx) diff --git a/src/infrastructure/rl_helpers.c b/src/infrastructure/rl_helpers.c index 3ff161b8..a94ed677 100644 --- a/src/infrastructure/rl_helpers.c +++ b/src/infrastructure/rl_helpers.c @@ -62,7 +62,10 @@ void buff_readline_init(t_rl *ret) /* Update the error-message context string to reflect the current line number. Bash writes "script: line N:" in error output — we mirror that here. - Only runs when should_update_ctx is set (non-interactive script mode). */ + Only runs when should_update_ctx is set (non-interactive script mode). + This is the READER's line, right for an error found while reading; a + command that runs later moves ctx to its own line (exec_where.c), and + ctx_src/ctx_line are kept in step so that knows what ctx says. */ void update_ctx(t_shell *state) { if (!state->rl.should_update_ctx) @@ -70,6 +73,9 @@ void update_ctx(t_shell *state) xfree(state->ctx); state->ctx = (char *)ft_asprintf("%s: line %i", state->dft_ctx, state->rl.line); + state->ctx_src = NULL; + state->ctx_line = state->rl.line; + state->ctx_num = 0; } /* Non-TTY path: read raw bytes straight from fd 0 (piped script). We stop on diff --git a/src/infrastructure/rl_private.h b/src/infrastructure/rl_private.h index 039755ca..97d7f67f 100644 --- a/src/infrastructure/rl_private.h +++ b/src/infrastructure/rl_private.h @@ -43,6 +43,69 @@ void update_ctx(t_shell *state); int get_more_input_notty(t_shell *state); int visible_width_cstr(const char *s); void rl_preinit(t_rl *l); +/* Keys readline binds to something of ours only while a feature is on + (rl_slot.c). A slot is one key sequence in one keymap; `fn` names the + stock function it holds and the one that replaces it. */ +# define RL_NFEAT 2 +# define RL_NSLOTS 17 + +enum e_feat +{ + FEAT_HIST, + FEAT_ACCEPT +}; + +enum e_slot_fn +{ + SLOT_PREV, + SLOT_NEXT, + SLOT_FWD, + SLOT_END +}; + +typedef struct s_slot +{ + const char *key; + int feat; + int vi; + int fn; +} t_slot; + +t_slot rl_slot_at(int i); +void rl_slot_sync(t_shell *state); +void rl_slot_pre(t_rl *l); +void rl_slot_post(t_rl *l); +void rl_slot_arm(t_rl *l, int i); +void rl_slot_settle(t_rl *l, int i); + +/* The inline suggestion (rl_suggest*.c): the ghost text after the cursor, + painted by the redisplay function of rl_rprompt.c. */ +typedef struct s_frame +{ + const char *ghost; + size_t gn; + int end; + int cols; + bool show; +} t_frame; + +char *rl_suggest_update(t_shell *state, int cols); +char *rl_suggest_live(t_rl *l); +char *rl_suggest_find(t_shell *state, t_rl *l); +int rl_suggest_accept(int count, int key); +void rl_suggest_end(t_shell *state); +void rl_suggest_stale(void); +void rl_suggest_unpaint(void); +bool rl_suggest_plain(const char *s); +bool rl_suggest_gate(t_rl *l, int cols); +size_t rl_ghost_fit(const char *s, int avail, int *used); +int rl_prompt_cols(t_rl *l); +void rl_rprompt_hook(void); +int rl_text_end(t_rl *l); +int rl_text_col(t_rl *l); +bool rl_rp_fits(t_rl *l, int cols, int end); +void rl_frame(t_rl *l, t_frame *f); +bool rl_push_col(t_string *f, int n); /* The line reader (src/platform/posix/rl_getc.c, rl_idle.c, rl_sig.c). */ # ifndef READERR @@ -86,6 +149,7 @@ typedef struct s_rl_bracket void rl_shell_enter(t_shell *state, t_rl_bracket *b); int rl_shell_leave(t_shell *state, t_rl_bracket *b); void rl_prerow_arm(t_string *rows); +FILE *rl_out_open(void); int rl_getc_hook(FILE *stream); int rl_idle_timeout(void); int rl_idle_fd(void); @@ -97,4 +161,16 @@ bool rl_should_abort(void); int rl_abort_value(void); void rl_abort_reset(void); +/* The terminal opened a second time, non-blocking, to read keys from + (rl_keyfd.c): `twin` reads the keys of `src`, the device `rdev`. */ +typedef struct s_rl_keyfd +{ + int twin; + int src; + dev_t rdev; +} t_rl_keyfd; + +int rl_keyfd(int fd); +void rl_keyfd_refresh(int fd); + #endif \ No newline at end of file diff --git a/src/job_control/job_signal2.c b/src/job_control/job_signal2.c index dfd198cc..2f5e4f35 100644 --- a/src/job_control/job_signal2.c +++ b/src/job_control/job_signal2.c @@ -61,7 +61,9 @@ static bool sig_worth_announcing(int sig) } /* The unsolicited stderr line bash writes when it NOTICES a background job - died from a signal (issue #17) -- "script: line 5: 1234 Killed sleep 9". + died from a signal (issue #17) -- "script: line 5: 1234 Killed sleep 9". + The pid is right-aligned in five columns, as bash's print_pipeline has it + (%5ld), so a pid under 10000 gets padded like `jobs -l` pads it. Three conditions, all of them load-bearing: @@ -96,7 +98,7 @@ void job_notify_async(t_shell *state) if (job->status == JOB_KILLED && !job->notified && sig_worth_announcing(job->term_sig)) { - ft_eprintf("%s: %d %-27s%s%s\n", state->ctx, job->pgid, + ft_eprintf("%s: %5d %-27s%s%s\n", state->ctx, job->pgid, job_status_desc(job), job_core_suffix(job), job->cmd); job->notified = true; job_remove(jt, job->id); diff --git a/src/lexer/README.md b/src/lexer/README.md index 1ce5154f..84ba10e9 100644 --- a/src/lexer/README.md +++ b/src/lexer/README.md @@ -67,8 +67,11 @@ Operator recognition used to walk a table on every call, with dozens of `parse_op()`: 1. tries `check_fd_redirect()` (`helper4.c`) for the fd-prefixed forms - (`2>`, `10<`, `3>&`, `2>>`), with `fd_redir_type` picking the longest - form first; + (`2>`, `10<`, `255>&`, `2>>`), with `fd_redir_type` picking the longest + form first. `tokenize_step` routes digits here only when they are an + IO_NUMBER (`io_number_at`): a whole token of digits, at most `INT_MAX`, + delimited by `<` or `>`, and not the operand of a `>&`/`<&` just before + it (`>&2>f` is `>&2` then `>f`, as in bash); 2. otherwise routes on the leading byte to `op_left` (`<` family: `<<<`, `<<-`, `<<`, `<(`, `<&`, `<>`, `<`), `op_right` (`>>`, `>(`, `>&`, `>|`, `>`), or `op_other` (`||`, `|`, the `&` family via `op_amp` -- `&&`, @@ -88,12 +91,13 @@ exists. ### 4.1 Boundaries (`helper2.c`) `is_word_boundary(s)` consults a 256-entry character-class table (`g_cl`: -metacharacter / blank / digit bits) instead of an `ft_strchr` walk -- these +metacharacter / blank bits) instead of an `ft_strchr` walk -- these predicates run over a million times on a large parse. A word ends on a -metacharacter, a blank (space or tab only; newline is a token), or a -fd-redirect start (`is_fd_redirect_start`: one or two digits then `<`/`>`), -so `echo2>file` is `echo` + `2>` + `file`. `is_space` and -`is_special_char` read the same table. +metacharacter or a blank (space or tab only; newline is a token), and on +nothing else: digits before a `<`/`>` do not end one, since an fd number is +a whole token or nothing (POSIX 2.10.1). So `echo2>file` is the word +`echo2` redirected to `file`, and `x=2>f` assigns `2`, as in bash. +`is_space` and `is_special_char` read the same table. ### 4.2 The word machine (`parse_lexeme.c`) diff --git a/src/lexer/helper2.c b/src/lexer/helper2.c index b4250d24..4960bd07 100644 --- a/src/lexer/helper2.c +++ b/src/lexer/helper2.c @@ -6,7 +6,7 @@ /* By: marvin +#+ +:+ +#+ */ /* +#+#+#+#+#+ +#+ */ /* Created: 2026/01/18 23:05:17 by marvin #+# #+# */ -/* Updated: 2026/07/21 00:00:00 by dlesieur ### ########.fr */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ /* */ /* ************************************************************************** */ @@ -15,20 +15,18 @@ #define CL_SPEC 1 #define CL_SPACE 2 -#define CL_DIGIT 4 /* Character-class table replacing the old per-call ft_strchr over SPECIAL_CHARS — these predicates run over a million times on a 50k-line parse and the strchr walk dominated them. Bit 1 = shell metacharacter (the SPECIAL_CHARS set, PLUS index 0: ft_strchr matches the terminator, so NUL has always counted as special — end of input ends a word). - Bit 2 = blank (space/tab), bit 4 = ASCII digit. Row comments mark the - non-zero islands. */ + Bit 2 = blank (space/tab). Row comments mark the non-zero islands. */ static const unsigned char g_cl[256] = { 1, 0, 0, 0, 0, 0, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, - 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 1, 1, 0, 1, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, @@ -50,23 +48,6 @@ bool is_space(char c) return ((g_cl[(unsigned char)c] & CL_SPACE) != 0); } -/* Returns true when the string starts with 1-2 digits immediately followed - by a redirect char. We cap at 2 digits -- POSIX only mandates fd < 10 but - bash handles up to 9 too; no sane shell supports fd 100 at lex time. This - check is also called from is_word_boundary so `2>&1` never merges with a - preceding word. */ -static bool is_fd_redirect_start(const char *s) -{ - int i; - - i = 0; - while ((g_cl[(unsigned char)s[i]] & CL_DIGIT) && i < 3) - i++; - if (i > 0 && i <= 2 && (s[i] == '<' || s[i] == '>')) - return (true); - return (false); -} - /* Any character that the shell treats as a metacharacter or whitespace: one table load instead of an ft_strchr walk per query. */ bool is_special_char(char c) @@ -74,20 +55,13 @@ bool is_special_char(char c) return ((g_cl[(unsigned char)c] & (CL_SPEC | CL_SPACE)) != 0); } -/* Check if the current position ends a bare-word token. A fd-redirect start - (e.g. "2>") is a boundary too -- otherwise `echo2>file` would tokenise as - one word instead of `echo` + `2>` + `file`. Only a digit can start a - fd-redirect, so the scan is gated on the digit bit. */ +/* Check if the current position ends a bare-word token: a metacharacter + or a blank. Digits before a `<` or `>` do not end one -- an fd number is + a whole token or nothing (io_number_at, helper4.c), so `echo2>file` is + the word `echo2` redirected to file, as in bash, not `echo` + `2>`. */ bool is_word_boundary(const char *s) { - unsigned char cl; - - cl = g_cl[(unsigned char)*s]; - if (cl & (CL_SPEC | CL_SPACE)) - return (true); - if ((cl & CL_DIGIT) && is_fd_redirect_start(s)) - return (true); - return (false); + return ((g_cl[(unsigned char)*s] & (CL_SPEC | CL_SPACE)) != 0); } /* Count decimal digits in v (minimum 1). Used by the debug table renderer diff --git a/src/lexer/helper4.c b/src/lexer/helper4.c index 1571db35..f05eb72d 100644 --- a/src/lexer/helper4.c +++ b/src/lexer/helper4.c @@ -6,11 +6,12 @@ /* By: marvin +#+ +:+ +#+ */ /* +#+#+#+#+#+ +#+ */ /* Created: 2026/01/22 10:13:59 by dlesieur #+# #+# */ -/* Updated: 2026/03/14 14:07:05 by marvin ### ########.fr */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ /* */ /* ************************************************************************** */ #include "lexer.h" +#include /* Build a token of type tt spanning fd_len bytes starting at start, store it in *out, and return fd_len so callers can use it as both token length and @@ -52,24 +53,61 @@ static int fd_redir_type(char *str, char *p, int fd_len, t_token *out) return (0); } -/* Check whether str starts with an fd-prefixed redirect (e.g. 2>, 1>>). - We allow 1-2 digit fd numbers -- 0, 1, 2 are the common ones; bash also - handles 9. We reject fd_len > 2 to avoid eating a decimal integer that - happens to precede a `>` in an expression context. Returns 0 if no match, - otherwise the total span length and fills *out with the token. */ -int check_fd_redirect(char *str, t_token *out) +/* The length of the IO_NUMBER at s, or 0. POSIX 2.10.1: a token made of + digits alone, delimited by `<` or `>`, is the fd the redirection acts + on. bash also wants the number to fit an int, and otherwise leaves it a + word: `echo x 99999999999>f` writes "x 99999999999" to f. There is no + cap on the digits below that -- `255>f` is fd 255, and `1234567>f` a + "Bad file descriptor" at run time, not the word `12` and a `34567>`. */ +static int io_number_len(const char *s) { - int fd_len; - char *p; + long v; + int i; - fd_len = 0; - p = str; - while (*p && ft_isdigit((unsigned char)*p)) + v = 0; + i = 0; + while (ft_isdigit((unsigned char)s[i])) { - fd_len++; - p++; + v = v * 10 + (s[i++] - '0'); + if (v > INT_MAX) + return (0); } - if (fd_len == 0 || fd_len > 2) + if (i > 0 && (s[i] == '<' || s[i] == '>')) + return (i); + return (0); +} + +/* A token that starts at s is an IO_NUMBER. Only a token START can be + one: `a2>f`, `x=2>f` and `"x"12>f` are the words `a2`, `x=2` and `x12` + followed by `>f`, as in bash -- they used to be cut before the digits, + which sent stderr to f and dropped the digits from the word. Right after + `>&` or `<&` the digits are the descriptor dup'd, never a new + redirection's: `>&2>f` is `>&2` then `>f` (bash's last_read_token rule), + not a syntax error. */ +int io_number_at(t_deque_tok *toks, const char *s) +{ + t_ltoken *last; + int n; + + n = io_number_len(s); + if (n == 0 || toks->deqtok.len == 0) + return (n); + last = (t_ltoken *)deque_idx(&toks->deqtok, toks->deqtok.len - 1); + if (last->tt == TT_DUP_IN || last->tt == TT_DUP_OUT) + return (0); + return (n); +} + +/* Emit the fd-prefixed redirect (e.g. 2>, 10>>, 255<&) at the start of + str. parse_op calls this where a token starts, and tokenize_step routes + there only an IO_NUMBER (io_number_at). Returns 0 if there is none, + otherwise the total span length, with *out filled. */ +int check_fd_redirect(char *str, t_token *out) +{ + int fd_len; + + fd_len = io_number_len(str); + if (fd_len == 0) return (0); - return (fd_redir_type(str, p, fd_len, out)); + return (fd_redir_type(str, str + fd_len, fd_len, out)); } diff --git a/src/lexer/tokenizer.c b/src/lexer/tokenizer.c index 9622ae56..ccbb8295 100644 --- a/src/lexer/tokenizer.c +++ b/src/lexer/tokenizer.c @@ -35,13 +35,16 @@ int extglob_ahead(const char *at); is db_pattern_match). The same span arms db_front_cell, so zsh's bare group may OPEN the operand: `[[ $TERM == (xterm*|screen*) ]]` is one word there and a - subshell anywhere else (dbracket_lex2.c, db_front_group). */ + subshell anywhere else (dbracket_lex2.c, db_front_group). + An IO_NUMBER goes to parse_op: digits are word characters, and the + catch-all would take the `2` of `2>f` for a word (io_number_at). */ static char *try_parse_lexeme(char **str, t_deque_tok *ret, int in_db) { char *prompt; int saved; - if (glob_zsh() && (*str)[0] == '=' && (*str)[1] == '(') + if ((glob_zsh() && (*str)[0] == '=' && (*str)[1] == '(') + || io_number_at(ret, *str)) return (0); saved = *glob_extglob_cell(); if (in_db) diff --git a/src/platform/posix/bind_rl.c b/src/platform/posix/bind_rl.c new file mode 100644 index 00000000..1b08b9a5 --- /dev/null +++ b/src/platform/posix/bind_rl.c @@ -0,0 +1,112 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* bind_rl.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include "zle.h" + +/* The readline half of `bind` (builtin_bind.c parses the options). +** +** Every request is a readline call bash makes too, so the answers -- +** bind -p's inputrc lines, bind -v's settings, -q's "can be invoked via" +** -- are readline's own, word for word. */ + +/* Readline built and every recorded request applied, as it is at a + prompt -- so `bind '...'; bind -p` in a script lists what it just + bound -- then -m's keymap made current. bash warns that line editing + is off there, then answers. Returns the keymap to put back. */ +static Keymap bind_rl_ready(t_shell *state, const char *map) +{ + Keymap km; + + if (state->metinp != INP_RL && !state->opt_interactive) + ft_eprintf("%s: bind: warning: line editing not enabled\n", + state->ctx); + rl_preinit(&state->rl); + km = rl_get_keymap(); + if (map && rl_get_keymap_by_name(map)) + rl_set_keymap(rl_get_keymap_by_name(map)); + return (km); +} + +/* -p -P -v -V -s -S -l: readline's dumpers, onto stdout, in the keymap + -m named (bind_rl_ready switched to it). */ +int bind_rl_list(t_shell *state, const char *map, char opt) +{ + FILE *saved; + Keymap km; + + km = bind_rl_ready(state, map); + saved = rl_outstream; + rl_outstream = stdout; + if (opt == 'p' || opt == 'P') + rl_function_dumper(opt == 'p'); + else if (opt == 'v' || opt == 'V') + rl_variable_dumper(opt == 'v'); + else if (opt == 's' || opt == 'S') + rl_macro_dumper(opt == 's'); + else if (opt == 'l') + rl_list_funmap_names(); + fflush(stdout); + rl_outstream = saved; + rl_set_keymap(km); + return (0); +} + +/* The first five, then "..." -- bash's query_bindings. The strings are + readline's, from its own allocator, so they go back with free(). */ +static void bind_rl_print_keys(const char *name, char **keys) +{ + int j; + + ft_printf("%s can be invoked via ", name); + j = 0; + while (j < 5 && keys[j]) + { + ft_printf("\"%s\"", keys[j]); + if (keys[j + 1]) + ft_printf(", "); + else + ft_printf(".\n"); + j++; + } + if (keys[j]) + ft_printf("...\n"); + j = 0; + while (keys[j]) + free(keys[j++]); + free(keys); +} + +/* -q name: which keys run it, in -m's keymap. */ +int bind_rl_query(t_shell *state, const char *map, const char *name) +{ + t_zle_fn f; + char **keys; + Keymap km; + + km = bind_rl_ready(state, map); + f = rl_named_function(name); + keys = NULL; + if (f) + keys = rl_invoking_keyseqs(f); + rl_set_keymap(km); + if (!f) + return (ft_eprintf("%s: bind: `%s': unknown function name\n", + state->ctx, name), 1); + if (!keys || !keys[0]) + { + free(keys); + return (ft_printf("%s is not bound to any keys.\n", name), 1); + } + bind_rl_print_keys(name, keys); + return (0); +} diff --git a/src/platform/posix/bind_rl2.c b/src/platform/posix/bind_rl2.c new file mode 100644 index 00000000..3067a127 --- /dev/null +++ b/src/platform/posix/bind_rl2.c @@ -0,0 +1,69 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* bind_rl2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include "zle.h" + +/* The two names `bind` checks when it is called, not when readline gets + the request: readline answers both from static tables, before it has + been initialised. */ +bool bind_rl_keymap_ok(const char *name) +{ + return (rl_get_keymap_by_name(name) != NULL); +} + +bool bind_rl_fn_ok(const char *name) +{ + return (rl_named_function(name) != NULL); +} + +/* What one request does, in whatever keymap is current. A readline line + goes through rl_parse_and_bind, which edits its argument: it gets a + copy. */ +static void bind_line_do(t_bind_line *b) +{ + t_zle_fn f; + char *copy; + + if (b->op == 'u') + { + f = rl_named_function(b->line); + if (f) + rl_unbind_function_in_map(f, rl_get_keymap()); + } + else if (b->op == 'r') + rl_bind_keyseq(b->line, NULL); + else + { + copy = ft_strdup(b->line); + if (copy) + rl_parse_and_bind(copy); + xfree(copy); + } +} + +/* One recorded request, in the keymap it named (-m), then the keymap + that was current before it is put back. */ +void bind_line_apply(t_bind_line *b) +{ + Keymap saved; + Keymap map; + + saved = rl_get_keymap(); + map = NULL; + if (b->map) + map = rl_get_keymap_by_name(b->map); + if (map) + rl_set_keymap(map); + bind_line_do(b); + rl_set_keymap(saved); +} diff --git a/src/platform/posix/builtin_fc2.c b/src/platform/posix/builtin_fc2.c index 72e4e809..f0426152 100644 --- a/src/platform/posix/builtin_fc2.c +++ b/src/platform/posix/builtin_fc2.c @@ -70,49 +70,50 @@ static const char *get_fc_editor(t_shell *state) } /* fc [-e editor] [first [last]]: write entries to a temp file, open the - editor, then queue the result for re-execution. `-e -` (a dash) would - suppress the editor in bash; we do not implement that form yet. */ -static int fc_run(t_shell *state, char **av, int ac) + editor, then queue the result for re-execution. No operand edits the + previous command -- the fc line itself is out of fc's view (fc_total). */ +static int fc_run(t_shell *state, t_fcopt *o) { + int total; int first; int last; const char *editor; - if (ac >= 3 && ft_strcmp(av[1], "-e") == 0) - editor = av[2]; - else + editor = o->editor; + if (!editor) editor = get_fc_editor(state); - if (ac > 1 && av[1][0] != '-') - first = fc_resolve_idx(state, av[1]); - else - first = fc_resolve_idx(state, NULL); - if (ac > 2 && av[2][0] != '-') - last = fc_resolve_idx(state, av[2]); - else - last = fc_resolve_idx(state, NULL); + total = fc_total(state); + if (fc_resolve_idx(state, NULL, total, &first) + || (o->nops > 0 && fc_resolve_idx(state, o->ops[0], total, &first))) + return (1); + last = first; + if (o->nops > 1 && fc_resolve_idx(state, o->ops[1], total, &last)) + return (1); + if (first > last) + return (fc_edit_run(state, editor, last, first)); return (fc_edit_run(state, editor, first, last)); } -/* fc [-l [-r] [-n] [first [last]] | [-e editor] [first [last]]]: fix - command. `-l` lists history without editing. Without `-l`, writes the - selected commands to a temp file, opens an editor, and re-executes the - result. Bails out early (status 1) when history is empty or inactive. */ +/* fc [-e editor] [-lnr] [first [last]]: list the history (-l) or edit and + re-run part of it. Options cluster (fc_parse). `-s` and `-e -` -- re-run + without an editor -- are refused OUT LOUD rather than falling into edit + mode, which is what every unrecognised spelling used to do: an editor + started where the caller expected output, as in `$(fc -ln -1)`. */ int builtin_fc(t_shell *state, t_vec argv) { - char **av; - int ac; - bool empty; + t_fcopt o; + int st; - av = (char **)argv.ctx; - ac = (int)argv.len; - empty = (!state->hist.hist_active || !state->hist.hist_cmds.len); - if (ac >= 2 && (!ft_strcmp(av[1], "-l") || !ft_strcmp(av[1], "-lr"))) - { - if (empty) - return (0); - return (fc_list(state, av + 2, ac - 2, av[1][2] == 'r')); - } - if (empty) + st = fc_parse(state, (char **)argv.ctx, (int)argv.len, &o); + if (st) + return (st); + if (o.list && !state->hist.hist_active) + return (0); + if (o.list) + return (fc_list(state, &o)); + if (o.subst) + return (ft_eprintf("%s: fc: -s: not supported\n", state->ctx), 2); + if (!state->hist.hist_active || fc_total(state) <= 0) return (ft_eprintf("%s: fc: no command history\n", state->ctx), 1); - return (fc_run(state, av, ac)); + return (fc_run(state, &o)); } diff --git a/src/platform/posix/capture_subshell_output.c b/src/platform/posix/capture_subshell_output.c index bff452f2..97dbbb72 100644 --- a/src/platform/posix/capture_subshell_output.c +++ b/src/platform/posix/capture_subshell_output.c @@ -44,6 +44,7 @@ static pid_t fork_and_run_inproc(t_shell *state, int pipefd[2], pseudo_traps_quiet(state); state->csf_depth++; state->cmdsub_in_place = cs_single_cmd(state, cmd); + state->hd_from_caller = true; exit(exec_string(state, (char *)cmd) & 0xFF); } return (pid); diff --git a/src/platform/posix/procsub_file.c b/src/platform/posix/procsub_file.c index ae892936..c80095e3 100644 --- a/src/platform/posix/procsub_file.c +++ b/src/platform/posix/procsub_file.c @@ -13,7 +13,9 @@ #include "shell.h" #include "libft.h" #include "sys.h" -#include +#include +#include +#include #include #include @@ -68,18 +70,24 @@ static int run_into_fd(t_shell *state, const char *cmd, int fd) } /* Create the temp file and record it so the session cleans it up. Returns - the path (owned by the caller), or NULL. */ + the path (owned by the caller), or NULL. + The name comes from mkstemp, not from the pid: a =(cmd) made inside + $( ) outlives its subshell, so /tmp keeps files from processes long + gone, and a later shell given the same pid found its name taken. The + O_EXCL open failed, the word vanished without a word, and `cat =(...)` + read its stdin instead. mkstemp keeps O_EXCL's safety and retries a + taken name itself. */ char *create_procsub_file(t_shell *state, const char *cmd) { t_procsub_entry entry; char buf[64]; int fd; - ft_snprintf(buf, sizeof(buf), "%s/hsh%d-%d", TMP_DIR, - (int)getpid(), (int)state->proc_subs.len); - fd = open(buf, O_RDWR | O_CREAT | O_EXCL, 0600); + ft_strlcpy(buf, TMP_DIR "/hsh.XXXXXX", sizeof(buf)); + fd = mkstemp(buf); if (fd < 0) - return (NULL); + return (ft_eprintf("%s: =(...): cannot create a temporary file: " + "%s\n", state->ctx, strerror(errno)), NULL); if (cmd && *cmd && run_into_fd(state, cmd, fd) < 0) return (close(fd), unlink(buf), NULL); if (!cmd || !*cmd) diff --git a/src/platform/posix/rl.c b/src/platform/posix/rl.c index 235fb659..1f8ddaf5 100644 --- a/src/platform/posix/rl.c +++ b/src/platform/posix/rl.c @@ -76,6 +76,7 @@ char *rl_editor_enter(t_shell *state, char *prompt) char *row; zle_enter(state); + rl_suggest_end(state); debug_dump_prompt(prompt); xfree(state->rl.shown_prompt); state->rl.shown_prompt = ft_strdup(prompt); @@ -93,6 +94,7 @@ void rl_editor_exit(t_shell *state) zle_enter(NULL); rl_winch_disarm(); rl_redisplay_function = rl_redisplay; + rl_suggest_end(state); xfree(state->rl.shown_prompt); state->rl.shown_prompt = NULL; } @@ -103,6 +105,7 @@ void rl_editor_exit(t_shell *state) unless HELLISH_RL_FORK=1 asked for the old child (rl_preinit). */ int get_more_input_readline(t_shell *state, char *prompt) { + rl_slot_sync(state); rl_preinit(&state->rl); if (state->rl.use_fork) return (rl_read_fork(state, prompt)); diff --git a/src/platform/posix/rl_editor.c b/src/platform/posix/rl_editor.c index 425ce393..126f346d 100644 --- a/src/platform/posix/rl_editor.c +++ b/src/platform/posix/rl_editor.c @@ -30,9 +30,14 @@ int exec_string(t_shell *state, char *content); ** `stty`, or a command whose job control hands the terminal back in ** cooked mode. TCSANOW, never a deprep/prep pair, which would echo ** typeahead. $? is the user's again afterwards, and a widget that ran -** `exit` ends the read (rl_should_abort). */ +** `exit` ends the read (rl_should_abort). +** +** readline's stream is buffered (rl_outstream.c): what it drew goes out +** first, so the widget's own output lands after it and a child the widget +** forks never inherits bytes still in that buffer. */ void rl_shell_enter(t_shell *state, t_rl_bracket *b) { + fflush(rl_outstream); b->status = state->last_cmd_st_exe; b->have_tty = (tcgetattr(STDIN_FILENO, &b->tty) == 0); rl_clear_signals(); diff --git a/src/platform/posix/rl_getc.c b/src/platform/posix/rl_getc.c index 2ee25959..2271cba4 100644 --- a/src/platform/posix/rl_getc.c +++ b/src/platform/posix/rl_getc.c @@ -33,7 +33,9 @@ ** swallows typeahead into its own buffer). ** ** One byte per read(2): queued Enters stay in the terminal, and each is -** read by the prompt that comes up for it. */ +** read by the prompt that comes up for it. The read is of the terminal's +** second, non-blocking descriptor where there is one (rl_keyfd.c), so the +** only place this ever waits is pselect. */ /* A signal readline caught since the last look: let readline handle it now -- cleanup, the application's handler, re-prep -- and remember an @@ -46,7 +48,10 @@ static void rl_take_signals(void) if (!sig) return ; if (sig == SIGINT) + { *rl_intr_cell() = 1; + rl_suggest_unpaint(); + } rl_check_signals(); } @@ -82,10 +87,12 @@ static int rl_wait(int fd, sigset_t *old, fd_set *set) return (pselect(nfds, set, NULL, NULL, rl_timeout(&ts), old)); } -/* One byte, or EOF, or RL_AGAIN when the read should simply be retried: a - descriptor left non-blocking by some program is put back, as readline's +/* One byte, or EOF, or RL_AGAIN when the read should simply be retried. + From the second descriptor (rl_keyfd.c) EAGAIN means the key pselect saw + is gone -- a ^C flushed it -- and pselect is where to wait now. fd 0 + itself, left non-blocking by some program, is put back, as readline's rl_getc does. Any other error ends the read the way readline's does. */ -static int rl_read1(int fd) +static int rl_read1(int fd, bool twin) { unsigned char c; ssize_t n; @@ -99,7 +106,7 @@ static int rl_read1(int fd) return (RL_AGAIN); if (errno == EAGAIN || errno == EWOULDBLOCK) { - if (fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) & ~O_NONBLOCK) < 0) + if (!twin && fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) & ~O_NONBLOCK) < 0) return (EOF); return (RL_AGAIN); } @@ -115,7 +122,7 @@ int rl_getc_hook(FILE *stream) int fd; int r; - fd = fileno(stream); + fd = rl_keyfd(fileno(stream)); r = RL_AGAIN; while (r == RL_AGAIN) { @@ -128,9 +135,11 @@ int rl_getc_hook(FILE *stream) r = rl_wait(fd, &old, &set); sigprocmask(SIG_SETMASK, &old, NULL); if (r > 0 && FD_ISSET(fd, &set)) - r = rl_read1(fd); + r = rl_read1(fd, fd != fileno(stream)); else r = (rl_idle_event(r), RL_AGAIN); } + if (r == '\r' || r == '\n') + rl_suggest_unpaint(); return (r); } diff --git a/src/platform/posix/rl_keyfd.c b/src/platform/posix/rl_keyfd.c new file mode 100644 index 00000000..2483a9e5 --- /dev/null +++ b/src/platform/posix/rl_keyfd.c @@ -0,0 +1,123 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_keyfd.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include +#include + +/* The terminal the line is read from, opened a second time to read the +** keys from: an open file description of its own, O_NONBLOCK, so that a +** read of it can never block. +** +** rl_getc_hook waits in pselect, which lets a signal in atomically, and +** reads once pselect reports a key. A ^C can be processed between the +** two, and the line discipline then does two things in this order: it +** sends SIGINT -- readline's handler runs before read() is entered, so +** there is no EINTR to come -- and it flushes the input queue, taking the +** key pselect saw. A blocking read() slept on the empty queue until the +** NEXT key, and gave that key to the line the ^C had ended: ^C in a ^R +** search, then `echo ...` ran as `cho ...` (rl_read_race_test.py). From +** here the flushed key is EAGAIN, and the loop goes back to pselect, +** which reports the signal. +** +** O_NONBLOCK is not set on fd 0 itself: that description is shared with +** every program the shell runs, and #34 is what a non-blocking terminal +** does to them. It sits at fd 255, where bash keeps its terminal, close on +** exec, and is checked at every prompt (rl_preinit): when fd 0 is no +** longer that terminal, or the slot shows something else (a redirection +** onto 255 closed ours), it is opened again. A terminal that cannot be +** opened again -- after `su`, the device is not ours -- is read through +** fd 0, as before. +** +** fd 0 is put back to blocking at every prompt too, as bash does before +** every readline(): a program that left it non-blocking would otherwise +** hand the next command an EAGAIN on stdin. */ + +static t_rl_keyfd *keyfd_cell(void) +{ + static t_rl_keyfd k = {-1, -1, 0}; + + return (&k); +} + +/* The descriptor to read the keys of `fd` from. */ +int rl_keyfd(int fd) +{ + t_rl_keyfd *k; + + k = keyfd_cell(); + if (k->twin >= 0 && k->src == fd) + return (k->twin); + return (fd); +} + +/* The second descriptor is still open and still the terminal `rdev`. */ +static bool keyfd_is(t_rl_keyfd *k, dev_t rdev) +{ + struct stat st; + + return (k->twin >= 0 && fstat(k->twin, &st) == 0 + && S_ISCHR(st.st_mode) && st.st_rdev == rdev); +} + +/* Open the terminal on `fd` again, non-blocking, parked at fd 255 or the + first free one above (10 and up where the limit is lower). */ +static void keyfd_open(t_rl_keyfd *k, int fd, dev_t rdev) +{ + char *path; + int nfd; + int high; + + path = ttyname(fd); + if (!path) + return ; + nfd = open(path, O_RDONLY | O_NONBLOCK | O_NOCTTY | O_CLOEXEC); + if (nfd < 0) + return ; + high = fcntl(nfd, F_DUPFD_CLOEXEC, 255); + if (high < 0) + high = fcntl(nfd, F_DUPFD_CLOEXEC, 10); + if (high >= 0) + { + close(nfd); + nfd = high; + } + k->twin = nfd; + k->src = fd; + k->rdev = rdev; +} + +/* Before a read of `fd`: fd 0 blocking again, and the second descriptor + made to match it. An old one is closed only while it still is the + terminal it was opened on: a slot that shows anything else was taken + over by the user, and that descriptor is theirs now. */ +void rl_keyfd_refresh(int fd) +{ + t_rl_keyfd *k; + struct stat st; + bool tty; + int fl; + + k = keyfd_cell(); + fl = fcntl(fd, F_GETFL); + if (fl >= 0 && (fl & O_NONBLOCK)) + fcntl(fd, F_SETFL, fl & ~O_NONBLOCK); + tty = (fstat(fd, &st) == 0 && S_ISCHR(st.st_mode)); + if (tty && k->src == fd && keyfd_is(k, st.st_rdev)) + return ; + if (keyfd_is(k, k->rdev)) + close(k->twin); + k->twin = -1; + k->src = -1; + if (tty) + keyfd_open(k, fd, st.st_rdev); +} diff --git a/src/platform/posix/rl_outstream.c b/src/platform/posix/rl_outstream.c new file mode 100644 index 00000000..d05dbdb6 --- /dev/null +++ b/src/platform/posix/rl_outstream.c @@ -0,0 +1,85 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_outstream.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#define _GNU_SOURCE +#include "rl_private.h" + +/* readline's output stream: the terminal on fd 2, as bash has it -- but +** BUFFERED, and written through tty_write_all. +** +** It was stderr, which is unbuffered, and readline 8.2 draws the prompt +** with putc: the row it owns reached the tty one byte per write() -- 26 of +** them for `\e[38;2;152;195;121m> \e[0m`. Every write() is a place the +** line discipline puts out an echo it still owes (n_tty_write runs +** process_echoes first, whatever ECHO is by then): a key typed before +** readline took raw mode, while the terminal's reader was behind, came +** out between two of those bytes -- inside the colour escape, whose tail +** then printed as text. That is the split prompt_atomic_test.py found on a +** loaded CI runner in the one row rl_prerow.c could not account for. +** +** bash line-buffers stderr for the whole shell (shell_initialize). Here +** only readline's stream is buffered: readline flushes it at the end of +** every redisplay and around raw mode, so a row goes out in one write(), +** and prompt_row_writes_test.py pins that no escape or glyph is ever cut +** by a write() boundary. The flush goes through tty_write_all, like every +** other prompt writer since #34, so a terminal some program left +** non-blocking loses nothing on EAGAIN. It writes to fd 2 as it is at the +** time, so `exec 2>file` still takes the prompt with it. The stream has +** no descriptor of its own: what needs one uses STDERR_FILENO. */ + +#ifdef __APPLE__ + +static int rl_out_cookie(void *cookie, const char *buf, int len) +{ + (void)cookie; + tty_write_all(STDERR_FILENO, buf, (size_t)len); + return (len); +} + +static FILE *rl_out_make(void) +{ + return (funopen(NULL, NULL, rl_out_cookie, NULL, NULL)); +} + +#else + +static ssize_t rl_out_cookie(void *cookie, const char *buf, size_t len) +{ + (void)cookie; + tty_write_all(STDERR_FILENO, buf, len); + return ((ssize_t)len); +} + +static FILE *rl_out_make(void) +{ + cookie_io_functions_t io; + + ft_memset(&io, 0, sizeof(io)); + io.write = rl_out_cookie; + return (fopencookie(NULL, "w", io)); +} + +#endif + +/* Made once, at readline's first setup. stderr if the stream cannot be + made -- the old behaviour, not a failure. */ +FILE *rl_out_open(void) +{ + static char buf[16384]; + FILE *f; + + f = rl_out_make(); + if (!f) + return (stderr); + setvbuf(f, buf, _IOFBF, sizeof(buf)); + return (f); +} diff --git a/src/platform/posix/rl_preinit.c b/src/platform/posix/rl_preinit.c index 0202a305..54d7accc 100644 --- a/src/platform/posix/rl_preinit.c +++ b/src/platform/posix/rl_preinit.c @@ -56,20 +56,20 @@ static void rl_mode_apply(int edit_mode) setup_emacs_mode(); } -/* Install the zle bindings this process has not installed yet. `bindkey` - can add one at any time, so this runs per prompt and is a no-op once - the registry has stopped growing. */ +/* Apply the `bind` requests and install the `bindkey` bindings this + process has not seen yet. Either can add one at any time, so this runs + per prompt and is a no-op once the registries have stopped growing. */ static void rl_bind_pending(t_rl *l) { + t_bind_line *b; t_zle_bind *a; + b = (t_bind_line *)bind_lines()->ctx; + while (l->bind_lines_done < bind_lines()->len) + bind_line_apply(&b[l->bind_lines_done++]); a = (t_zle_bind *)zle_binds()->ctx; while (l->binds_done < zle_binds()->len) - { - zle_bind_raw(&a[l->binds_done]); - rl_bind_keyseq(a[l->binds_done].seq, zle_dispatch); - l->binds_done++; - } + zle_bind_install(&a[l->binds_done++]); } /* HELLISH_RL_FORK=1: read in a forked child, as every release before this @@ -103,7 +103,7 @@ static void rl_first_init(t_rl *l) { setlocale(LC_ALL, ""); rl_instream = stdin; - rl_outstream = stderr; + rl_outstream = rl_out_open(); rl_getc_function = rl_getc_hook; rl_change_environment = 0; rl_variable_bind("revert-all-at-newline", "on"); @@ -111,6 +111,8 @@ static void rl_first_init(t_rl *l) rl_initialize(); l->rl_ready = true; l->mode_applied = -1; + l->feat_on[0] = -1; + l->feat_on[1] = -1; l->use_fork = !RL_INPROC_OK || rl_fork_wanted(); } @@ -124,11 +126,15 @@ void rl_preinit(t_rl *l) rl_first_init(l); rl_reset_screen_size(); rl_abort_reset(); + rl_keyfd_refresh(fileno(rl_instream)); if (l->mode_applied != l->edit_mode) { rl_mode_apply(l->edit_mode); l->mode_applied = l->edit_mode; l->binds_done = 0; + l->bind_lines_done = 0; } + rl_slot_pre(l); rl_bind_pending(l); + rl_slot_post(l); } diff --git a/src/platform/posix/rl_prerow.c b/src/platform/posix/rl_prerow.c index 1df23881..2a61afc7 100644 --- a/src/platform/posix/rl_prerow.c +++ b/src/platform/posix/rl_prerow.c @@ -49,7 +49,7 @@ static int rp_emit(void) f = rp_cell(); if (f->len) - tty_write_all(fileno(rl_outstream), (char *)f->ctx, f->len); + rl_out_write((char *)f->ctx, f->len); xfree(f->ctx); f->ctx = NULL; f->len = 0; @@ -68,3 +68,13 @@ void rl_prerow_arm(t_string *rows) *f = *rows; rl_startup_hook = rp_emit; } + +/* What hellish composes itself for the terminal readline is drawing on -- + these rows, the right prompt, a repaint, a mascot frame -- goes straight + to fd 2 in one write, after anything readline still holds in its buffered + stream (rl_outstream.c), so the two arrive in the order they were made. */ +void rl_out_write(const char *buf, size_t len) +{ + fflush(rl_outstream); + tty_write_all(STDERR_FILENO, buf, len); +} diff --git a/src/platform/posix/rl_repaint.c b/src/platform/posix/rl_repaint.c index e9542dea..f84eb925 100644 --- a/src/platform/posix/rl_repaint.c +++ b/src/platform/posix/rl_repaint.c @@ -99,7 +99,7 @@ static void rp_upper_rows(const char *s, int rows) s += (*s == '\n'); } vec_push_str(&f, "\033[K"); - tty_write_all(fileno(rl_outstream), f.ctx, f.len); + rl_out_write(f.ctx, f.len); xfree(f.ctx); } @@ -122,7 +122,7 @@ static void rp_swap(t_shell *st, t_string *p, int cols) rl_set_prompt(row); st->rl.rp_prompt_w = visible_width_cstr(row); rl_forced_update_display(); - tty_write_all(fileno(rl_outstream), "\033[?2026l", 8); + rl_out_write("\033[?2026l", 8); xfree(st->rl.shown_prompt); st->rl.shown_prompt = (char *)p->ctx; } diff --git a/src/platform/posix/rl_rprompt.c b/src/platform/posix/rl_rprompt.c index b4e4155a..be0b74c8 100644 --- a/src/platform/posix/rl_rprompt.c +++ b/src/platform/posix/rl_rprompt.c @@ -50,109 +50,75 @@ ** column lost its tail on screen -- the line you were about to ** run was not the line you could see. ** -** Where the text ends is an UPPER bound (rp_text_end): clearing a few +** Where the text ends is an UPPER bound (rl_text_end): clearing a few ** blank cells too far right costs nothing, clearing one too far left ** erases what was typed. */ -/* The last column the line can occupy, or -1 when the cursor may not be - on the prompt's row (a search prompt owns it, the line has a newline). - Per byte: a tab takes at most 8 columns, a control character is drawn - as ^X (2), a byte a non-UTF-8 locale prints as \ooo takes 4, and every - other byte at most 1 -- a UTF-8 character never takes more columns than - it has bytes. */ -static int rp_text_end(t_rl *l) +/* The ghost (rl_suggest.c) goes where the cursor is, so the clock is + judged against the end of the ghost and left out while the ghost reaches + it; it comes back when the ghost is gone. Returns false when no ghost + fits in the room left (a wide character at the very edge). */ +static void rp_ghost(t_rl *l, t_frame *f) { - unsigned char c; - int end; - int i; + int w; - if (RL_ISSTATE(RL_STATE_ISEARCH) || RL_ISSTATE(RL_STATE_NSEARCH)) - return (-1); - end = l->rp_prompt_w; - i = 0; - while (rl_line_buffer && i < rl_end) - { - c = (unsigned char)rl_line_buffer[i++]; - if (c == '\n') - return (-1); - if (c == '\t') - end += 8; - else if (c < ' ' || c == 0x7f) - end += 2; - else if (c >= 0x80 && MB_CUR_MAX == 1) - end += 4; - else - end += 1; - } - return (end); -} - -/* Append CSI G (cursor to column n). False if it could not be built, - and then nothing may be cleared: the clear would land at the cursor. */ -static bool rp_push_col(t_string *f, int n) -{ - char *s; - - s = ft_itoa(n); - if (!s) - return (false); - vec_push_str(f, "\033["); - vec_push_str(f, s); - vec_push_str(f, "G"); - xfree(s); - return (true); + f->end = rl_prompt_cols(l) + visible_width_cstr(rl_line_buffer); + f->gn = rl_ghost_fit(f->ghost, f->cols - 1 - f->end, &w); + f->show = rl_rp_fits(l, f->cols, f->end + w); } -/* One frame, one write: save, clear from just past the text to the end of - the row, the clock if it is shown, restore. */ -static void rp_frame(t_rl *l, int cols, int end, bool show) +/* Paint what this frame needs, if anything: the ghost; or, without one, + the clock shown or taken down, and a ghost that was up taken down. When + the cursor may be off the prompt's row nothing is touched. */ +static void rp_paint(t_rl *l, t_frame *f) { - t_string f; - - vec_init(&f); - f.elem_size = 1; - vec_push_str(&f, "\0337"); - if (!rp_push_col(&f, end + 1)) - return (xfree(f.ctx)); - vec_push_str(&f, "\033[K"); - if (show && !rp_push_col(&f, cols - l->rp_w + 1)) - return (xfree(f.ctx)); - if (show) - vec_push_str(&f, (char *)l->rp_txt.ctx); - vec_push_str(&f, "\0338"); - tty_write_all(fileno(rl_outstream), f.ctx, f.len); - xfree(f.ctx); - l->rp_painted = show; + f->gn = 0; + if (f->ghost) + rp_ghost(l, f); + if (!f->gn) + { + f->end = rl_text_col(l); + if (f->end < 0 || f->end >= f->cols) + return ; + f->show = rl_rp_fits(l, f->cols, f->end); + if (!f->show && !l->rp_painted && !l->sg_painted) + return ; + } + rl_frame(l, f); } -/* readline's redisplay, then ours. The clock is shown with one blank - between it and the text; otherwise a painted one is taken down. */ +/* readline's redisplay, then ours. With no right prompt, no ghost and none + painted last frame this is readline's redisplay and nothing else. */ static void rp_redisplay(void) { t_shell *state; + t_frame f; int rows; - int cols; - int end; rl_redisplay(); state = *zle_state_cell(); - if (!state || state->rl.rp_w <= 0) + if (!state) return ; - rl_get_screen_size(&rows, &cols); - end = rp_text_end(&state->rl); - if (end < 0 || end >= cols) + rl_get_screen_size(&rows, &f.cols); + f.ghost = rl_suggest_update(state, f.cols); + if (!f.ghost && !state->rl.sg_painted && state->rl.rp_w <= 0) return ; - if (cols > state->rl.rp_w + 1 && end + 1 < cols - state->rl.rp_w) - rp_frame(&state->rl, cols, end, true); - else if (state->rl.rp_painted) - rp_frame(&state->rl, cols, end, false); + rp_paint(&state->rl, &f); } /* Arm the painter for this readline() call. Nothing is painted until readline's first redisplay, which precedes the first keystroke, so the - clock appears together with the prompt. */ + clock appears together with the prompt. It is needed with a right prompt + or with the suggestion on; a widget that sets POSTDISPLAY brings it in + by itself (rl_rprompt_hook). */ void rl_rprompt_install(t_shell *state) { state->rl.rp_painted = false; + if (state->rl.rp_w > 0 || state->rl.feat_want[FEAT_ACCEPT]) + rl_rprompt_hook(); +} + +void rl_rprompt_hook(void) +{ rl_redisplay_function = rp_redisplay; } diff --git a/src/platform/posix/rl_rprompt2.c b/src/platform/posix/rl_rprompt2.c new file mode 100644 index 00000000..7446e8d0 --- /dev/null +++ b/src/platform/posix/rl_rprompt2.c @@ -0,0 +1,118 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_rprompt2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" + +/* The pieces rl_rprompt.c paints with: where the text ends, whether the + right prompt fits, and the one-write frame. */ + +/* The last column the line can occupy, or -1 when the cursor may not be + on the prompt's row (a search prompt owns it, the line has a newline). + Per byte: a tab takes at most 8 columns, a control character is drawn + as ^X (2), a byte a non-UTF-8 locale prints as \ooo takes 4, and every + other byte at most 1 -- a UTF-8 character never takes more columns than + it has bytes. */ +int rl_text_end(t_rl *l) +{ + unsigned char c; + int end; + int i; + + if (RL_ISSTATE(RL_STATE_ISEARCH) || RL_ISSTATE(RL_STATE_NSEARCH)) + return (-1); + end = l->rp_prompt_w; + i = 0; + while (rl_line_buffer && i < rl_end) + { + c = (unsigned char)rl_line_buffer[i++]; + if (c == '\n') + return (-1); + if (c == '\t') + end += 8; + else if (c < ' ' || c == 0x7f) + end += 2; + else if (c >= 0x80 && MB_CUR_MAX == 1) + end += 4; + else + end += 1; + } + return (end); +} + +/* Where to clear from: the upper bound above, unless a ghost is on screen. + Clearing a few cells too far right is free; but a ghost left in them + would stay, so with one painted the exact end of the (plain) line is + used -- the same width the ghost was placed with. */ +int rl_text_col(t_rl *l) +{ + int end; + + end = rl_text_end(l); + if (end >= 0 && l->sg_painted && rl_suggest_plain(rl_line_buffer)) + end = rl_prompt_cols(l) + visible_width_cstr(rl_line_buffer); + return (end); +} + +/* The clock fits when one blank separates it from what ends at `end`. */ +bool rl_rp_fits(t_rl *l, int cols, int end) +{ + return (l->rp_w > 0 && cols > l->rp_w + 1 && end + 1 < cols - l->rp_w); +} + +/* Append CSI G (cursor to column n). False if it could not be built, + and then nothing may be cleared: the clear would land at the cursor. */ +bool rl_push_col(t_string *f, int n) +{ + char *s; + + s = ft_itoa(n); + if (!s) + return (false); + vec_push_str(f, "\033["); + vec_push_str(f, s); + vec_push_str(f, "G"); + xfree(s); + return (true); +} + +/* One frame, one write: save; then either the ghost, dim, from the cursor + (which readline left at the end of the text) or a jump to just past the + text; clear to the end of the row; the clock if it is shown; restore. + The ghost is cleared past as well, so a longer one that was there before + cannot leave a tail. */ +void rl_frame(t_rl *l, t_frame *f) +{ + t_string b; + + vec_init(&b); + b.elem_size = 1; + vec_push_str(&b, "\0337"); + if (f->gn) + { + vec_push_str(&b, "\033[2m"); + vec_push_nstr(&b, f->ghost, f->gn); + vec_push_str(&b, "\033[0m"); + } + else if (!rl_push_col(&b, f->end + 1)) + return (xfree(b.ctx)); + vec_push_str(&b, "\033[K"); + if (f->show && !rl_push_col(&b, f->cols - l->rp_w + 1)) + return (xfree(b.ctx)); + if (f->show) + vec_push_str(&b, (char *)l->rp_txt.ctx); + vec_push_str(&b, "\0338"); + rl_out_write(b.ctx, b.len); + xfree(b.ctx); + l->rp_painted = f->show; + l->sg_painted = f->gn > 0; + l->sg_col = f->end; +} diff --git a/src/platform/posix/rl_slot.c b/src/platform/posix/rl_slot.c new file mode 100644 index 00000000..199428b3 --- /dev/null +++ b/src/platform/posix/rl_slot.c @@ -0,0 +1,131 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_slot.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" + +/* Keys that hold one of readline's stock functions until a feature wants +** them: Up / Down for history search (HELLISH_HISTORY_PREFIX) and Right / +** End / ^F for accepting the inline suggestion (HELLISH_AUTOSUGGEST). +** A slot is "ours" while slot_bits has its bit; it is "the user's" once a +** bind / bindkey / inputrc line has set it, and then it is never touched +** again (slot_user). The arrows come in two encodings (CSI and the +** application-cursor SS3 form) and two keymaps, hence the rows. */ + +/* The whole table, by index: 0-7 are the history keys (Up / Down, in the + CSI and SS3 encodings, emacs then vi), 8-12 the emacs accept keys and + 13-16 the vi ones (which have no ^F). `vi` picks vi_insertion_keymap + over the emacs one. */ +t_slot rl_slot_at(int i) +{ + static const char *keys[] = {"\033[A", "\033[B", "\033OA", "\033OB", + "\033[C", "\033OC", "\006", "\033[F", "\033OF"}; + t_slot s; + int k; + int j; + + s.feat = FEAT_HIST; + s.vi = i / 4; + s.key = keys[i % 4]; + s.fn = SLOT_PREV + i % 2; + if (i < 8) + return (s); + j = i - 8; + k = 4 + j; + s.feat = FEAT_ACCEPT; + s.vi = j > 4; + if (s.vi) + k = j - 1 + (j > 6); + s.key = keys[k]; + s.fn = SLOT_FWD + (k > 6); + return (s); +} + +/* The stock function of a slot, and what replaces it while its feature + is on. */ +static void rl_slot_funcs(const t_slot *s, rl_command_func_t **stock, + rl_command_func_t **repl) +{ + *stock = rl_get_previous_history; + *repl = rl_history_search_backward; + if (s->fn == SLOT_NEXT) + { + *stock = rl_get_next_history; + *repl = rl_history_search_forward; + } + if (s->fn == SLOT_FWD || s->fn == SLOT_END) + *repl = rl_suggest_accept; + if (s->fn == SLOT_FWD) + *stock = rl_forward_char; + if (s->fn == SLOT_END) + *stock = rl_end_of_line; +} + +/* A marker no user can bind: a slot still holding it after the user's + bind lines ran was not touched by them. */ +static int rl_slot_mark(int count, int key) +{ + (void)count; + (void)key; + return (0); +} + +/* Before the user's bindings are replayed: park every slot that is the + stock function (or our replacement) on the marker. */ +void rl_slot_arm(t_rl *l, int i) +{ + t_slot s; + Keymap map; + rl_command_func_t *stock; + rl_command_func_t *repl; + rl_command_func_t *cur; + + s = rl_slot_at(i); + rl_slot_funcs(&s, &stock, &repl); + map = emacs_standard_keymap; + if (s.vi) + map = vi_insertion_keymap; + cur = rl_function_of_keyseq(s.key, map, NULL); + if (l->slot_user >> i & 1) + return ; + if (cur == stock || (cur == repl && (l->slot_bits >> i & 1))) + rl_bind_keyseq_in_map(s.key, rl_slot_mark, map); +} + +/* After the replay: a slot still on the marker gets the wanted function + (the replacement when its feature is on, stock when off); any other + function there is the user's own choice, which is kept for good. */ +void rl_slot_settle(t_rl *l, int i) +{ + t_slot s; + Keymap map; + rl_command_func_t *stock; + rl_command_func_t *repl; + rl_command_func_t *cur; + + s = rl_slot_at(i); + rl_slot_funcs(&s, &stock, &repl); + map = emacs_standard_keymap; + if (s.vi) + map = vi_insertion_keymap; + cur = rl_function_of_keyseq(s.key, map, NULL); + if (l->slot_user >> i & 1) + return ; + l->slot_bits &= ~(1u << i); + if (cur != rl_slot_mark) + l->slot_user |= 1u << i; + else if (l->feat_want[s.feat]) + l->slot_bits |= 1u << i; + if (cur == rl_slot_mark && l->feat_want[s.feat]) + rl_bind_keyseq_in_map(s.key, repl, map); + else if (cur == rl_slot_mark) + rl_bind_keyseq_in_map(s.key, stock, map); +} diff --git a/src/platform/posix/rl_slot_env.c b/src/platform/posix/rl_slot_env.c new file mode 100644 index 00000000..e1375d37 --- /dev/null +++ b/src/platform/posix/rl_slot_env.c @@ -0,0 +1,79 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_slot_env.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include "env.h" +#include + +/* HELLISH_HISTORY_PREFIX, read live from the shell's own variables so an + assignment at the prompt, in the rc file or inherited all count. Off + only for 0 / off (any case); unset, empty or anything else is on. */ +static int hp_wanted(t_shell *state) +{ + t_env *e; + + e = env_get(&state->env, "HELLISH_HISTORY_PREFIX"); + if (!e || !e->value) + return (1); + if (!ft_strcmp(e->value, "0") || !strcasecmp(e->value, "off")) + return (0); + return (1); +} + +/* HELLISH_AUTOSUGGEST, read the same way but the other way round: off + unless it is 1 or on (any case). A terminal that cannot move the cursor + (TERM unset or dumb) never gets a ghost. */ +static int as_wanted(t_shell *state) +{ + t_env *e; + + e = env_get(&state->env, "TERM"); + if (!e || !e->value || !ft_strcmp(e->value, "dumb")) + return (0); + e = env_get(&state->env, "HELLISH_AUTOSUGGEST"); + if (!e || !e->value) + return (0); + return (!ft_strcmp(e->value, "1") || !strcasecmp(e->value, "on")); +} + +/* HELLISH_SUGGEST_MAX: how many of the newest history entries a search + looks at. Anything that is not a plain number >= 1 is the default; more + than nine digits is "all of it" for any history there can be. */ +static int sg_limit(t_shell *state) +{ + t_env *e; + size_t i; + + e = env_get(&state->env, "HELLISH_SUGGEST_MAX"); + if (!e || !e->value || !e->value[0]) + return (1000); + i = 0; + while (e->value[i] && ft_isdigit(e->value[i])) + i++; + if (e->value[i] || i > 9 || ft_atoi(e->value) < 1) + return (1000); + return (ft_atoi(e->value)); +} + +/* Interactive reads only, right before rl_preinit: note what is wanted and +** switch the slot hooks on. Doing it inside rl_preinit's caller keeps +** `bind -p` (which also runs rl_preinit, in -c mode) exactly as bash's. +** Up / Down become history search by the text before the cursor (off with +** HELLISH_HISTORY_PREFIX=0|off); Right / End / ^F accept the suggestion +** while HELLISH_AUTOSUGGEST is on. */ +void rl_slot_sync(t_shell *state) +{ + state->rl.slot_live = true; + state->rl.feat_want[FEAT_HIST] = hp_wanted(state); + state->rl.feat_want[FEAT_ACCEPT] = as_wanted(state); + state->rl.sg_max = sg_limit(state); +} diff --git a/src/platform/posix/rl_slot_run.c b/src/platform/posix/rl_slot_run.c new file mode 100644 index 00000000..8e9d518f --- /dev/null +++ b/src/platform/posix/rl_slot_run.c @@ -0,0 +1,69 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_slot_run.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include "zle.h" + +/* rl_preinit, after the editing mode is set and BEFORE the user's `bind` +** / `bindkey` lines are replayed: park the slots of every feature that +** needs it on a marker (only when something changed -- the wanted state or +** new requests -- so a quiet prompt costs a few compares). A mode switch +** only selects another keymap, it rebuilds none, so the stock keys survive +** it; its replay is "pending" (binds_done rewound), which is what triggers +** the work. */ +void rl_slot_pre(t_rl *l) +{ + bool pending; + int i; + + l->feat_open = 0; + if (!l->slot_live) + return ; + pending = l->binds_done < zle_binds()->len + || l->bind_lines_done < bind_lines()->len; + i = 0; + while (i < RL_NFEAT) + { + if (pending || l->feat_want[i] != l->feat_on[i]) + l->feat_open |= 1u << i; + i++; + } + i = 0; + while (l->feat_open && i < RL_NSLOTS) + { + if (l->feat_open >> rl_slot_at(i).feat & 1) + rl_slot_arm(l, i); + i++; + } +} + +/* rl_preinit, after the replay: settle each parked slot. */ +void rl_slot_post(t_rl *l) +{ + int i; + + i = 0; + while (l->feat_open && i < RL_NSLOTS) + { + if (l->feat_open >> rl_slot_at(i).feat & 1) + rl_slot_settle(l, i); + i++; + } + i = 0; + while (i < RL_NFEAT) + { + if (l->feat_open >> i & 1) + l->feat_on[i] = l->feat_want[i]; + i++; + } + l->feat_open = 0; +} diff --git a/src/platform/posix/rl_suggest.c b/src/platform/posix/rl_suggest.c new file mode 100644 index 00000000..d0492120 --- /dev/null +++ b/src/platform/posix/rl_suggest.c @@ -0,0 +1,140 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_suggest.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" + +/* The inline suggestion: the newest history entry that starts with what +** is typed, shown dim after the cursor and never part of the line. This +** file decides WHAT may be suggested and WHEN; rl_suggest2.c turns it into +** the text and rl_rprompt.c paints it. +** +** Anything a terminal could act on is kept out of the ghost. An entry is +** written to the tty as it is, so one holding ESC (a colour, a title, a +** cursor move), a newline, a tab or any other control byte is skipped, as +** is a C1 control in its UTF-8 form: running `printf '\e[31m'` once must +** not repaint every later prompt. */ + +/* True when `s` holds nothing a terminal would interpret. 0x7f is DEL; the + pair 0xC2 0x80..0x9F is U+0080..U+009F, which some terminals read as a + control sequence introducer. */ +static bool sg_clean(const unsigned char *s) +{ + size_t i; + + i = 0; + while (s[i]) + { + if (s[i] < 0x20 || s[i] == 0x7f) + return (false); + if (s[i] == 0xc2 && s[i + 1] >= 0x80 && s[i + 1] < 0xa0) + return (false); + i++; + } + return (true); +} + +/* The newest of the last sg_max entries that starts with the first `n` + bytes of the line, is longer than it and is clean. A bounded look keeps + a huge history from slowing a keystroke down. */ +static char *sg_scan(t_shell *state, t_rl *l, size_t n) +{ + char **a; + size_t i; + size_t seen; + + a = (char **)state->hist.hist_cmds.ctx; + i = state->hist.hist_cmds.len; + seen = 0; + while (i > 0 && seen++ < (size_t)l->sg_max) + { + i--; + if (a[i] && !ft_strncmp(a[i], rl_line_buffer, n) + && ft_strlen(a[i]) > n && sg_clean((unsigned char *)a[i])) + return (a[i]); + } + return (NULL); +} + +/* The entry to suggest for the line, or NULL. +** +** Typing extends the line, and an entry that matches the longer line +** matched the shorter one too -- so when the line still starts with the one +** the last scan was for, the answer is the same entry as long as it still +** matches (or "nothing" for good). That makes a key O(line) with no scan, +** and a paste of 4 KB one scan instead of 4096. Anything else (a +** Backspace, a recalled line) scans again. The history cannot change under +** it: it only grows between reads, and the cache is dropped per read. */ +char *rl_suggest_find(t_shell *state, t_rl *l) +{ + size_t n; + size_t p; + char *e; + + n = (size_t)rl_end; + p = 0; + if (l->sg_known) + p = ft_strlen(l->sg_prev); + if (l->sg_known && p <= n && !ft_strncmp(rl_line_buffer, l->sg_prev, p) + && (!l->sg_ent || (ft_strlen(l->sg_ent) > n + && !ft_strncmp(l->sg_ent, rl_line_buffer, n)))) + return (l->sg_ent); + xfree(l->sg_prev); + xfree(l->sg_ent); + e = sg_scan(state, l, n); + l->sg_ent = NULL; + if (e) + l->sg_ent = ft_strdup(e); + l->sg_prev = ft_strdup(rl_line_buffer); + l->sg_known = true; + return (l->sg_ent); +} + +/* The line is plain text this painter can measure: no control byte (a ^X + or a tab takes more columns than it has bytes), and in a single-byte + locale no byte readline would draw as \ooo. */ +bool rl_suggest_plain(const char *s) +{ + size_t i; + + i = 0; + while (s[i]) + { + if ((unsigned char)s[i] < 0x20 || (unsigned char)s[i] == 0x7f) + return (false); + if ((unsigned char)s[i] >= 0x80 && MB_CUR_MAX == 1) + return (false); + i++; + } + return (true); +} + +/* Where a ghost may be shown at all: the cursor at the end of the PS1 +** line, the editor in a plain insert state, not finishing, and the line -- +** with the prompt -- narrower than the terminal so that neither it nor the +** ghost can wrap. The cursor is then on the prompt's row at column +** rp_prompt_w + width of the text, which is where the ghost goes. +** +** Not looked at: whether more input is already waiting. Asking costs a +** select(2) per key, which the per-key syscall budget does not allow; the +** cache above makes the extra frames cheap instead. */ +bool rl_suggest_gate(t_rl *l, int cols) +{ + if (rl_done || rl_point != rl_end || !l->ps1_read || !rl_line_buffer) + return (false); + if (RL_ISSTATE(RL_STATE_ISEARCH | RL_STATE_NSEARCH)) + return (false); + if (rl_get_keymap() == vi_movement_keymap) + return (false); + if (!rl_suggest_plain(rl_line_buffer)) + return (false); + return (rl_prompt_cols(l) + visible_width_cstr(rl_line_buffer) < cols - 1); +} diff --git a/src/platform/posix/rl_suggest2.c b/src/platform/posix/rl_suggest2.c new file mode 100644 index 00000000..e8b6e5f8 --- /dev/null +++ b/src/platform/posix/rl_suggest2.c @@ -0,0 +1,102 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_suggest2.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" + +/* The ghost for the line as it is now. */ + +/* Forget a POSTDISPLAY a widget set once the line is not the one it was + set for -- "until the buffer next changes". */ +static void sg_post_drop(t_rl *l) +{ + l->sg_txt = NULL; + xfree(l->sg_post); + xfree(l->sg_post_at); + l->sg_post = NULL; + l->sg_post_at = NULL; +} + +/* The text to paint after the cursor for the line readline just drew, or + NULL. A POSTDISPLAY a widget set wins over the history suggestion, and + shows even with HELLISH_AUTOSUGGEST off. l->sg_txt remembers the answer + (it points into sg_ent / sg_post) for the accept key and for widgets. */ +char *rl_suggest_update(t_shell *state, int cols) +{ + t_rl *l; + char *e; + + l = &state->rl; + l->sg_txt = NULL; + if (l->sg_post && ft_strcmp(l->sg_post_at, rl_line_buffer)) + sg_post_drop(l); + if (!l->sg_post && !l->feat_want[FEAT_ACCEPT]) + return (NULL); + if (!rl_suggest_gate(l, cols)) + return (NULL); + if (l->sg_post) + l->sg_txt = l->sg_post; + else if (rl_end > 0) + { + e = rl_suggest_find(state, l); + if (e) + l->sg_txt = e + rl_end; + } + if (l->sg_txt && !*l->sg_txt) + l->sg_txt = NULL; + return (l->sg_txt); +} + +/* What the accept keys may insert right now: the ghost of the last frame, + if the line still is what it was drawn for. Keys can arrive faster than + frames (a paste), so the entry is matched against the line again. */ +char *rl_suggest_live(t_rl *l) +{ + size_t n; + + n = (size_t)rl_end; + if (!l->sg_txt || rl_point != rl_end) + return (NULL); + if (l->sg_post && !ft_strcmp(l->sg_post_at, rl_line_buffer)) + return (l->sg_post); + if (!l->sg_post && l->sg_ent && ft_strlen(l->sg_ent) > n + && !ft_strncmp(l->sg_ent, rl_line_buffer, n)) + return (l->sg_ent + n); + return (NULL); +} + +/* How many bytes of `s` fit in `avail` columns, cut on a character + boundary (a half-written UTF-8 sequence would print as garbage), and + their width in `used`. A byte that is not a character stops it. */ +size_t rl_ghost_fit(const char *s, int avail, int *used) +{ + mbstate_t st; + wchar_t wc; + size_t i; + size_t n; + int w; + + ft_memset(&st, 0, sizeof(st)); + i = 0; + *used = 0; + while (s[i]) + { + n = mbrtowc(&wc, s + i, MB_CUR_MAX, &st); + if (n == 0 || n >= (size_t)(-2)) + break ; + w = wcwidth(wc); + if (w < 0 || *used + w > avail) + break ; + *used += w; + i += n; + } + return (i); +} diff --git a/src/platform/posix/rl_suggest3.c b/src/platform/posix/rl_suggest3.c new file mode 100644 index 00000000..ae6ebbf1 --- /dev/null +++ b/src/platform/posix/rl_suggest3.c @@ -0,0 +1,102 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_suggest3.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include "zle.h" +#include "env.h" + +/* The accept keys and POSTDISPLAY. */ + +/* What the key just pressed does when there is nothing to accept: the + stock function of its accept slot (End goes to the end of the line, the + rest move one character). */ +static rl_command_func_t *sg_stock(const char *keyseq) +{ + int i; + + i = 0; + while (keyseq && i < RL_NSLOTS) + { + if (rl_slot_at(i).feat == FEAT_ACCEPT && rl_slot_at(i).fn == SLOT_END + && !ft_strcmp(rl_slot_at(i).key, keyseq)) + return (rl_end_of_line); + i++; + } + return (rl_forward_char); +} + +/* Right, End and ^F while HELLISH_AUTOSUGGEST is on: take the ghost into +** the line when there is one, otherwise be the key's usual self. The +** whole suggestion is inserted, not just what was visible of it. */ +int rl_suggest_accept(int count, int key) +{ + t_shell **cell; + char *g; + + cell = zle_state_cell(); + g = NULL; + if (*cell) + g = rl_suggest_live(&(*cell)->rl); + if (g && *g) + { + rl_insert_text(g); + return (0); + } + return ((*sg_stock(rl_executing_keyseq))(count, key)); +} + +/* Hand a widget the text after the cursor as the variable POSTDISPLAY, and + remember what it was given to tell, afterwards, whether it was changed. */ +void rl_suggest_publish(t_shell *state) +{ + t_rl *l; + char *g; + + l = &state->rl; + g = rl_suggest_live(l); + if (!g) + g = ""; + xfree(l->sg_pub); + l->sg_pub = ft_strdup(g); + env_set(&state->env, env_create(ft_strdup("POSTDISPLAY"), + ft_strdup(g), false)); +} + +/* A widget assigned POSTDISPLAY: show that, for the line as the widget + left it. */ +static void sg_post_set(t_rl *l, const char *v) +{ + l->sg_txt = NULL; + xfree(l->sg_post); + xfree(l->sg_post_at); + l->sg_post = ft_strdup(v); + l->sg_post_at = ft_strdup(rl_line_buffer); + rl_rprompt_hook(); +} + +/* After the widget, and after its BUFFER was installed (the ghost belongs + to the new line): a POSTDISPLAY different from what was published is the + widget's. */ +void rl_suggest_collect(t_shell *state) +{ + t_rl *l; + char *v; + + l = &state->rl; + v = env_expand(state, "POSTDISPLAY"); + if (!v) + v = ""; + if (l->sg_pub && ft_strcmp(v, l->sg_pub)) + sg_post_set(l, v); + xfree(l->sg_pub); + l->sg_pub = NULL; +} diff --git a/src/platform/posix/rl_suggest4.c b/src/platform/posix/rl_suggest4.c new file mode 100644 index 00000000..b3781c61 --- /dev/null +++ b/src/platform/posix/rl_suggest4.c @@ -0,0 +1,98 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* rl_suggest4.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/10/17 12:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/10/17 12:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "rl_private.h" +#include "zle.h" + +/* Taking the ghost down, and forgetting it. */ + +/* Wipe the ghost from the screen without touching the text: from the + column it was painted at, to the end of the row, with the cursor put + back after. Used when the line is about to be accepted or the read + interrupted -- neither of which gives the painter another frame -- so + that no ghost ends up in the scrollback. */ +void rl_suggest_unpaint(void) +{ + t_shell **cell; + t_string f; + + cell = zle_state_cell(); + if (!*cell || !(*cell)->rl.sg_painted) + return ; + (*cell)->rl.sg_painted = false; + vec_init(&f); + f.elem_size = 1; + vec_push_str(&f, "\0337"); + if (rl_push_col(&f, (*cell)->rl.sg_col + 1)) + { + vec_push_str(&f, "\033[K\0338"); + rl_out_write(f.ctx, f.len); + } + xfree(f.ctx); +} + +/* A resize wiped the screen under the line: nothing of the ghost is + there any more, and the next frame paints it afresh (or not). */ +void rl_suggest_stale(void) +{ + t_shell **cell; + + cell = zle_state_cell(); + if (*cell) + (*cell)->rl.sg_painted = false; +} + +/* The read is over (or about to start): nothing of the ghost, of the + search it was found with or of a widget's POSTDISPLAY outlives it. The + history may have changed since, so the cache cannot either. */ +void rl_suggest_end(t_shell *state) +{ + t_rl *l; + + l = &state->rl; + xfree(l->sg_ent); + xfree(l->sg_prev); + xfree(l->sg_post); + xfree(l->sg_post_at); + xfree(l->sg_pub); + l->sg_ent = NULL; + l->sg_prev = NULL; + l->sg_post = NULL; + l->sg_post_at = NULL; + l->sg_pub = NULL; + l->sg_txt = NULL; + l->sg_known = false; + l->sg_painted = false; +} + +/* Columns taken by everything left of the text on the prompt's row: the + row readline was handed, and the mode string it puts in front of it + (the "(ins)" of vi mode) when show-mode-in-prompt is on. */ +int rl_prompt_cols(t_rl *l) +{ + const char *on; + const char *name; + const char *v; + + on = rl_variable_value("show-mode-in-prompt"); + if (!on || ft_strcmp(on, "on")) + return (l->rp_prompt_w); + name = "emacs-mode-string"; + if (rl_get_keymap() == vi_insertion_keymap) + name = "vi-ins-mode-string"; + if (rl_get_keymap() == vi_movement_keymap) + name = "vi-cmd-mode-string"; + v = rl_variable_value(name); + if (!v) + return (l->rp_prompt_w); + return (l->rp_prompt_w + visible_width_cstr(v)); +} diff --git a/src/platform/posix/rl_winch.c b/src/platform/posix/rl_winch.c index 5e90539c..3b912dbc 100644 --- a/src/platform/posix/rl_winch.c +++ b/src/platform/posix/rl_winch.c @@ -65,7 +65,7 @@ int rl_winch_cols(void) { struct winsize ws; - if (ioctl(fileno(rl_outstream), TIOCGWINSZ, &ws) != 0 || ws.ws_col == 0) + if (ioctl(STDERR_FILENO, TIOCGWINSZ, &ws) != 0 || ws.ws_col == 0) return (-1); return (ws.ws_col); } diff --git a/src/platform/posix/rl_winch2.c b/src/platform/posix/rl_winch2.c index de474ff4..b30ae1e7 100644 --- a/src/platform/posix/rl_winch2.c +++ b/src/platform/posix/rl_winch2.c @@ -41,7 +41,8 @@ int rl_winch_poll(void) return (0); st->cols = cols; rl_clear_visible_line(); - tty_write_all(fileno(rl_outstream), "\033[J", 3); + rl_out_write("\033[J", 3); + rl_suggest_stale(); rl_reset_screen_size(); rl_forced_update_display(); return (0); diff --git a/src/platform/posix/zle_rl.c b/src/platform/posix/zle_rl.c index 8b13813e..f5e5a94e 100644 --- a/src/platform/posix/zle_rl.c +++ b/src/platform/posix/zle_rl.c @@ -70,6 +70,7 @@ void zle_publish(t_shell *state) ft_strdup(s + rl_point), false)); env_set(&state->env, env_create(ft_strdup("CURSOR"), ft_itoa(rl_point), false)); + rl_suggest_publish(state); } /* Read the variables back and install them in readline. @@ -118,6 +119,7 @@ static void zle_run(t_shell *state, t_zle_widget *w) rl_shell_exec(state, w->fn); if (was) zle_collect(state, was); + rl_suggest_collect(state); zle_params_restore(state, &saved); xfree(was); } @@ -138,7 +140,10 @@ int zle_dispatch(int count, int key) w = zle_widget_get(name); if (!w) return (0); - zle_run(state, w); + if (w->bashx) + zle_run_x(state, w); + else + zle_run(state, w); if (zle_active()) rl_redisplay(); return (0); diff --git a/src/platform/posix/zle_rl2.c b/src/platform/posix/zle_rl2.c index b03bb59c..bea0f134 100644 --- a/src/platform/posix/zle_rl2.c +++ b/src/platform/posix/zle_rl2.c @@ -69,5 +69,6 @@ void zle_do_kill_buffer(void) could disagree with it about trailing state. */ void zle_do_accept_line(void) { + rl_suggest_unpaint(); rl_done = 1; } diff --git a/src/platform/posix/zle_rl4.c b/src/platform/posix/zle_rl4.c new file mode 100644 index 00000000..981ce22e --- /dev/null +++ b/src/platform/posix/zle_rl4.c @@ -0,0 +1,90 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* zle_rl4.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "shell.h" +#include "libft.h" +#include "zle.h" +#include + +/* Built-in widgets: a key bound to one of the editor's own commands. +** +** bindkey '^[[A' history-beginning-search-backward +** +** is how every zsh config asks for "Up searches the history for lines +** that start with what I typed" (#136). Only shell-function widgets used +** to work: the name above bound the key to the shared dispatcher, which +** found no widget by that name and did nothing -- the key went dead. +** +** Here the name is resolved to readline's own function and the key is +** bound to THAT, not to the dispatcher. It has to be direct: readline's +** history searches keep their place by checking that the previous command +** was themselves (rl_last_func), and a key routed through the dispatcher +** would make every press a new search for the line the last one found. +** +** zsh and readline name most of these alike; the table is the ones they do +** not. A name readline does not know, and a widget a plugin registered +** under a built-in's name (zsh-history-substring-search does exactly that), +** stay with the dispatcher. */ +static const char *zle_rl_name(const char *w) +{ + static const char *map[][2] = { + {"history-beginning-search-backward", "history-search-backward"}, + {"history-beginning-search-forward", "history-search-forward"}, + {"up-line-or-beginning-search", "history-search-backward"}, + {"down-line-or-beginning-search", "history-search-forward"}, + {"history-substring-search-up", "history-substring-search-backward"}, + {"history-substring-search-down", "history-substring-search-forward"}, + {"history-incremental-search-backward", "reverse-search-history"}, + {"history-incremental-search-forward", "forward-search-history"}, + {"up-line-or-history", "previous-history"}, + {"down-line-or-history", "next-history"}, + {"up-history", "previous-history"}, + {"down-history", "next-history"}, + {"expand-or-complete", "complete"}, + {"complete-word", "complete"}, + {"delete-char-or-list", "delete-char"}, + {NULL, NULL}}; + int i; + + i = 0; + while (map[i][0] && ft_strcmp(map[i][0], w)) + i++; + if (map[i][0]) + return (map[i][1]); + return (w); +} + +/* readline's function for built-in widget `w`, or NULL. zsh's `.name` + is the built-in version of a widget a plugin may have wrapped. */ +t_zle_fn zle_builtin_func(const char *w) +{ + if (!w || !*w) + return (NULL); + if (w[0] == '.') + w++; + return (rl_named_function(zle_rl_name(w))); +} + +/* Bind one recorded `bindkey`: a widget the shell defined runs through + the dispatcher, a built-in one is readline's own function. */ +void zle_bind_install(t_zle_bind *b) +{ + t_zle_fn f; + + zle_bind_raw(b); + f = NULL; + if (!zle_widget_get(b->widget)) + f = zle_builtin_func(b->widget); + if (!f) + f = zle_dispatch; + rl_bind_keyseq(b->seq, f); +} diff --git a/src/platform/posix/zle_rl5.c b/src/platform/posix/zle_rl5.c new file mode 100644 index 00000000..aaf18af9 --- /dev/null +++ b/src/platform/posix/zle_rl5.c @@ -0,0 +1,119 @@ +/* ************************************************************************** */ +/* */ +/* ::: :::::::: */ +/* zle_rl5.c :+: :+: :+: */ +/* +:+ +:+ +:+ */ +/* By: dlesieur +#+ +:+ +#+ */ +/* +#+#+#+#+#+ +#+ */ +/* Created: 2026/09/25 00:00:00 by dlesieur #+# #+# */ +/* Updated: 2026/09/25 00:00:00 by dlesieur ### ########.fr */ +/* */ +/* ************************************************************************** */ + +#include "shell.h" +#include "libft.h" +#include "zle.h" +#include "env.h" +#include "ft_builtins.h" +#include + +/* `bind -x`: a key that runs a shell command, the way bash runs it +** (bash_execute_unix_command). The command sees the line as READLINE_LINE, +** the cursor as READLINE_POINT and the mark as READLINE_MARK -- counted in +** CHARACTERS, not bytes -- and whatever it leaves in them becomes the line, +** the cursor and the mark. The visible line is cleared first, so the +** command's output starts on a clean row, and the prompt and line are drawn +** again after it. The three names exist only while the command runs. +** fzf's key bindings for bash are built on exactly this. */ + +/* Characters before byte offset `bytes`, and the byte offset of character + `chars` (the end of the line past the last one). UTF-8: a character is + every byte that is not a continuation byte. */ +static int zle_x_chars(const char *s, int bytes) +{ + int i; + int n; + + i = 0; + n = 0; + while (i < bytes && s[i]) + if (((unsigned char)s[i++] & 0xC0) != 0x80) + n++; + return (n); +} + +static int zle_x_bytes(const char *s, int chars) +{ + int i; + int n; + + i = 0; + n = 0; + while (s[i]) + { + if (((unsigned char)s[i] & 0xC0) != 0x80 && n++ == chars) + return (i); + i++; + } + return (i); +} + +static void zle_x_publish(t_shell *state) +{ + const char *s; + + s = rl_line_buffer; + if (!s) + s = ""; + env_set(&state->env, env_create(ft_strdup("READLINE_LINE"), + ft_strdup(s), true)); + env_set(&state->env, env_create(ft_strdup("READLINE_POINT"), + ft_itoa(zle_x_chars(s, rl_point)), true)); + env_set(&state->env, env_create(ft_strdup("READLINE_MARK"), + ft_itoa(zle_x_chars(s, rl_mark)), true)); +} + +/* The number in `name`, as a byte offset into the new line, or -1 when + it is unset or not a number -- bash leaves the position alone then. */ +static int zle_x_offset(t_shell *state, const char *name) +{ + char *v; + int i; + + v = env_expand(state, (char *)name); + if (!v || !*v) + return (-1); + i = (v[0] == '-'); + while (v[i] && ft_isdigit(v[i])) + i++; + if (v[i] || ft_atoi(v) < 0) + return (-1); + return (zle_x_bytes(rl_line_buffer, ft_atoi(v))); +} + +void zle_run_x(t_shell *state, t_zle_widget *w) +{ + char *line; + int at; + + rl_clear_visible_line(); + fflush(rl_outstream); + zle_x_publish(state); + rl_shell_exec(state, w->fn); + line = env_expand(state, "READLINE_LINE"); + if (line && ft_strcmp(line, rl_line_buffer)) + { + rl_replace_line(line, 0); + rl_point = rl_end; + } + at = zle_x_offset(state, "READLINE_POINT"); + if (at >= 0) + rl_point = at; + at = zle_x_offset(state, "READLINE_MARK"); + if (at >= 0) + rl_mark = at; + try_unset(state, "READLINE_LINE"); + try_unset(state, "READLINE_POINT"); + try_unset(state, "READLINE_MARK"); + rl_forced_update_display(); +} diff --git a/tests/.mini_install.sh b/tests/.mini_install.sh new file mode 100644 index 00000000..b1fc4308 --- /dev/null +++ b/tests/.mini_install.sh @@ -0,0 +1,5 @@ +say(){ printf '\033[1;36mhellish:\033[0m %s\n' "$*"; } +say 'Installing opencode version 1.2.3 for x86_64' +payload=$(printf '%4067s' '') +payload="${payload}Manually add this path to PATH" +say "$payload" diff --git a/tests/assign_shaped_words b/tests/assign_shaped_words index d6fa9696..376b7456 100644 --- a/tests/assign_shaped_words +++ b/tests/assign_shaped_words @@ -44,3 +44,17 @@ x=5 e=( $x $((x+1)) ); echo "${e[@]}" A=old; A=new e=( B=$A ); echo "${e[@]}" "$A" A=1 e=( B=$A ) sh -c 'echo "$A" "$B"' A=1 e=( B=$A ) true; echo "[$A]" "[${e[*]}]" +f(){ local v=$1 w=${1:+ }; printf '[%s][%s]\n' "$v" "$w"; }; f 'x y' +f(){ declare v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ typeset v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ readonly v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ declare -x v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ local -r v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ local old=$1 new=$2 space=${2:+ }; printf '[%s][%s][%s]\n' "$old" "$new" "$space"; }; f vim 'sudo -e' +x='a *'; f(){ local v=$x; printf '[%s]\n' "$v"; }; f +f(){ command local v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ command declare v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ builtin local v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ command -p declare v=$1; printf '[%s]\n' "$v"; }; f 'x y' +f(){ local $1; k1=a k2=b; printf '[%s][%s]\n' "$k1" "$k2"; }; f 'k1 k2'; printf '[%s]\n' "${k1-gone}" +set -- 'p q'; declare g=$1; readonly r=$1; printf '[%s][%s]\n' "$g" "$r" diff --git a/tests/autosuggest_test.py b/tests/autosuggest_test.py new file mode 100644 index 00000000..a7765f87 --- /dev/null +++ b/tests/autosuggest_test.py @@ -0,0 +1,623 @@ +#!/usr/bin/env python3 +"""Inline autosuggestion (#140): the newest history entry that starts with +what is typed, dim after the cursor, zsh-autosuggestions style. + +OFF unless HELLISH_AUTOSUGGEST is 1 / on. Right, End and ^F accept it; with +nothing to accept they are the plain keys. It is never part of the line, so +what is checked is what a terminal would show: a pyte screen for the cells +right of the cursor and the cursor column, the raw bytes for the dim +attribute (pyte has no SGR 2), and the output of the lines that are run. + +Cells: both readers (in-process and HELLISH_RL_FORK=1). Each case gets a +fresh shell and a history file of its own. + +Usage: python3 autosuggest_test.py /path/to/hellish [case-name-part [inproc]] +""" +import fcntl +import os +import pty +import select +import signal +import struct +import sys +import tempfile +import termios +import time + +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else "../build/bin/hellish") +FAILS = [] +ROWS = 30 +PROMPT = "IN#> " +PW = len(PROMPT) +DIM = b"\x1b[2m" +ESC = "\x1b" +RIGHT, RIGHT_O = "\x1b[C", "\x1bOC" +END, END_O = "\x1b[F", "\x1bOF" +LEFT = "\x1b[D" +CTRL_F, CTRL_U, CTRL_C, CTRL_D, CTRL_R, CTRL_G = ( + "\x06", "\x15", "\x03", "\x04", "\x12", "\x07") +BKSP = "\x7f" +ON = "HELLISH_AUTOSUGGEST=1" + +try: + import pyte + + class Screen(pyte.Screen): + """pyte dispatches CSI with parameters; save/restore take none.""" + + def save_cursor(self, *_): + super().save_cursor() + + def restore_cursor(self, *_): + super().restore_cursor() + + HAVE_PYTE = True +except ImportError: # pragma: no cover + HAVE_PYTE = False + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + + (" " + detail if not ok else "")) + if not ok: + FAILS.append(name) + + +class Session: + def __init__(self, home, env=None, rc=None, cols=100, fork=False): + e = { + "HOME": home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "xterm-256color", "LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", + "HELLISH_BANNER": "0", "HELLISH_NO_BANNER": "1", + "HELLISH_NO_UPDATE_CHECK": "1", "HELLISH_NO_ANIM": "1", + "HISTFILE": os.path.join(home, "hist"), "PS1": PROMPT, + "ASAN_OPTIONS": "detect_leaks=" + os.environ.get("AS_LEAKS", "0"), + } + if fork: + e["HELLISH_RL_FORK"] = "1" + e.update(env or {}) + args = [SHELL, "--norc"] + if rc: + path = os.path.join(home, "rc") + with open(path, "w") as f: + f.write(rc) + args = [SHELL, "--rcfile=" + path] + self.cols = cols + self.buf = bytearray() + self.screen = Screen(cols, ROWS) + self.stream = pyte.ByteStream(self.screen) + self.pid, self.fd = pty.fork() + if self.pid == 0: + fcntl.ioctl(0, termios.TIOCSWINSZ, + struct.pack("HHHH", ROWS, cols, 0, 0)) + os.chdir(home) + os.execve(SHELL, args, e) + os._exit(127) + self.until(lambda: b"#> " in self.buf, 10.0) + self.settle() + + def pump(self, secs): + r, _, _ = select.select([self.fd], [], [], secs) + if not r: + return False + try: + d = os.read(self.fd, 65536) + except OSError: + return False + if not d: + return False + self.buf.extend(d) + self.stream.feed(d) + return True + + def until(self, pred, cap): + end = time.monotonic() + cap + while not pred() and time.monotonic() < end: + self.pump(0.05) + return pred() + + def settle(self, quiet=0.25, cap=3.0): + end = time.monotonic() + cap + while time.monotonic() < end and self.pump(quiet): + pass + + def run(self, line): + n = self.buf.count(b"#> ") + os.write(self.fd, line.encode() + b"\r") + self.until(lambda: self.buf.count(b"#> ") > n, 8.0) + self.settle() + + def keys(self, data): + if isinstance(data, str): + data = data.encode() + self.mark = len(self.buf) + os.write(self.fd, data) + self.settle() + + def since(self): + """The bytes the terminal got since the last keys().""" + return bytes(self.buf[self.mark:]) + + def row(self, y=None): + y = self.screen.cursor.y if y is None else y + line = self.screen.buffer[y] + return "".join(line[x].data for x in range(self.cols)).rstrip() + + def x(self): + return self.screen.cursor.x + + def ghost(self): + """What is on the cursor's row right of the cursor.""" + y = self.screen.cursor.y + line = self.screen.buffer[y] + return "".join(line[x].data + for x in range(self.x(), self.cols)).rstrip() + + def below_blank(self): + y = self.screen.cursor.y + return all(self.row(r) == "" for r in range(y + 1, ROWS)) + + def screen_text(self): + return "\n".join(self.row(y) for y in range(ROWS)) + + def alive(self): + try: + pid, _ = os.waitpid(self.pid, os.WNOHANG) + except ChildProcessError: + return False + return pid == 0 + + def close(self): + try: + os.write(self.fd, b"\x05\x15exit\r") + self.until(lambda: not self.alive(), 3.0) + self.settle(0.2, 1.0) + os.kill(self.pid, 9) + except OSError: + pass + check("sanitizer: no AddressSanitizer / LeakSanitizer report", + b"Sanitizer" not in self.buf, repr(bytes(self.buf[-300:]))) + try: + os.waitpid(self.pid, 0) + except ChildProcessError: + pass + os.close(self.fd) + + +def shell(fork, seed=(), on=True, **kw): + """A shell with autosuggest on (unless `on` is False) and `seed` run.""" + home = tempfile.mkdtemp(prefix="autosuggest_") + s = Session(home, fork=fork, **kw) + if on: + s.run(ON) + for line in seed: + s.run(line) + s.run("clear") + return s + + +def case_off_by_default(fork, tag): + s = shell(fork, ["echo hello"], on=False) + s.keys("ech") + check(tag + "1 off by default: nothing right of the cursor", + s.ghost() == "" and DIM not in s.since(), repr(s.ghost())) + s.close() + + +def case_on(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys("ech") + check(tag + "2 on: ghost right of the cursor", + s.ghost() == "o hello", repr(s.ghost())) + check(tag + "2 on: the cursor stays after the typed text", + s.x() == PW + 3, str(s.x())) + check(tag + "2 on: painted dim", DIM + b"o hello" in s.since(), + repr(s.since())) + check(tag + "2 on: ghost is not part of the line", + s.row() == PROMPT + "echo hello", s.row()) + s.keys(CTRL_U) + check(tag + "2 on: ^U takes it away", s.ghost() == "", s.ghost()) + s.close() + + +def case_accept(fork, tag): + for name, key in (("Right", RIGHT), ("SS3 Right", RIGHT_O), + ("End", END), ("SS3 End", END_O), + ("^F", CTRL_F)): + s = shell(fork, ["echo hello"]) + s.keys("ech") + s.keys(key) + check(tag + "3 %s accepts: line and cursor" % name, + s.row() == PROMPT + "echo hello" and s.x() == PW + 10 + and s.ghost() == "", "%r x=%d" % (s.row(), s.x())) + start = len(s.buf) + s.run("") + out = bytes(s.buf[start:]).decode("utf-8", "replace") + check(tag + "3 %s: Enter runs the accepted line" % name, + "hello\n" in out.replace("\r", ""), repr(out)) + s.close() + + +def case_enter_plain(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys("ech") + s.keys("\r") + s.settle() + rows = [s.row(y) for y in range(ROWS)] + mine = [r for r in rows if r.startswith(PROMPT + "ech")] + check(tag + "4 Enter without accepting runs only what was typed", + "hello" not in s.screen_text() and mine + and all(r == PROMPT + "ech" for r in mine), repr(rows[:6])) + s.close() + + +def case_newest(fork, tag): + s = shell(fork, ["echo one", "echo two"]) + s.keys("echo ") + check(tag + "5 most recent wins", s.ghost() == "two", repr(s.ghost())) + s.close() + + +def case_edit(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys("xyz") + check(tag + "6 no match, no ghost", s.ghost() == "", repr(s.ghost())) + s.keys(CTRL_U + "ech") + s.keys("q") + check(tag + "6 diverging clears it", s.ghost() == "", repr(s.ghost())) + s.keys(BKSP) + check(tag + "6 Backspace to a match brings it back", + s.ghost() == "o hello", repr(s.ghost())) + s.keys(BKSP * 2) + check(tag + "6 shorter prefix, longer ghost", + s.ghost() == "cho hello", repr(s.ghost())) + s.keys(BKSP) + check(tag + "6 empty line, no ghost", s.ghost() == "", repr(s.ghost())) + s.close() + + +def case_cursor(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys("ech") + s.keys(LEFT) + check(tag + "7 cursor not at the end: no ghost", + s.ghost() == "h" and s.row() == PROMPT + "ech" + and DIM not in s.since(), repr((s.ghost(), s.row()))) + s.keys(RIGHT) + check(tag + "7 Right mid-line only moves the cursor", + s.x() == PW + 3 and s.row() == PROMPT + "echo hello", + "x=%d %r" % (s.x(), s.row())) + s.keys(RIGHT) + check(tag + "7 Right at the end accepts", + s.row() == PROMPT + "echo hello", s.row()) + s.keys(CTRL_U + "zzz" + LEFT * 2) + s.keys(END) + check(tag + "7 End with no ghost is end-of-line", + s.x() == PW + 3 and s.row() == PROMPT + "zzz", str(s.x())) + s.keys(LEFT) + s.keys(CTRL_F) + check(tag + "7 ^F with no ghost is forward-char", + s.x() == PW + 3 and s.row() == PROMPT + "zzz", str(s.x())) + s.close() + + +def case_interrupt(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys("ech") + s.keys(CTRL_C) + check(tag + "8 ^C leaves no ghost on the old row", + all("o hello" not in s.row(y) for y in range(ROWS)), + repr(s.screen_text()[:120])) + check(tag + "8 ^C: a clean prompt", s.row() == PROMPT.rstrip(), + repr(s.row())) + s.keys("ech") + check(tag + "8 ^C: the next prompt still suggests", + s.ghost() == "o hello", repr(s.ghost())) + s.keys(CTRL_U) + s.keys(CTRL_D) + s.until(lambda: not s.alive(), 3.0) + check(tag + "8 ^D on an empty line exits", not s.alive()) + s.close() + + +def case_history_keys(fork, tag): + s = shell(fork, ["echo one", "echo two", "ls /nonexistent_zz"]) + s.keys("echo") + s.keys(b"\x1b[A") + check(tag + "9 Up still searches by prefix", + s.row() == PROMPT + "echo two", s.row()) + s.keys(b"\x1b[A") + check(tag + "9 Up again", s.row() == PROMPT + "echo one", s.row()) + s.keys(b"\x1b[B") + check(tag + "9 Down", s.row() == PROMPT + "echo two", s.row()) + s.close() + + +def case_search(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys(CTRL_R) + s.keys("ech") + check(tag + "10 no ghost while searching", + DIM not in s.since() and "reverse-i-search" in s.row(), + repr(s.row())) + s.keys(CTRL_G) + s.keys("ech") + check(tag + "10 and again once it is over", + s.ghost() == "o hello", repr(s.ghost())) + s.close() + + +def case_vi(fork, tag): + s = shell(fork, ["echo hello"]) + s.run("set -o vi") + s.keys("ech") + check(tag + "11 vi insert mode shows the ghost", + s.ghost() == "o hello", repr(s.ghost())) + s.keys(ESC) + time.sleep(0.8) + s.settle() + check(tag + "11 vi command mode shows none", + s.row() == "(cmd)" + PROMPT + "ech", repr(s.row())) + s.keys("ao") + check(tag + "11 back in insert mode it is back", + s.ghost() == " hello" and s.x() == 5 + PW + 4, repr(s.ghost())) + s.close() + + +def case_escape(fork, tag): + s = Session(tempfile.mkdtemp(prefix="autosuggest_"), fork=fork) + s.run(ON) + s.run("echo ok") + s.keys(b"echo \x16\x1b[31mRED\x16\x1b[0m # zzz\r") + s.settle() + s.run("clear") + s.keys("echo") + row = s.screen.buffer[s.screen.cursor.y] + check(tag + "12 an entry holding ESC is never the ghost", + s.ghost() == " ok", repr(s.ghost())) + check(tag + "12 no ESC from the entry reaches the terminal", + b"\x1b[31m" not in s.since() and b"zzz" not in s.since(), + repr(s.since())) + check(tag + "12 no colour on the row", + all(row[x].fg == "default" for x in range(s.cols))) + s.keys(CTRL_U) + s.run("clear") + s.run("printf '\\e[31mRED\\e[0m\\n' # yyy") + s.run("clear") + s.keys("printf") + check(tag + "12 a literal backslash-e stays text, no colour", + b"\x1b[31m" not in s.since() + and all(s.screen.buffer[s.screen.cursor.y][x].fg == "default" + for x in range(s.cols)), repr(s.since())) + s.close() + + +def case_multiline(fork, tag): + s = shell(fork, []) + s.keys("for i in 1 2\r") + s.keys("do echo L$i\r") + s.keys("done\r") + s.settle() + s.run("history -s $'while false\\ndo :\\ndone'") + s.run("clear") + s.keys("while") + check(tag + "13 an entry with a newline is never suggested", + s.ghost() == "" and DIM not in s.since(), repr(s.ghost())) + s.keys(CTRL_U + "for") + check(tag + "13 a typed loop never paints a newline", + "\n" not in s.ghost() and s.below_blank(), repr(s.ghost())) + s.close() + + +def case_truncate(fork, tag): + s = shell(fork, ["echo " + "x" * 60], cols=40) + s.keys("echo") + check(tag + "14 long ghost is cut to the row", + s.ghost() == " " + "x" * 29 and s.x() == PW + 4 + and s.below_blank(), "x=%d %r" % (s.x(), s.ghost())) + s.keys(CTRL_U) + s.close() + + +def case_utf8(fork, tag): + s = shell(fork, ["echo h\u00e9llo w\u00f6rld"]) + s.keys("echo h\u00e9") + check(tag + "15 UTF-8: ghost right after the typed text", + s.ghost() == "llo w\u00f6rld" and s.x() == PW + 7, + "x=%d %r" % (s.x(), s.ghost())) + s.keys(RIGHT) + check(tag + "15 UTF-8: accepted whole", + s.row() == PROMPT + "echo h\u00e9llo w\u00f6rld", s.row()) + s.close() + + +def case_rprompt(fork, tag): + s = shell(fork, ["echo hello", "echo " + "y" * 95], + env={"RPROMPT": "RP"}) + s.keys("echo h") + check(tag + "16 ghost and RPROMPT both shown", + s.row().startswith(PROMPT + "echo hello") + and s.row().endswith("RP") and s.x() == PW + 6, repr(s.row())) + s.keys(CTRL_U + "echo") + check(tag + "16 ghost reaching RPROMPT hides it", + not s.row().endswith("RP") and "yyy" in s.row() + and s.below_blank(), repr(s.row())) + s.keys(CTRL_U) + check(tag + "16 RPROMPT comes back with the ghost gone", + s.row().endswith("RP") and s.row().startswith(PROMPT), + repr(s.row())) + s.close() + + +def case_paste(fork, tag): + s = shell(fork, ["echo " + "p" * 100]) + t0 = time.monotonic() + s.keys(b"echo " + b"p" * 4000) + s.settle() + check(tag + "17 4 KB paste: no hang", s.alive() + and time.monotonic() - t0 < 10.0) + s.keys(CTRL_U) + s.run("echo done") + check(tag + "17 4 KB paste: shell usable, no garbage rows", + "ppp" not in s.screen_text() and "done" in s.screen_text(), + repr(s.screen_text()[:200])) + s.close() + + +def case_max(fork, tag): + s = Session(tempfile.mkdtemp(prefix="autosuggest_"), fork=fork) + s.run(ON) + s.run("HELLISH_SUGGEST_MAX=1") + s.run("echo old") + s.run("ls /nonexistent_zz") + s.run("clear") + s.keys("echo") + check(tag + "18 SUGGEST_MAX=1 looks at the newest entry only", + s.ghost() == "", repr(s.ghost())) + s.keys(CTRL_U) + s.run("HELLISH_SUGGEST_MAX=abc") + s.keys("echo") + check(tag + "18 non-numeric is the default", + s.ghost() == " old", repr(s.ghost())) + s.keys(CTRL_U) + s.run("HELLISH_SUGGEST_MAX=0") + s.keys("echo") + check(tag + "18 zero is the default", + s.ghost() == " old", repr(s.ghost())) + s.close() + + +def case_unset(fork, tag): + s = shell(fork, ["echo hello"]) + s.keys("ech") + check(tag + "19 on", s.ghost() == "o hello", repr(s.ghost())) + s.keys(CTRL_U) + s.run("unset HELLISH_AUTOSUGGEST") + s.keys("ech") + check(tag + "19 unset: next line has no ghost", + s.ghost() == "" and DIM not in s.since(), repr(s.ghost())) + s.keys(RIGHT) + check(tag + "19 unset: Right is forward-char", + s.row() == PROMPT + "ech" and s.x() == PW + 3, s.row()) + s.keys(LEFT + RIGHT) + check(tag + "19 unset: Left, Right moves the cursor only", + s.x() == PW + 3 and s.row() == PROMPT + "ech", s.row()) + s.keys(CTRL_U) + s.run("HELLISH_AUTOSUGGEST=ON") + s.keys("ech") + check(tag + "19 ON (any case) turns it back on", + s.ghost() == "o hello", repr(s.ghost())) + s.keys(CTRL_U) + s.run("HELLISH_AUTOSUGGEST=0") + s.keys("ech") + check(tag + "19 0 is off", s.ghost() == "", repr(s.ghost())) + s.close() + + +def case_user_bind(fork, tag): + home = tempfile.mkdtemp(prefix="autosuggest_") + s = Session(home, fork=fork, rc="HELLISH_AUTOSUGGEST=1\n" + "bind '\"\\e[C\": backward-char'\n") + s.run("echo hello") + s.run("clear") + s.keys("ech") + check(tag + "20 the ghost is still shown", s.ghost() == "o hello", + repr(s.ghost())) + s.keys(RIGHT) + check(tag + "20 the user's binding wins on Right", + s.x() == PW + 2 and s.row() != PROMPT + "echo hello", s.row()) + s.keys("\x01\x0b" + "ech") + s.keys(END) + check(tag + "20 the other accept keys are still ours", + s.row() == PROMPT + "echo hello", s.row()) + s.close() + + +POST_RC = ("setopt zsh\n" + "HELLISH_AUTOSUGGEST=1\n" + "w() { POSTDISPLAY=' '; }\n" + "zle -N w\nbindkey '^X^H' w\n" + "r() { BUFFER=\"$BUFFER$POSTDISPLAY\"; }\n" + "zle -N r\nbindkey '^X^R' r\n") + + +def case_postdisplay(fork, tag): + home = tempfile.mkdtemp(prefix="autosuggest_") + s = Session(home, fork=fork, rc=POST_RC) + s.run("echo hello") + s.run("clear") + s.keys("abc") + s.keys(b"\x18\x08") + check(tag + "21 a widget's POSTDISPLAY is painted", + s.ghost() == "" or s.ghost() == " " + and s.x() == PW + 3, "x=%d %r" % (s.x(), s.ghost())) + check(tag + "21 painted dim", DIM + b" " in s.since(), + repr(s.since())) + s.keys("d") + check(tag + "21 gone when the buffer changes", s.ghost() == "", + repr(s.ghost())) + s.keys(CTRL_U + "ech") + s.keys(b"\x18\x12") + check(tag + "21 a widget reads the suggestion from POSTDISPLAY", + s.row() == PROMPT + "echo hello", s.row()) + s.keys("\x01\x0b") + s.keys(b"\x18\x12") + check(tag + "21 POSTDISPLAY is empty when there is none", + s.row() == PROMPT.rstrip(), repr(s.row())) + s.close() + + +def case_resize(fork, tag): + s = shell(fork, ["echo " + "z" * 50]) + s.keys("echo") + s.screen.resize(ROWS, 30) + s.cols = 30 + fcntl.ioctl(s.fd, termios.TIOCSWINSZ, struct.pack("HHHH", ROWS, 30, 0, 0)) + s.settle(0.4, 4.0) + g = s.ghost() + check(tag + "22 resize with a ghost: alive, never wrapped", + s.alive() and len(g) <= 30 - 1 - (PW + 4) + and set(g) <= set("z "), repr(g)) + s.keys(CTRL_U) + s.run("echo alive") + check(tag + "22 resize: shell still usable", "alive" in s.screen_text(), + repr(s.screen_text()[:100])) + s.close() + + +def case_dumb(fork, tag): + s = shell(fork, ["echo hello"], env={"TERM": "dumb"}) + s.keys("ech") + check(tag + "23 TERM=dumb: no ghost", + DIM not in s.since() and s.ghost() == "", repr(s.since())) + s.close() + + +CASES = (case_off_by_default, case_on, case_accept, case_enter_plain, + case_newest, case_edit, case_cursor, case_interrupt, + case_history_keys, case_search, case_vi, case_escape, + case_multiline, case_truncate, case_utf8, case_rprompt, + case_paste, case_max, case_unset, case_user_bind, + case_postdisplay, case_resize, case_dumb) + + +def main(): + if not HAVE_PYTE: + print("skip: needs pyte (python3 -m pip install pyte)") + sys.exit(0) + signal.signal(signal.SIGPIPE, signal.SIG_DFL) + only = sys.argv[2] if len(sys.argv) > 2 else "" + readers = (False,) if sys.argv[3:4] == ["inproc"] else (False, True) + for fork in readers: + tag = "forked/" if fork else "in-process/" + for case in CASES: + if only in case.__name__: + case(fork, tag) + if FAILS: + print("\n%d FAILED" % len(FAILS)) + sys.exit(1) + print("\nall clear") + + +main() diff --git a/tests/bind_builtin b/tests/bind_builtin new file mode 100644 index 00000000..379f4123 --- /dev/null +++ b/tests/bind_builtin @@ -0,0 +1,37 @@ +type bind +bind; echo rc=$? +bind --; echo rc=$? +bind -q nosuchfn; echo rc=$? +bind -Z; echo rc=$? +bind -q; echo rc=$? +bind -m; echo rc=$? +bind -f /nonexistent/x; echo rc=$? +bind -m nosuchmap -p >/dev/null; echo rc=$? +bind -u nosuchfn; echo rc=$? +bind '"\C-x\C-y": history-substring-search-backward'; bind -q history-substring-search-backward; echo rc=$? +bind -q history-substring-search-forward; echo rc=$? +bind '"\C-x\C-y": history-search-backward'; bind -p | grep -F '"\C-x\C-y"' +bind '"\C-x\C-y": history-search-backward' '"\C-x\C-t": history-search-forward'; bind -q history-search-forward | grep -c 'C-x.C-t' +bind -u forward-char; bind -q forward-char; echo rc=$? +bind '"\C-x\C-y": kill-whole-line'; bind -r '\C-x\C-y'; bind -q kill-whole-line; echo rc=$? +bind -m vi-insert '"\C-x\C-w": kill-whole-line'; bind -m vi-insert -q kill-whole-line; bind -q kill-whole-line; echo rc=$? +bind -m emacs -p | grep -c '^"\\C-a": beginning-of-line$' +bind 'set completion-ignore-case on'; bind -v | grep completion-ignore-case +bind 'set completion-ignore-case off'; bind -V | grep -c completion-ignore-case +bind -l | grep -c '^history-search-' +bind -lp | grep -c '^beginning-of-line$' +bind -X; echo rc=$? +bind -s; echo rc=$? +bind '"\C-x\C-m": "echo macro"'; bind -s | grep -F 'C-x\C-m' +f=$(mktemp); printf '"\\C-x\\C-e": end-of-line\n' > "$f"; bind -f "$f"; bind -p | grep -F '"\C-x\C-e"'; rm -f "$f" +bind -qforward-word | grep -c 'forward-word can be invoked' +bind -x '"\C-x\C-z": echo hi'; bind -X; echo rc=$? +bind -x 'C-x: echo'; echo rc=$? +bind -x '"\C-x: echo'; echo rc=$? +bind -x '"\C-x"echo'; echo rc=$? +bind -x '"\C-x" "echo two words"'; bind -X +bind -x '"\C-x":"echo q"'; bind -X +bind -x '"\C-x": printf "%s\n" $READLINE_LINE'; bind -X +bind -x '"\C-x": a'; bind -x '"\C-x": b'; bind -X +bind -x '"\C-x\C-z": echo hi'; bind -p | grep -c C-z +bind -x; echo rc=$? diff --git a/tests/bind_x_tty_test.py b/tests/bind_x_tty_test.py new file mode 100644 index 00000000..2829f410 --- /dev/null +++ b/tests/bind_x_tty_test.py @@ -0,0 +1,77 @@ +#!/usr/bin/env python3 +"""bind -x: a key that runs a shell command, at a real prompt. + + bind -x '"\\C-xa": READLINE_LINE="echo from-x $READLINE_LINE"' + +The command sees the line being edited as READLINE_LINE and the cursor as +READLINE_POINT (in characters, not bytes); what it leaves in them becomes +the line and the cursor, and whatever it prints appears on a clean row, +with the prompt and the line drawn again after it. fzf's key bindings for +bash are built on this. Every expectation below was checked against an +interactive bash 5.3.9 driven the same way. + +Waits on output, not on the clock. + +Usage: python3 bind_x_tty_test.py [/path/to/hellish] +""" +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] +RC = (r"""PS1='P$ ' +bind -x '"\C-xa": READLINE_LINE="echo from-x $READLINE_LINE"; READLINE_POINT=0' +bind -x '"\C-xb": echo OUT-B' +bind -x '"\C-xc": echo "P=$READLINE_POINT M=$READLINE_MARK"' +""") +CX = "\x18" + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def step(t, keys, want, name): + mark = len(t.out) + t.send(keys) + ok = t.expect(want) and t.expect(b"P$ ") + check(name, ok, repr(t.since(mark)[-300:])) + + +def main(): + t = Tty(SHELL, rc=RC, env={"LANG": "C.UTF-8"}) + check("start/prompt", t.expect(b"P$ "), repr(t.out[-300:])) + # the command rewrote the line and put the cursor at 0: what is typed + # next lands in front, so the line that runs is `echo echo from-x zz` + step(t, "zz" + CX + "a" + "echo \r", b"\necho from-x zz\r\n", + "line-and-point/come-back-from-the-command") + # a command that prints, in the middle of a line: the output, then the + # line as it was, still there to be run + mark = len(t.out) + t.send("echo keep" + CX + "b") + check("output/is-shown", t.expect(b"OUT-B"), repr(t.since(mark)[-300:])) + step(t, "\r", b"\nkeep\r\n", "output/the-line-survives-it") + # READLINE_POINT counts characters: e-acute is two bytes, one character + t.send(u"é12".encode("utf-8")) + mark = len(t.out) + t.send(CX + "c") + check("point/is-in-characters", t.expect(b"P=3 M=0"), + repr(t.since(mark)[-300:])) + t.send("\x15") + step(t, "echo \"[${READLINE_LINE-unset}${READLINE_POINT-}]\"\r", + b"\n[unset]\r\n", "variables/exist-only-while-it-runs") + st = t.close() + check("exit/not-signalled", os.WIFEXITED(st), "wait status %r" % st) + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/chunk_status b/tests/chunk_status new file mode 100644 index 00000000..f476e378 --- /dev/null +++ b/tests/chunk_status @@ -0,0 +1,13 @@ +eval $'. /dev/null; false\n# only a comment'; echo "rc=$?" +eval $'. /dev/null; false\n\n\n'; echo "rc=$?" +eval $'alias zq=true\nfalse\n# tail'; echo "rc=$?" +eval $'cat <&3 3< "$f"; . "$f"; echo "rc=$?"; rm -f "$f" +f=$(mktemp); printf 'cat < "$f"; . "$f"; echo "rc=$?"; rm -f "$f" diff --git a/tests/complete_make_test.py b/tests/complete_make_test.py new file mode 100755 index 00000000..236da2a9 --- /dev/null +++ b/tests/complete_make_test.py @@ -0,0 +1,477 @@ +#!/usr/bin/env python3 +"""`make ` completes target names read from the Makefile in the cwd. + +Issue #140, "make target completion" plus the provider architecture: an +ARGUMENT word that no `complete` spec claimed is offered to a per-command +provider (src/completion/complete_provider.c); `make` and `gmake` are the +first rows. The Makefile is parsed in-process -- nothing is forked and make +is never run -- so the rules below are the shell's own, pinned here. + +Each case builds a fresh temp directory holding only its fixture files and +drives a real pty. Assertions are on the EXACT completed line or the EXACT +listing, never on "contains". + +Cells: the in-process reader, and HELLISH_RL_FORK=1 (the forked reader). + +Usage: python3 complete_make_test.py /path/to/hellish [--asan] [-v] + --inproc skip the forked-reader cell (quicker, for mutation runs) + --asan finish each session with `exit` (so LeakSanitizer reports) + and run only cases 2 and 21. +""" +import fcntl +import os +import pty +import re +import select +import shutil +import struct +import subprocess +import sys +import tempfile +import termios +import time + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +VERBOSE = "-v" in sys.argv[1:] +ASAN = "--asan" in sys.argv[1:] +INPROC_ONLY = "--inproc" in sys.argv[1:] +ARGS = [a for a in sys.argv[1:] if not a.startswith("-")] +SHELL = os.path.abspath(ARGS[0] if ARGS + else os.path.join(ROOT, "build/bin/hellish")) +BASH = os.path.expanduser("~/bash-5.3.9/bin/bash") +FAILS = [] +EXTRA = {} +TMP = [] + +ANSI = re.compile(r"\x1b\[[0-9;?]*[A-Za-z]") +CRASH_MARKERS = ("free(): invalid", "double free", "corrupted", + "munmap_chunk", "AddressSanitizer", "LeakSanitizer", + "not malloc()-ed", "SIGSEGV", "Aborted") +SEEN_CRASH = [] + +mk = "Makefile" +TAB = b"\t" +ENTER = b"\n" +KILL = b"\x15" + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name) + if not ok: + if detail: + print(" " + detail.replace("\n", "\n ")) + FAILS.append(name) + + +def drain_idle(fd, chunks, quiet=0.35, cap=8.0): + """Read until the pty has been SILENT for `quiet` seconds (or `cap`). + + Not a fixed sleep. This file used to give the shell a flat wall-clock + budget per keystroke -- 0.8s to start, then 1.3s to complete a word -- + and that is a race, not a wait: it passes on an idle machine and fails + on a busy one. Run alone it was green; run after fifteen other pty + tests it reported three completion failures that were nothing but an + expired stopwatch, which is a harness bug wearing a shell bug's + clothes. + + Waiting for quiet is both more robust under load and faster when idle. + `cap` is the backstop so a genuinely wedged shell still fails instead + of hanging. + """ + last = time.time() + end = last + cap + while time.time() < end: + r, _, _ = select.select([fd], [], [], 0.05) + if r: + try: + c = os.read(fd, 65536) + except OSError: + break + if not c: + break + chunks.append(c) + last = time.time() + elif time.time() - last >= quiet: + break + return chunks + + +def spawn(cwd, home): + env = { + "HOME": home, "PATH": "/usr/bin:/bin", "TERM": "xterm-256color", + "LANG": "C.UTF-8", "PS1": "$ ", + "HELLISH_NO_BANNER": "1", "HELLISH_NO_UPDATE_CHECK": "1", + "HELLISH_NO_ANIM": "1", + "INPUTRC": "/dev/null", + "ASAN_OPTIONS": "detect_leaks=%d" % (1 if ASAN else 0), + } + env.update(EXTRA) + pid, fd = pty.fork() + if pid == 0: + # nothing may leave this block except _exit (see + # completion_posix_test.py for the cascade a leak causes) + try: + os.environ.clear() + os.environ.update(env) + os.chdir(cwd) + os.execv(SHELL, [SHELL, "--norc"]) + except BaseException: + pass + os._exit(127) + fcntl.ioctl(fd, termios.TIOCSWINSZ, struct.pack("HHHH", 60, 200, 0, 0)) + return pid, fd + + +def session(cwd, steps, settle=0.6): + """Run `steps` (bytes to type, or a callable run between keystrokes) + in a fresh shell; return the output of each bytes step, in order.""" + home = tempfile.mkdtemp(prefix="hellish_mk_home_") + TMP.append(home) + pid, fd = spawn(cwd, home) + chunks = [] + drain_idle(fd, chunks, quiet=0.4, cap=6.0) + outs = [] + for step in steps: + if callable(step): + step() + continue + mark = len(chunks) + try: + os.write(fd, step) + except OSError: + break + drain_idle(fd, chunks, quiet=settle, cap=settle * 6 + 6) + outs.append(b"".join(chunks[mark:]).decode(errors="replace")) + if ASAN: + try: + os.write(fd, KILL + b"exit\n") + except OSError: + pass + drain_idle(fd, chunks, quiet=1.0, cap=20.0) + try: + os.waitpid(pid, os.WNOHANG) + except OSError: + pass + try: + os.kill(pid, 9) + os.waitpid(pid, 0) + except OSError: + pass + text = b"".join(chunks).decode(errors="replace") + if VERBOSE: + print(" << %r" % text) + SEEN_CRASH.extend(m for m in CRASH_MARKERS if m in text) + return outs + + +def listing(out): + """The names readline column-printed, as a set (see + completion_posix_test.py).""" + i = out.find("\a\r\n") + if i < 0: + return set() + block = out[i + 3:] + j = block.find("\r\n$ ") + if j >= 0: + block = block[:j] + return set(ANSI.sub("", block).split()) + + +def line_is(out, want): + return want in ANSI.sub("", out).replace("\a", "") + + +def fixture(files, setup=None): + d = tempfile.mkdtemp(prefix="hellish_mk_") + TMP.append(d) + for name, data in files.items(): + p = os.path.join(d, name) + os.makedirs(os.path.dirname(p), exist_ok=True) + mode = "wb" if isinstance(data, bytes) else "w" + with open(p, mode) as f: + f.write(data) + if setup: + setup(d) + return d + + +def lst(files, typed, setup=None): + """The listing after `typed` + TAB TAB, in a fresh directory.""" + d = fixture(files, setup) + return listing(session(d, [typed + b"\t\t\t"])[0]) + + +def comp(files, typed, setup=None): + """The text readline echoed after `typed` + TAB.""" + d = fixture(files, setup) + return session(d, [typed + b"\t"])[0] + + +def case21(): + d = fixture({mk: "a1:\na2:\n", "sub/Makefile": "s1:\ns2:\n"}) + path = os.path.join(d, mk) + + def rewrite_size(): + with open(path, "w") as f: + f.write("a1:\na2:\na3:\n") + + def rewrite_same_size(): + with open(path, "w") as f: + f.write("b1:\nb2:\nb3:\n") + st = os.stat(path) + os.utime(path, (st.st_atime + 100, st.st_mtime + 100)) + + outs = session(d, [b"make \t\t\t", KILL, rewrite_size, b"make \t\t\t", KILL, + rewrite_same_size, b"make \t\t\t", KILL, + b"cd sub\n", b"make \t\t\t"]) + got = [listing(o) for o in (outs[0], outs[2], outs[4], outs[7])] + want = [{"a1", "a2"}, {"a1", "a2", "a3"}, {"b1", "b2", "b3"}, + {"s1", "s2"}] + for label, g, w in zip(("first", "size change", "same size, newer mtime", + "after cd"), got, want): + check("21 cache: %s" % label, g == w, str(sorted(g))) + + +def cases(): + # 1 + got = lst({"fixture.txt": "x", "other.dat": "x"}, b"make ") + check("1 no Makefile: filenames are offered", + got == {"fixture.txt", "other.dat"}, str(sorted(got))) + + # 2 + base = "all clean fclean re test".split() + body = "".join("%s:\n\t@true\n" % t for t in base) + out = comp({mk: body}, b"make f") + check("2 make f -> make fclean ", line_is(out, "make fclean "), + repr(out[-120:])) + got = lst({mk: body, "zzfile": "x"}, b"make ") + check("2 make lists exactly the targets", got == set(base), + str(sorted(got))) + + if ASAN: + case21() + return + + # 3 + files = {"GNUmakefile": "g_a:\ng_b:\n", "makefile": "l_a:\nl_b:\n", + "Makefile": "u_a:\nu_b:\n"} + check("3 GNUmakefile beats makefile beats Makefile", + lst(files, b"make ") == {"g_a", "g_b"}) + del files["GNUmakefile"] + check("3 makefile beats Makefile", lst(files, b"make ") == {"l_a", "l_b"}) + del files["makefile"] + check("3 Makefile alone", lst(files, b"make ") == {"u_a", "u_b"}) + + # 4 + def link(d): + os.symlink("real.mk", os.path.join(d, "Makefile")) + got = lst({"real.mk": "sa:\nsb:\n"}, b"make ", link) + check("4 symlinked Makefile is followed", got == {"sa", "sb"}, + str(sorted(got))) + + def adir(d): + os.makedirs(os.path.join(d, "Makefile")) + got = lst({"fixture.txt": "x", "Makefile/in": "all:\nclean:\n"}, + b"make ", None) + check("4 a DIRECTORY named Makefile -> filenames", + got == {"fixture.txt", "Makefile/"} + or got == {"fixture.txt", "Makefile"}, str(sorted(got))) + if os.geteuid() == 0: + print("skip 4 unreadable Makefile (running as root)") + else: + def noperm(d): + os.chmod(os.path.join(d, mk), 0) + got = lst({mk: "ta:\ntb:\n", "fixture.txt": "x"}, b"make ", noperm) + check("4 unreadable Makefile -> filenames", + got == {mk, "fixture.txt"}, str(sorted(got))) + + # 5 + got = lst({mk: "", "fixture.txt": "x"}, b"make ") + check("5 empty Makefile -> filenames", got == {mk, "fixture.txt"}, + str(sorted(got))) + out = comp({mk: "first:\nlast:"}, b"make la") + check("5 no trailing newline: last target found", + line_is(out, "make last "), repr(out[-100:])) + + # 6 + got = lst({mk: "a b c: x\ndc:: y\n"}, b"make ") + check("6 a b c: and dc:: are all rules", got == {"a", "b", "c", "dc"}, + str(sorted(got))) + + # 7 + body = ("V := 1\nW ::= 2\nX = a:b\nY ?= c:d\nZ += e:f\n" + "real: x\nreal2:\n") + got = lst({mk: body}, b"make ") + check("7 assignments are not rules", got == {"real", "real2"}, + str(sorted(got))) + + # 8 + body = "$(NAME): x\n%.o: %.c\n.PHONY: all ok\nok:\nok2:\n" + got = lst({mk: body}, b"make ") + check("8 $(var) targets, patterns, .PHONY and its list are not offered", + got == {"ok", "ok2"}, str(sorted(got))) + + # 9 + got = lst({mk: "ok1:\n\techo a:b\nok2:\n"}, b"make ") + check("9 recipe lines are not rules", got == {"ok1", "ok2"}, + str(sorted(got))) + + # 10 + got = lst({mk: "define T\nfoo: bar\nendef\nok1:\nok2:\n"}, b"make ") + check("10 define ... endef is skipped", got == {"ok1", "ok2"}, + str(sorted(got))) + + # 11 + got = lst({mk: "lo\\\nng: x\nother:\n"}, b"make ") + check("11 backslash-newline is one space, as in GNU make: lo ng", + got == {"lo", "ng", "other"}, str(sorted(got))) + got = lst({mk: "all: a \\\n\tb\nother:\n"}, b"make ") + check("11 a continued prerequisite line is not a recipe or a rule", + got == {"all", "other"}, str(sorted(got))) + + # 12 + d = {mk: b"all:\r\nclean: x\r\n\techo hi\r\n"} + got = lst(d, b"make ") + out = comp(d, b"make cl") + check("12 CRLF Makefile: names without CR", + got == {"all", "clean"} and line_is(out, "make clean "), + "%s %r" % (sorted(got), out[-100:])) + + # 13 + got = lst({mk: "# hidden: x\nhelp: ## show help\nok:\n"}, b"make ") + check("13 comments hidden, ## help rule offered", + got == {"help", "ok"}, str(sorted(got))) + + # 14 + files = {mk: "include Makefile\ninclude sub.mk\n-include missing.mk\n" + "include a.mk\nmain1:\nmain2:\n", + "sub.mk": "subtgt:\n", "a.mk": "include b.mk\natgt:\n", + "b.mk": "include a.mk\nbtgt:\n"} + got = lst(files, b"make ") + check("14 includes: one level only, missing skipped, cycles harmless", + got == {"main1", "main2", "subtgt", "atgt"}, str(sorted(got))) + + # 15 + body = "".join("t%d:\n" % i for i in range(5000)) + d = fixture({mk: body}) + home = tempfile.mkdtemp(prefix="hellish_mk_home_") + TMP.append(home) + pid, fd = spawn(d, home) + chunks = [] + drain_idle(fd, chunks, quiet=0.4, cap=6.0) + os.write(fd, b"make t4999\t") + t0 = time.time() + seen = "" + while time.time() - t0 < 10 and "t4999 " not in seen: + r, _, _ = select.select([fd], [], [], 0.05) + if r: + try: + seen += os.read(fd, 65536).decode(errors="replace") + except OSError: + break + dt = time.time() - t0 + os.kill(pid, 9) + os.waitpid(pid, 0) + check("15 5000 rules: make t4999 completes within 2 s", + "t4999 " in ANSI.sub("", seen) and dt < 2.0, "%.2fs" % dt) + + # 16 + body = ("dup:\ndup: y\nx.y:\nsub/t:\na-b:\nc+d:\n") + got = lst({mk: body}, b"make ") + check("16 duplicates once; . / - + names offered", + got == {"dup", "x.y", "sub/t", "a-b", "c+d"}, str(sorted(got))) + + # 17 + files = {mk: "zz_target:\nzz_other:\n", "fixture.txt": "x"} + for typed in (b"make -C sub ", b"make -f other.mk ", + b"make --directory=sub ", b"make --file x "): + got = lst(files, typed) + check("17 %s -> filenames" % typed.decode(), + got == {mk, "fixture.txt"}, str(sorted(got))) + + # 18 + files = {mk: "all:\nclean:\n"} + out = comp(files, b"make V=1 cl") + check("18 make V=1 cl -> clean", line_is(out, "make V=1 clean "), + repr(out[-100:])) + got = lst(files, b"make -") + check("18 make - offers no targets", got == set(), str(sorted(got))) + + # 19 + for pre in (b"sudo make cl", b"env X=1 make cl", b"true && make cl", + b"gmake cl", b"(make cl", b"/usr/bin/make cl"): + out = comp(files, pre) + want = pre.decode() + "ean " + check("19 %s" % pre.decode(), line_is(out, want), + repr(out[-100:])) + + # 20 + d = fixture({mk: "all:\nclean:\n"}) + # `complete` specs are only consulted under `shopt -s progcomp` + outs = session(d, [b"shopt -s progcomp\n", + b"complete -W 'zz yy' make\n", b"make \t\t\t"]) + check("20 user spec wins", listing(outs[2]) == {"zz", "yy"}, + str(sorted(listing(outs[2])))) + d = fixture({mk: "all:\nclean:\n"}) + outs = session(d, [b"shopt -s progcomp\n", + b"complete -W 'zz yy' make\n", b"complete -r make\n", + b"make \t\t\t"]) + check("20 complete -r make: targets again", + listing(outs[3]) == {"all", "clean"}, str(sorted(listing(outs[3])))) + + case21() + + # 22 + out = comp({mk: "all:\nclean:\n", "out.txt": "x"}, b"make out") + check("22 no target matches -> make out.txt", + line_is(out, "make out.txt "), repr(out[-100:])) + + # 23 + d = fixture({mk: "all:\nclean:\n"}) + outs = session(d, [b"make cl", b"\x1b[D", b"\t", KILL, + b"echo ALIVE\n"]) + # readline completes the word that CONTAINS the cursor, using the text + # to its left ("c"), so the line may grow; what is pinned is that the + # shell survives and keeps reading. + check("23 mid-word cursor TAB: shell alive, no crash marker", + "ALIVE" in outs[-1] and not SEEN_CRASH, repr(outs[-1][-100:])) + + # 24 + cmd = "complete -p make; echo $?" + if not os.access(BASH, os.X_OK): + print("skip 24 (no pinned bash at %s)" % BASH) + else: + a = subprocess.run([SHELL, "--norc", "-c", cmd], + capture_output=True, text=True, + env={"PATH": "/usr/bin:/bin", "HOME": "/"}) + b = subprocess.run([BASH, "--norc", "-c", cmd], + capture_output=True, text=True, + env={"PATH": "/usr/bin:/bin", "HOME": "/"}) + check("24 complete -p make: same stdout and status as bash", + a.stdout == b.stdout and a.returncode == b.returncode, + "hellish %r/%d bash %r/%d" % (a.stdout, a.returncode, + b.stdout, b.returncode)) + + +def main(): + if not os.access(SHELL, os.X_OK): + sys.exit("complete_make_test: not executable: %s" % SHELL) + cells = (("in-process", {}), ("forked", {"HELLISH_RL_FORK": "1"})) + try: + for cell, env in cells: + if INPROC_ONLY and env: + continue + print("== %s reader" % cell) + EXTRA.clear() + EXTRA.update(env) + cases() + finally: + for d in TMP: + shutil.rmtree(d, ignore_errors=True) + check("no run corrupted the heap", not SEEN_CRASH, + "saw %s" % sorted(set(SEEN_CRASH))) + print("\n%d checks failed" % len(FAILS)) + sys.exit(1 if FAILS else 0) + + +main() diff --git a/tests/curl_install_output_test.py b/tests/curl_install_output_test.py new file mode 100644 index 00000000..92d775b6 --- /dev/null +++ b/tests/curl_install_output_test.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python3 +"""Regression for issue #164: a curl-fed installer must not lose its PATH hint. + +The real installer prints a long message through printf '%s'. Put the +important text across the old 4095-byte output boundary: before the fix the +line ended at "Manually", hiding the rest of the instruction. +""" +import os +import subprocess +import sys + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 else + os.path.join(ROOT, "build", "bin", "hellish")) +FIXTURE = os.path.join(os.path.dirname(os.path.abspath(__file__)), + ".mini_install.sh") +ENV = dict(os.environ, HELLISH_NO_BANNER="1", HELLISH_NO_UPDATE_CHECK="1", + HELLISH_NO_ANIM="1") + + +def main(): + with open(FIXTURE, "rb") as source: + script = source.read() + proc = subprocess.run([SHELL], input=script, capture_output=True, + env=ENV, timeout=15) + expected = b"Manually add this path to PATH" + ok = proc.returncode == 0 and expected in proc.stdout + print(("ok " if ok else "FAIL ") + + "curl-fed installer prints the complete PATH instruction") + if not ok: + print(" exit=%s, stdout tail=%r, stderr=%r" % + (proc.returncode, proc.stdout[-100:], proc.stderr[-300:])) + return 0 if ok else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/exec_error_tty_test.py b/tests/exec_error_tty_test.py new file mode 100644 index 00000000..d5fd9f7f --- /dev/null +++ b/tests/exec_error_tty_test.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +"""A path that cannot run, typed at an interactive prompt (issue #133). + + ╰─❯ ./life + hellish: ./life: No such file or directory + + +The message was wrong -- the file was there -- and the prompt did not come +back. tests/scripts/53_exec_diagnostics.sh grades the words and statuses +against bash in a script; this runs the same failures where the report +happened, in a job-control shell on a terminal: the child takes the +terminal, fails, and the shell must take it back and prompt at once. Each +case must print bash's diagnostic, then a prompt, then answer `echo $?` +with bash's status. Waits on output: a hang is a timeout, a named failure. + +The same failures through `exec` come after them. At a prompt a failed +exec must leave the shell up, as bash's does -- a script or subshell +exits instead (tests/scripts/55_exec_builtin.sh). `exec nosuch` used to +print "command not found", then read the shell it had just freed: under +ASan the shell died on the spot. exec names the file by its full path, as +bash's does (@D@ is the directory the cases run in). + +Usage: python3 exec_error_tty_test.py [/path/to/hellish] +""" +import os +import shutil +import stat +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] + +# what is typed, what the shell says about it, the status +CASES = [ + ("./notexec", b"./notexec: Permission denied", 126), + ("./dir", b"./dir: Is a directory", 126), + ("./nope", b"./nope: No such file or directory", 127), + ("./nointerp", + b"./nointerp: /nonexistent/interp: bad interpreter: No such file", 126), + ("./elfjunk", b"./elfjunk: cannot execute binary file: Exec format error", + 126), + ("./notexec/x", b"./notexec/x: Not a directory", 126), + ("exec nosuch_cmd_q", b"exec: nosuch_cmd_q: not found", 127), + ("exec -- nosuch_cmd_q", b"exec: nosuch_cmd_q: not found", 127), + ("exec ./notexec", b"@D@/notexec: Permission denied", 126), + ("exec ./dir", b"@D@/dir: Is a directory", 126), + ("exec ./nope", b"@D@/nope: No such file or directory", 127), +] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def fixtures(): + d = tempfile.mkdtemp(prefix="exec-error-tty-") + os.mkdir(os.path.join(d, "dir")) + for name, data, mode in ( + ("notexec", b"echo not-run\n", 0o644), + ("nointerp", b"#!/nonexistent/interp\necho ran\n", 0o755), + ("elfjunk", b"\x7fELFnot-a-real-header\n", 0o755)): + p = os.path.join(d, name) + with open(p, "wb") as f: + f.write(data) + os.chmod(p, mode | stat.S_IRUSR) + return d + + +def main(): + d = fixtures() + t = Tty(SHELL) + check("start/prompt", t.expect(b"P$ "), repr(t.out[-300:])) + t.send("cd %s\r" % d) + t.expect(b"P$ ") + for cmd, said, status in CASES: + said = said.replace(b"@D@", d.encode()) + mark = len(t.out) + t.send(cmd + "\r") + ok = t.expect(said, timeout=5) and t.expect(b"P$ ", timeout=5) + check(cmd + "/says-why-and-prompts", ok, repr(t.since(mark)[-300:])) + t.send("echo \"st=$?\"\r") + ok = t.expect(b"st=%d\r\n" % status, timeout=5) + check(cmd + "/status-%d" % status, ok and t.expect(b"P$ "), + repr(t.since(mark)[-300:])) + st = t.close() + check("exit/not-signalled", os.WIFEXITED(st), "wait status %r" % st) + shutil.rmtree(d, ignore_errors=True) + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/fc_listing_tty_test.py b/tests/fc_listing_tty_test.py new file mode 100644 index 00000000..4b93c71e --- /dev/null +++ b/tests/fc_listing_tty_test.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 +"""fc's listing, as an interactive bash prints it (issue #137). + +fc matched its options as whole words -- `-l` and `-lr` only -- so every +other spelling of a listing fell into EDIT mode and started $EDITOR on the +history: `fc -ln -1` (which is how oh-my-zsh's sudo widget asks for the +previous command), `-nl`, `-lnr`, `-l -n`. It also counted the running fc +line itself as the most recent entry, so `-1` named fc, and it printed +`Ncmd` where bash prints `N cmd`. + +Every expectation below was produced by an interactive bash 5.2 on a pty, +from the same three input lines. One case is POSIX rather than bash: +`fc -l 3 1` lists 3 2 1 (first after last lists backwards). bash printed +only entry 1 in that exact position, and 3 2 1 one entry further on -- +a quirk this does not copy. + +History is interactive-only, so this needs a terminal; it waits on output, +not on the clock. + +Usage: python3 fc_listing_tty_test.py [/path/to/hellish] +""" +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] + +H1 = "1\t HISTFILE=/dev/null\n" +H2 = "2\t echo one\n" +H3 = "3\t echo two\n" +N1, N2, N3 = "\t HISTFILE=/dev/null\n", "\t echo one\n", "\t echo two\n" + +# fc invocation, what it prints, its status. +CASES = [ + ("fc -ln -1", N3, 0), + ("fc -nl -1", N3, 0), + ("fc -l -1", H3, 0), + ("fc -lr -3 -1", H3 + H2 + H1, 0), + ("fc -lnr -2", N3 + N2, 0), + ("fc -l -n -1", N3, 0), + ("fc -ln", N1 + N2 + N3, 0), + ("fc -l 1 2", H1 + H2, 0), + ("fc -l", H1 + H2 + H3, 0), + ("fc -ln 2", N2 + N3, 0), + ("fc -rn -l -2", N3 + N2, 0), + ("fc -l ech", H3, 0), + ("fc -l 'echo o'", H2 + H3, 0), + ("fc -l 3 1", H3 + H2 + H1, 0), + ("fc -lr 1 3", H3 + H2 + H1, 0), + ("fc -l -- -2", H2 + H3, 0), + ("fc -l zzz", "", 1), + ("fc -lx", "", 2), +] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def one(cmd, want, status): + """A fresh shell per case, so entry numbers are fixed: 1 HISTFILE, + 2 echo one, 3 echo two, and the fc line itself is 4.""" + t = Tty(SHELL, env={"FCEDIT": "/bin/false", "EDITOR": "/bin/false"}) + t.expect(b"P$ ") + for c in ("HISTFILE=/dev/null", "echo one", "echo two"): + t.send(c + "\r") + t.expect(b"P$ ") + t.send("echo BEGIN; %s 2>/dev/null; echo \"END rc=$?\"\r" % cmd) + t.expect(b"\nBEGIN\r\n") + mark = t.pos + ok = t.expect(b"END rc=") + got = t.out[mark:t.pos - len(b"END rc=")].replace(b"\r", b"").decode() + start = t.pos + ok = ok and t.expect(b"\r\n") + rc = t.out[start:t.pos - 2].decode() if ok else "?" + t.close() + check(cmd, ok and got == want and rc == str(status), + "got %r rc=%s, want %r rc=%d" % (got, rc, want, status)) + + +def main(): + for cmd, want, status in CASES: + one(cmd, want, status) + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/frontend_budget_test.py b/tests/frontend_budget_test.py index f21c1f58..39918ef5 100755 --- a/tests/frontend_budget_test.py +++ b/tests/frontend_budget_test.py @@ -55,6 +55,16 @@ "forked": {"enter": 85, "enter_write": 13, "enter_ioctl": 16, "key": 9, "key_write": 3}, } +# The inline suggestion (#140): the same per-key budget with a ghost on +# screen at every key, with and without a right prompt; and with nothing +# to paint and no right prompt a key is readline's one write and no more +# (the bound has room for the ^U that ends each typing phase). +BUDGET["in-process/as+rp"] = dict(BUDGET["in-process"], nomatch_write=2.2) +BUDGET["in-process/as"] = dict(BUDGET["in-process"], nomatch_write=1.2) +BUDGET["forked/as+rp"] = dict(BUDGET["forked"], nomatch_write=2.2) +BUDGET["forked/as"] = dict(BUDGET["forked"], nomatch_write=1.2) +SEED = "x" * 60 +AS_ON = {"HELLISH_AUTOSUGGEST": "1"} FORKING = ("clone", "clone3", "fork", "vfork", "pipe2", "pipe", "wait4", "execve") @@ -67,7 +77,7 @@ def check(name, ok, detail=""): class Traced: - def __init__(self, home, trace, extra): + def __init__(self, home, trace, extra, seed=()): env = { "HOME": home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), "TERM": "xterm-256color", "LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", @@ -89,6 +99,11 @@ def __init__(self, home, trace, extra): os._exit(127) self.until(lambda: MARK in self.buf, 20.0) self.settle() + for line in seed: + n = self.buf.count(MARK) + os.write(self.fd, line.encode() + b"\r") + self.until(lambda: self.buf.count(MARK) > n, 10.0) + self.settle() def pump(self, secs): r, _, _ = select.select([self.fd], [], [], secs) @@ -125,9 +140,9 @@ def enters(self): self.until(lambda: self.buf.count(MARK) >= n + ENTERS, 30.0) self.settle() - def typing(self): + def typing(self, ch=b"x"): for _ in range(KEYS): - os.write(self.fd, b"x") + os.write(self.fd, ch) self.pump(0.03) self.settle() os.write(self.fd, b"\x15") @@ -169,13 +184,14 @@ def counts(lines, t0, t1): return c -def cell(label, extra, base): - trace = os.path.join(base, "trace-" + label) +def cell(label, extra, base, seed=()): + trace = os.path.join(base, "trace-" + label.replace("/", "_")) home = tempfile.mkdtemp(dir=base) - s = Traced(home, trace, extra) + s = Traced(home, trace, extra, seed) s.phase("enter", s.enters) s.phase("idle", s.idle) s.phase("key", s.typing) + s.phase("nomatch", lambda: s.typing(b"q")) s.phase("winch", s.winch_same) s.phase("rows", s.rows_only) s.close() @@ -193,7 +209,7 @@ def cell(label, extra, base): check("%s: a bare Enter does at most %d %ss" % (label, b[key], name), n <= b[key], "%.1f per Enter" % n) forks = sum(by["enter"][k] for k in FORKING) - if label == "in-process": + if label.startswith("in-process"): check("%s: a bare Enter creates no process" % label, forks == 0, "%s" % {k: by["enter"][k] for k in FORKING if by["enter"][k]}) else: @@ -209,9 +225,15 @@ def cell(label, extra, base): per <= b["key"], "%.1f per key: %s" % ( per, dict(by["key"].most_common(6)))) n = by["key"]["write"] / float(KEYS) + print("# %s: per key %.1f syscalls, %.1f writes" % (label, per, n)) check("%s: a keystroke does at most %d writes" % (label, b["key_write"]), n <= b["key_write"], "%.1f per key" % n) + if "nomatch_write" in b: + n = by["nomatch"]["write"] / float(KEYS) + check("%s: a key with nothing to paint does at most %.1f writes" + % (label, b["nomatch_write"]), n <= b["nomatch_write"], + "%.1f per key" % n) check("%s: an idle prompt costs nothing" % label, sum(by["idle"].values()) == 0, "%d syscalls while idle: %s" % (sum(by["idle"].values()), @@ -230,6 +252,12 @@ def main(): try: cell("in-process", {}, base) cell("forked", {"HELLISH_RL_FORK": "1"}, base) + seed = (SEED,) + cell("in-process/as+rp", AS_ON, base, seed) + cell("in-process/as", dict(AS_ON, RPROMPT=""), base, seed) + cell("forked/as+rp", dict(AS_ON, HELLISH_RL_FORK="1"), base, seed) + cell("forked/as", dict(AS_ON, RPROMPT="", HELLISH_RL_FORK="1"), + base, seed) finally: shutil.rmtree(base, ignore_errors=True) if FAILS: diff --git a/tests/function_heredoc_tty_test.py b/tests/function_heredoc_tty_test.py new file mode 100644 index 00000000..02230bd1 --- /dev/null +++ b/tests/function_heredoc_tty_test.py @@ -0,0 +1,135 @@ +#!/usr/bin/env python3 +"""A heredoc in a one-line function body, typed at a prompt, survives. + + P$ f() { cat < body + heredoc> EOF + P$ f + Segmentation fault (3.1.3: the shell itself died) + +A function body's heredoc is re-read at every call, so its body must live +on the AST node. A one-line definition is complete before its body is +read, nothing pre-extracted it, and the body was written into the +DEFINING command's redirect slot -- gone by the first call. The script +half of this is pinned by tests/scripts/52_heredoc_logical_line.sh; this +is the half only a terminal shows: the body arrives through readline's +`heredoc>` prompt, a different reader than a script's buffer. + +Waits on output, never on the clock: each step returns the moment the +expected text arrives, so the file runs in about a second. + +Usage: python3 function_heredoc_tty_test.py [/path/to/hellish] +""" +import os +import pty +import select +import sys +import tempfile +import time + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +class Tty: + def __init__(self): + self.home = tempfile.mkdtemp(prefix="fnheredoc-") + with open(os.path.join(self.home, ".hellishrc"), "w") as f: + f.write("PS1='P$ '\n") + env = {"HOME": self.home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "dumb", "LANG": "C.UTF-8", "HELLISH_NO_BANNER": "1", + "HELLISH_NO_ANIM": "1", "HELLISH_NO_UPDATE_CHECK": "1", + "ASAN_OPTIONS": "detect_leaks=0"} + self.pid, self.fd = pty.fork() + if self.pid == 0: + os.chdir(self.home) + os.environ.clear() + os.environ.update(env) + os.execv(SHELL, [SHELL, "-i"]) + os._exit(127) + self.out = b"" + self.pos = 0 + + def expect(self, needle, timeout=10.0): + """Read until `needle` appears past what earlier expects consumed, + then consume through it.""" + end = time.time() + timeout + while self.out.find(needle, self.pos) < 0 and time.time() < end: + r, _, _ = select.select([self.fd], [], [], 0.05) + if not r: + continue + try: + d = os.read(self.fd, 65536) + except OSError: + break + if not d: + break + self.out += d + at = self.out.find(needle, self.pos) + if at < 0: + return False + self.pos = at + len(needle) + return True + + def send(self, s): + os.write(self.fd, s.encode()) + + def close(self): + try: + os.write(self.fd, b"exit\r") + except OSError: + pass + end = time.time() + 5 + while time.time() < end: + pid, st = os.waitpid(self.pid, os.WNOHANG) + if pid: + return st + self.expect(b"\x00never", 0.1) + os.kill(self.pid, 9) + return os.waitpid(self.pid, 0)[1] + + +def main(): + t = Tty() + ok = t.expect(b"P$ ") + check("prompt/appears", ok) + t.send("f() { cat < ")) + t.send("one $((6*7))\r") + t.expect(b"heredoc> ") + t.send("EOF\r") + t.expect(b"P$ ") + t.send("f; f; echo alive=$?\r") + got = t.expect(b"alive=0") + body = t.out.decode("utf-8", "replace") + check("call/shell-survives", got, repr(body[-300:])) + check("call/body-each-time", body.count("one 42") >= 2, repr(body[-300:])) + t.send("g() { cat <<'EOF'; }\r") + t.expect(b"heredoc> ") + t.send("lit $((1+1))\r") + t.expect(b"heredoc> ") + t.send("EOF\r") + t.expect(b"P$ ") + mark = len(t.out) + t.send("g; echo alive2=$?\r") + ok = t.expect(b"alive2=0") + after = t.out[mark:].replace(b"\r", b"") + check("quoted-delimiter/body-stays-literal", + ok and after.count(b"lit $((1+1))") >= 1 and b"lit 2" not in after, + repr(after[-200:])) + st = t.close() + check("exit/not-signalled", not os.WIFSIGNALED(st), + "signal %d" % os.WTERMSIG(st) if os.WIFSIGNALED(st) else "") + print("%d failed" % len(FAILS)) + sys.exit(1 if FAILS else 0) + + +main() diff --git a/tests/hard/17_bg_signal_report.sh b/tests/hard/17_bg_signal_report.sh index 24417e11..4f0ce449 100755 --- a/tests/hard/17_bg_signal_report.sh +++ b/tests/hard/17_bg_signal_report.sh @@ -10,7 +10,9 @@ # # The shell's own stderr is captured to a file and replayed on stdout with the # volatile parts normalised away: -# - the pid, which changes every run; +# - the pid's value, which changes every run -- but not its width: bash +# right-aligns it in five columns (%5ld), so a pid under 10000 comes +# padded, and a narrower field shows up as NARROW-PID; # - the "line N", because hellish's script line numbers are a KNOWN separate # defect (update_ctx reports the end of the input batch rather than the # running command) -- recorded in backlog.md. The message text, its stream, @@ -27,7 +29,8 @@ work=$(mktemp -d) trap 'rm -rf "$work"' EXIT norm() { - sed -e 's/^.*: line [0-9][0-9]*: [0-9][0-9]* */PREFIX /' "$1" + sed -e 's/^.*: line [0-9][0-9]*: [ 0-9]\{5,\} /PREFIX /' \ + -e 's/^.*: line [0-9][0-9]*: [ 0-9]* /NARROW-PID /' "$1" } echo "== SIGKILL: announced, then retired ==" diff --git a/tests/helpers/ptyexpect.py b/tests/helpers/ptyexpect.py new file mode 100644 index 00000000..b4aeeb50 --- /dev/null +++ b/tests/helpers/ptyexpect.py @@ -0,0 +1,127 @@ +"""An interactive hellish on a pty, driven by what it PRINTS, not by a clock. + +Most pty tests here sleep a fixed time after every keystroke, long enough +for the slowest CI runner -- so every step costs that much on every runner, +and a test with thirty steps waits out thirty worst cases. This waits for +the text it needs and moves on the moment it arrives; the timeout is only +the failure bound. A shell that answers in 3 ms makes a step cost 3 ms. + + from ptyexpect import Tty # tests add tests/helpers to sys.path + t = Tty(shell) # PS1 is 'P$ ', HOME is a temp dir + t.expect(b"P$ ") # consume through the first prompt + t.send("echo hi\r") + assert t.expect(b"hi\r\n") + status = t.close() # a wait status: WIFSIGNALED works + +Lives in tests/helpers/ so tests/pty_suite.sh, which runs every tests/*.py, +does not mistake it for a test. +""" +import os +import pty +import select +import shutil +import tempfile +import time + + +class Tty: + """One interactive shell on a pty, in a throwaway HOME.""" + + def __init__(self, shell, rc="PS1='P$ '\n", env=None, path=None, + cols=100, rows=30, args=("-i",)): + self.home = tempfile.mkdtemp(prefix="ptyexpect-") + with open(os.path.join(self.home, ".hellishrc"), "w") as f: + f.write(rc) + base = {"HOME": self.home, + "PATH": path or os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "dumb", "LANG": "C.UTF-8", "USER": "tester", + "HELLISH_NO_BANNER": "1", "HELLISH_NO_ANIM": "1", + "HELLISH_NO_UPDATE_CHECK": "1", + "ASAN_OPTIONS": "detect_leaks=0"} + base.update(env or {}) + self.pid, self.fd = pty.fork() + if self.pid == 0: + os.chdir(self.home) + os.environ.clear() + os.environ.update(base) + os.execv(shell, [shell] + list(args)) + os._exit(127) + try: + import fcntl + import struct + import termios + fcntl.ioctl(self.fd, termios.TIOCSWINSZ, + struct.pack("HHHH", rows, cols, 0, 0)) + except OSError: + pass + self.out = b"" + self.pos = 0 + + def _read(self, timeout): + r, _, _ = select.select([self.fd], [], [], timeout) + if not r: + return True + try: + d = os.read(self.fd, 65536) + except OSError: + return False + if not d: + return False + self.out += d + return True + + def expect(self, needle, timeout=10.0): + """Read until `needle` appears after what earlier expects consumed, + then consume through it. False on timeout or when the shell is + gone -- the caller turns that into a named failure.""" + if isinstance(needle, str): + needle = needle.encode() + end = time.time() + timeout + while self.out.find(needle, self.pos) < 0 and time.time() < end: + if not self._read(0.05): + break + at = self.out.find(needle, self.pos) + if at < 0: + return False + self.pos = at + len(needle) + return True + + def since(self, mark): + """Everything printed after byte offset `mark`, CRs dropped.""" + return self.out[mark:].replace(b"\r", b"") + + def send(self, s): + os.write(self.fd, s.encode() if isinstance(s, str) else s) + + def _reap(self, timeout): + end = time.time() + timeout + while time.time() < end: + pid, st = os.waitpid(self.pid, os.WNOHANG) + if pid: + return st + self._read(0.05) + return None + + def close(self, timeout=5.0): + """Ask the shell to leave, and return its wait status. `exit` on + its own first: an interrupt sent right before it makes readline + drop the type-ahead, and the shell would sit at its prompt. Only a + shell still busy gets the interrupt, then `exit` again.""" + status = None + for keys in (b"exit\r", b"\x03", b"exit\r"): + try: + os.write(self.fd, keys) + except OSError: + pass + status = self._reap(timeout / 3) + if status is not None: + break + if status is None: + os.kill(self.pid, 9) + status = os.waitpid(self.pid, 0)[1] + try: + os.close(self.fd) + except OSError: + pass + shutil.rmtree(self.home, ignore_errors=True) + return status diff --git a/tests/history_fd_test.py b/tests/history_fd_test.py new file mode 100644 index 00000000..a13b1614 --- /dev/null +++ b/tests/history_fd_test.py @@ -0,0 +1,175 @@ +#!/usr/bin/env python3 +"""The history file's descriptor stays the shell's, and only the shell's. + +An interactive shell streams each command into its history file through a +descriptor it keeps open. That descriptor was opened the plain way: + + - lowest free number, so fd 3, without close-on-exec: every program the + shell ran inherited the history file open for writing (`ls -l + /proc/self/fd` listed ~/.minishell_history); + - in the 3..9 range scripts use themselves: after `exec 3>f` the next + commands' history went into f -- the user's file -- and after + `exec 3>&-` it went nowhere for the rest of the session. + +bash never lends its history file to a child, and a user's redirection +never receives shell history. The fix opens it close-on-exec at fd >= 10 +and remembers it by identity: a number the user has redirected or closed +is theirs from then on -- not written to, not closed -- and the history +file is opened again. + +Each session waits on output markers, not on sleeps. Linux only (/proc). + +Usage: python3 history_fd_test.py /path/to/hellish +""" +import os +import pty +import re +import select +import shutil +import sys +import tempfile +import time + +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else "../build/bin/hellish") +FAILS = [] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + + (" " + detail if not ok else "")) + if not ok: + FAILS.append(name) + + +def read_file(path): + try: + with open(path, errors="replace") as f: + return f.read() + except OSError: + return None + + +def session(home, rc, cmds): + """Run cmds in an interactive shell; return its output, ANSI removed. + Each command is followed by a marker echo that is waited for, so the + next line is sent only once the previous one has finished.""" + with open(os.path.join(home, ".hellishrc"), "w") as f: + f.write(rc) + env = { + "HOME": home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "dumb", "LANG": "C.UTF-8", "PS1": "> ", + "HELLISH_NO_BANNER": "1", "HELLISH_NO_UPDATE_CHECK": "1", + "HELLISH_NO_ANIM": "1", "ASAN_OPTIONS": "detect_leaks=0", + } + pid, fd = pty.fork() + if pid == 0: + os.chdir(home) + os.environ.clear() + os.environ.update(env) + os.execv(SHELL, [SHELL]) + os._exit(127) + buf = bytearray() + + def wait_for(token, cap=15.0): + end = time.monotonic() + cap + while token not in buf and time.monotonic() < end: + r, _, _ = select.select([fd], [], [], 0.05) + if not r: + continue + try: + d = os.read(fd, 65536) + except OSError: + return False + if not d: + return False + buf.extend(d) + return token in buf + + ok = True + for i, c in enumerate(cmds): + os.write(fd, ('%s; echo "@@"MARK%d\n' % (c, i)).encode()) + ok = wait_for(b"@@MARK%d" % i) and ok + os.write(fd, b"exit\n") + end = time.monotonic() + 5 + while time.monotonic() < end: + done, _ = os.waitpid(pid, os.WNOHANG) + if done: + break + select.select([fd], [], [], 0.05) + try: + os.read(fd, 65536) + except OSError: + pass + else: + os.kill(pid, 9) + os.waitpid(pid, 0) + os.close(fd) + out = re.sub(rb"\x1b\[[0-9;?]*[a-zA-Z]|[\x01\x02]", b"", bytes(buf)) + return ok, out.decode("utf-8", "replace") + + +def main(): + if not os.path.isdir("/proc/self/fd"): + print("skip: no /proc/self/fd on this system") + return 0 + + home = tempfile.mkdtemp(prefix="hellish_histfd_") + hist = os.path.join(home, ".minishell_history") + u3 = os.path.join(home, "user3.txt") + u10 = os.path.join(home, "user10.txt") + try: + ok, out = session(home, "", [ + "ls -l /proc/self/fd", + "exec 3>%s" % u3, + "echo HIST_A", + "exec 10>%s" % u10, + "echo HIST_B", + "echo user-data >&10", + "exec 10>&- 3>&-", + "echo HIST_C", + ]) + check("session ran to the end", ok, repr(out[-300:])) + check("a child does not inherit the history file", + ".minishell_history" not in out, + repr([l for l in out.splitlines() if "history" in l])) + check("exec 3>f: no history lands in the user's fd 3", + read_file(u3) == "", repr(read_file(u3))) + check("exec 10>f: the user's fd 10 holds only what they wrote", + read_file(u10) == "user-data\n", repr(read_file(u10))) + h = read_file(hist) or "" + for want in ("echo HIST_A", "echo HIST_B", "echo HIST_C", + "exec 10>&- 3>&-"): + check("history file records %r" % want, want in h, repr(h)) + finally: + shutil.rmtree(home, ignore_errors=True) + + # The rc takes over the descriptor, then moves the session to another + # HISTFILE: the re-home must not close what is now the user's fd. + home = tempfile.mkdtemp(prefix="hellish_histfd_") + alt = os.path.join(home, "alt_hist") + rc10 = os.path.join(home, "rc10.txt") + try: + ok, out = session(home, "exec 10>%s\nHISTFILE=%s\n" % (rc10, alt), [ + "echo rc-data >&10", + "ls -l /proc/self/fd", + "echo HIST_D", + ]) + check("rehome session ran to the end", ok, repr(out[-300:])) + check("rehome: the rc's fd 10 survives it", + read_file(rc10) == "rc-data\n", repr(read_file(rc10))) + check("rehome: a child inherits neither history file", + "alt_hist" not in out and ".minishell_history" not in out, + repr([l for l in out.splitlines() if "hist" in l])) + a = read_file(alt) or "" + check("rehome: HISTFILE records the session", "echo HIST_D" in a, + repr(a)) + finally: + shutil.rmtree(home, ignore_errors=True) + + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/history_prefix_search_test.py b/tests/history_prefix_search_test.py new file mode 100644 index 00000000..f0681bdb --- /dev/null +++ b/tests/history_prefix_search_test.py @@ -0,0 +1,283 @@ +#!/usr/bin/env python3 +"""Up searches the history for lines starting with what was typed (#136). + + ❯ git -> git push (not whatever ran last) + ❯ git -> git status + +zsh configs ask for it with `bindkey '^[[A' history-beginning-search- +backward` (or oh-my-zsh's up-line-or-beginning-search), bash ones with +`bind '"\\e[A": history-search-backward'`. Neither worked: a built-in +widget name bound the key to the shell-widget dispatcher, which found no +widget by that name and did nothing -- Up went dead -- and there was no +`bind` builtin at all. Both now reach readline's own history search, +bound directly, so repeated presses keep walking back (readline tracks the +search by checking the previous command was itself). + +Each line's result is checked by RUNNING it: the four history entries +print four different words, and Enter shows which one the keys recalled. + +History is interactive-only, so this needs a terminal; it waits on output, +not on the clock. + +Usage: python3 history_prefix_search_test.py [/path/to/hellish] +""" +import os +import subprocess +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] +UP, DOWN = "\x1b[A", "\x1b[B" +HISTORY = ["echo alpha", "printf 'beta\\n'", "echo gamma", "printf 'delta\\n'"] + +BINDKEY = ("bindkey '^[[A' history-beginning-search-backward\n" + "bindkey '^[[B' history-beginning-search-forward\n") +BIND = ("bind '\"\\e[A\": history-search-backward'\n" + "bind '\"\\e[B\": history-search-forward'\n") +OMZ = ("bindkey '^[[A' up-line-or-beginning-search\n" + "bindkey '^[[B' down-line-or-beginning-search\n") +WIDGET = ("my-up() { zle history-beginning-search-backward; }\n" + "zle -N my-up\nbindkey '^[[A' my-up\n") + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +READER = {} # env of the reader under test (in-process / forked) + + +def session(rc, typed, want, name, env=None): + """A fresh shell with `rc`, the four history lines, then `typed` and + Enter: the recalled line must print `want`.""" + e = dict(READER) + e.update(env or {}) + t = Tty(SHELL, rc="PS1='P$ '\nHISTFILE=/dev/null\n" + rc, env=e) + ok = t.expect(b"P$ ") + for c in HISTORY: + t.send(c + "\r") + ok = ok and t.expect(b"P$ ") + mark = len(t.out) + t.send(typed + "\r") + ok = ok and t.expect(b"\n" + want.encode() + b"\r\n") and t.expect(b"P$ ") + check(name, ok, repr(t.since(mark)[-300:])) + st = t.close() + check(name + "/exits-cleanly", os.WIFEXITED(st), "wait status %r" % st) + + +def run(name, rc, steps, env=None, hist=HISTORY, args=("-i",), bad=()): + """A shell with `rc` and the `hist` lines run; then each (keys, wait) + step: send `keys`, wait for the bytes `wait`. A step's wait is what + proves what the keys did. `bad` bytes must never be printed.""" + e = dict(READER) + e.update(env or {}) + t = Tty(SHELL, rc="PS1='P$ '\nHISTFILE=/dev/null\n" + rc, env=e, + args=args) + ok = t.expect(b"P$ ") + for c in hist: + t.send(c + "\r") + ok = ok and t.expect(b"P$ ") + for keys, wait in steps: + t.send(keys) + ok = ok and t.expect(wait) + ok = ok and t.expect(b"P$ ") + for b in bad: + ok = ok and b not in t.out + check(name, ok, repr(t.out[-300:])) + t.close() + + +def out(word): + return b"\n" + word.encode() + b"\r\n" + + +def edge(): + """Edge cases of the search itself.""" + run("edge/no-match-leaves-the-line", "", + [("echo q" + UP + "\r", out("q"))]) + run("edge/prefix-equal-to-an-entry", "", + [("echo gamma" + UP + UP + "\r", out("gamma"))]) + h3 = ["echo alpha", "echo;echo beta"] + run("edge/trailing-space-is-part-of-prefix", "", + [("echo " + UP + "\r", out("alpha"))], hist=h3) + run("edge/no-trailing-space-matches-both", "", + [("echo" + UP + "\r", out("beta"))], hist=h3) + # readline matches on the text BEFORE the cursor ("echo "); the text + # after it ("gam") is not part of the prefix and is replaced. + run("edge/cursor-mid-line-uses-text-before-it", "", + [("echo gam" + "\x1b[D" * 3 + UP + "\r", out("gamma"))], + hist=["echo alpha", "echo gamma", "echo gamut-not-this"][:2] + + ["printf 'delta\\n'"]) + run("edge/utf8-prefix", "", + [("echo \u00e9" + UP + UP + "\r", out("\u00e91"))], + hist=["echo \u00e91", "echo \u00fc2", "echo \u00e93"]) + run("edge/utf8-prefix-newest", "", + [("echo \u00e9" + UP + "\r", out("\u00e93"))], + hist=["echo \u00e91", "echo \u00fc2", "echo \u00e93"]) + run("edge/multiline-entry-by-prefix", "", + [("for" + UP + "\r", b"\nL1\r\nL2\r\n")], + hist=["echo a", "for i in 1 2\rdo echo L$i\rdone", "echo z"]) + run("edge/history-c-recalls-nothing", "", + [("history -c\r", b"P$ "), ("echo hi" + UP + "\r", out("hi"))]) + run("edge/history-d-deleted-line-gone", "", + [("history -d 3\r", b"P$ "), ("echo" + UP + "\r", out("alpha"))]) + run("edge/oldest-match-stays-put", "", + [("echo" + UP * 3 + "\r", out("alpha"))]) + # readline's own behaviour: Down past the newest match rings the bell + # and stays on that match (it does not go back to the typed prefix); + # the cursor stays after the prefix, so " hi" lands inside the line. + run("edge/down-past-newest-stays-on-newest-match", "", + [("echo" + UP + DOWN + DOWN + " hi\r", out("hi gamma"))]) + run("edge/ctrl-c-mid-walk-restarts", "", + [("echo" + UP + UP + "\x03", b"P$ "), + ("echo" + UP + "\r", out("gamma"))]) + + +def values(): + """Values of HELLISH_HISTORY_PREFIX, and where it is set.""" + up = [("echo" + UP + "\r", out("gamma"))] + plain = [("echo" + UP + "\r", out("delta"))] + run("val/uppercase-OFF-is-off", "HELLISH_HISTORY_PREFIX=OFF\n", plain) + run("val/no-stays-on", "HELLISH_HISTORY_PREFIX=no\n", up) + run("val/1-stays-on", "HELLISH_HISTORY_PREFIX=1\n", up) + run("val/readonly-0-is-off", "declare -r HELLISH_HISTORY_PREFIX=0\n", + plain, bad=(b"readonly", b"rror")) + run("val/local-in-function-leaves-on", + "f() { local HELLISH_HISTORY_PREFIX=0; :; }\nf\n", up, + bad=(b"rror",)) + run("val/prompt-command-turns-off", + "PROMPT_COMMAND='HELLISH_HISTORY_PREFIX=0'\n", plain) + run("val/prompt-command-turns-on", + "PROMPT_COMMAND='unset HELLISH_HISTORY_PREFIX'\n", up, + env={"HELLISH_HISTORY_PREFIX": "0"}) + run("val/exported-env-0-then-unset-in-shell", "", + [("unset HELLISH_HISTORY_PREFIX\r", b"P$ ")] + up, + env={"HELLISH_HISTORY_PREFIX": "0"}) + + +def modes(): + """Mode switches, user bindings, and other ways of starting.""" + up = [("echo" + UP + "\r", out("gamma"))] + run("mode/vi-then-emacs-keeps-search", "set -o vi\n", + [("set -o emacs\r", b"P$ ")] + up) + run("mode/emacs-then-vi-keeps-search", "", + [("set -o vi\r", b"P$ ")] + up) + run("mode/vi-emacs-vi-emacs", "", + [("set -o vi\r", b"P$ "), ("set -o emacs\r", b"P$ "), + ("set -o vi\r", b"P$ ")] + up) + run("mode/off-survives-vi-switch", "HELLISH_HISTORY_PREFIX=0\n", + [("set -o vi\r", b"P$ "), ("printf 'omega\\n'\r", b"P$ "), + ("echo" + UP + "\r", out("omega"))]) + run("user/bind-r-keeps-up-unbound", "bind -r '\\e[A'\n", + [("true\r", b"P$ "), ("echo" + UP + " hi\r", out("hi"))]) + # An explicit `bind` of the stock function, typed at a prompt, is the + # user's from then on: toggling the variable must never take it back. + run("user/runtime-bind-previous-history-survives-toggling", "", + [("bind '\"\\e[A\": previous-history'\r", b"P$ "), + ("HELLISH_HISTORY_PREFIX=0\r", b"P$ "), + ("unset HELLISH_HISTORY_PREFIX\r", b"P$ "), + ("printf 'omega\\n'\r", b"P$ "), + ("echo" + UP + "\r", out("omega"))]) + run("start/norc-gets-the-default", "", up, + args=("--norc", "-i"), env={"PS1": "P$ "}) + run("start/login-shell-gets-the-default", "", up, + args=("-l", "-i"), env={"PS1": "P$ "}) + + +def noninteractive(): + """bind -p of a non-interactive shell stays bash's, byte for byte.""" + bash = os.path.expanduser("~/bash-5.3.9/bin/bash") + cmd = "bind -p | grep -E 'history-search|previous-history|next-history'" + env = dict(os.environ, LC_ALL="C.UTF-8", LANG="C.UTF-8") + a = subprocess.run([SHELL, "-c", cmd], env=env, capture_output=True) + if not os.path.exists(bash): + print("skip noninteractive/bind-p-matches-bash (no oracle)") + return + b = subprocess.run([bash, "-c", cmd], env=env, capture_output=True) + check("noninteractive/bind-p-matches-bash", + a.stdout == b.stdout and a.stdout != b"", repr(a.stdout)) + + +def toggle(name): + """Runtime: default on, =0 -> plain, unset -> prefix again.""" + e = dict(READER) + t = Tty(SHELL, rc="PS1='P$ '\nHISTFILE=/dev/null\n", env=e) + ok = t.expect(b"P$ ") + for c in HISTORY + ["HELLISH_HISTORY_PREFIX=0", "printf 'omega\\n'"]: + t.send(c + "\r") + ok = ok and t.expect(b"P$ ") + t.send("echo" + UP + "\r") + ok = ok and t.expect(b"\nomega\r\n") and t.expect(b"P$ ") + t.send("unset HELLISH_HISTORY_PREFIX\r") + ok = ok and t.expect(b"P$ ") + t.send("echo" + UP + "\r") + ok = ok and t.expect(b"\ngamma\r\n") and t.expect(b"P$ ") + check(name, ok, repr(t.out[-300:])) + t.close() + + +def matrix(): + """Every case, for the reader currently in READER.""" + session("", "echo" + UP, "gamma", "default/up-finds-the-last-echo") + session("", "echo" + UP + UP, "alpha", "default/up-again-walks-back") + session("", "echo" + UP + UP + DOWN, "gamma", "default/down-forward") + session("", UP, "delta", "default/empty-line-up-is-plain") + session("", "echo" + "\x1bOA", "gamma", "default/app-cursor-up") + session("", "echo" + "\x1bOA\x1bOA\x1bOB", "gamma", + "default/app-cursor-down") + session("", "echo" + UP, "delta", "env0/plain", + {"HELLISH_HISTORY_PREFIX": "0"}) + session("HELLISH_HISTORY_PREFIX=off\n", "echo" + UP, "delta", + "rc-off/plain") + session("HELLISH_HISTORY_PREFIX=\n", "echo" + UP, "gamma", + "rc-empty/prefix") + toggle("toggle/runtime") + session("set -o vi\n", "echo" + UP, "gamma", "vi/insert-prefix") + session("set -o vi\n", "echo" + "\x1bOA", "gamma", "vi/insert-app-up") + rc = os.path.join(tempfile.mkdtemp(), "inputrc") + with open(rc, "w") as f: + f.write('"\\e[A": end-of-line\n') + session("", "echo" + UP + " hi", "hi", "inputrc/binding-wins", + {"INPUTRC": rc}) + session("bind '\"\\e[A\": previous-history'\n", "echo" + UP, "delta", + "bind/previous-history-wins") + session(BINDKEY, "echo" + UP, "gamma", "bindkey/up-finds-the-last-echo") + session(BINDKEY, "echo" + UP + UP, "alpha", "bindkey/up-again-walks-back") + session(BINDKEY, "echo" + UP + UP + DOWN, "gamma", + "bindkey/down-walks-forward") + session(BINDKEY, "printf" + UP + UP, "beta", "bindkey/another-prefix") + session(BIND, "echo" + UP, "gamma", "bind/up-finds-the-last-echo") + session(BIND, "echo" + UP + UP, "alpha", "bind/up-again-walks-back") + session(OMZ, "echo" + UP + UP, "alpha", "omz-names/up-line-or-beginning") + session(WIDGET, "echo" + UP, "gamma", "zle-builtin/called-from-a-widget") + session(BIND, "echo" + UP, "gamma", "bind-explicit/survives-off", + {"HELLISH_HISTORY_PREFIX": "0"}) + + +def main(): + noninteractive() + for cell, env in (("inproc", {}), ("fork", {"HELLISH_RL_FORK": "1"})): + READER.clear() + READER.update(env) + print("== reader: " + cell) + before = len(FAILS) + matrix() + edge() + values() + modes() + for i in range(before, len(FAILS)): + FAILS[i] = cell + ":" + FAILS[i] + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/io_number b/tests/io_number new file mode 100644 index 00000000..8d66422a --- /dev/null +++ b/tests/io_number @@ -0,0 +1,45 @@ +echo hi 255>/dev/null; echo rc=$? +echo hi 10>outfiles/a; echo rc=$?; cat outfiles/a +echo x 123>&1; echo rc=$? +echo x 1024>/dev/null; echo rc=$? +echo x 007>/dev/null; echo rc=$? +echo x 0010>&1; echo rc=$? +{ echo x 1234567>/dev/null; } 2>&1 | sed 's/^.*: //' +{ echo x 2147483647>/dev/null; echo rc=$?; } 2>&1 | sed 's/^.*: //' +echo x 2147483648>outfiles/a; echo rc=$?; cat outfiles/a +echo x 99999999999>outfiles/a; echo rc=$?; cat outfiles/a +echo x 99999999999999999999999>outfiles/a; cat outfiles/a +echo a2>outfiles/a; echo rc=$?; cat outfiles/a +echo a12>outfiles/a; echo rc=$?; cat outfiles/a +echo2>/dev/null; echo rc=$? +x=2>outfiles/a; echo "[$x]"; cat outfiles/a +x=12>outfiles/a y=3; echo "[$x][$y]" +echo "x"12>outfiles/a; echo rc=$?; cat outfiles/a +echo 'x'2>outfiles/a; cat outfiles/a +echo x\12>outfiles/a; cat outfiles/a +f=a; echo $f2>outfiles/b; echo rc=$?; cat outfiles/b +a=(1 2); echo ${a[1]}2>outfiles/a; cat outfiles/a +echo $(echo a2>/dev/null) +echo "$(echo b2>/dev/null)" +echo hi >&2>/dev/null; echo rc=$? +echo hi >& 2>/dev/null; echo rc=$? +echo hi 1>&2 2>/dev/null; echo rc=$? +{ echo in <&0>outfiles/a; }; cat outfiles/a +exec 100>outfiles/a; echo y >&100; exec 100>&-; cat outfiles/a +exec 255>outfiles/a; echo z >&255; exec 255>&-; cat outfiles/a +exec 3>outfiles/a 42>&3; echo q >&42; exec 3>&- 42>&-; cat outfiles/a +echo x 2<>/dev/null 3>>/dev/null 4>|/dev/null 12<>/dev/null; echo rc=$? +eval $'cat 11<outfiles/a; echo rc=$?; cat outfiles/a +{ echo grp >&13; }13>outfiles/a; cat outfiles/a +f() { echo fn >&14; }; f 14>outfiles/a; cat outfiles/a +for i in 1 2; do echo $i >&15; done 15>outfiles/a; cat outfiles/a +echo a;2>/dev/null echo b +echo $((2>1)) $((12>3)) +((2>1)) && echo arith +case 12 in 12) echo twelve;; esac +echo 1 2>/dev/null 3 4 +echo 12 >outfiles/a; cat outfiles/a +echo 3>&1 4 diff --git a/tests/omz_sudo_widget_test.py b/tests/omz_sudo_widget_test.py new file mode 100644 index 00000000..8ef9898e --- /dev/null +++ b/tests/omz_sudo_widget_test.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +"""ESC ESC, oh-my-zsh's sudo widget, end to end (issue #137). + + ╰─ ❯ hellish: ${${(Az)aliases[$cmd]}[1]:-$cmd}: bad substitution + hellish: ${__hsh_nest_0[1]:-$cmd}: bad substitution + +The framework installs omz's `sudo` plugin by default and it binds ESC ESC, +so pressing Escape twice on a fresh shell printed that. Four defects, found +one behind the other by pressing the key: + + 1. `aliases` read empty: its WRITES reached the alias table (#114), its + reads did not, so "what does this alias run" had no answer. + 2. `${${(...)...}[1]:-x}` -- a subscript AND an operator after a nested + flagged expansion -- was a bad substitution, and so was a flag on a + subscripted name, `${(z)aliases[ll]}`. + 3. `local new=$2` word-split $2: bash and zsh never split an assignment + argument of a declaration builtin, hellish did for all but `export`, + so `vim f` became `sudo/f` instead of `sudo -e f`. + 4. On an empty line the widget asks `$(fc -ln -1)` for the previous + command; `-ln` was not a listing to fc, which opened $EDITOR instead. + +The edit is checked by RUNNING the line with a fake sudo and vim on PATH +that print their arguments -- not by reading readline's redraw, which is +free to repaint the line however it likes. + +The plugin comes from the same cache as plugin_corpus_test.py (downloaded +once, PLUGIN_CACHE / XDG_CACHE_HOME); with neither a cached copy nor a +network this SKIPS, out loud. Waits on output, never on the clock. + +Usage: python3 omz_sudo_widget_test.py [/path/to/hellish] +""" +import os +import stat +import sys +import tempfile +import urllib.error +import urllib.request + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +URL = ("https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/" + "plugins/sudo/sudo.plugin.zsh") +CACHE = os.environ.get( + "PLUGIN_CACHE", + os.path.join(os.environ.get("XDG_CACHE_HOME", + os.path.expanduser("~/.cache")), + "hellish-plugin-corpus")) +FAILS = [] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def plugin(): + """The cached plugin, fetched once. None means SKIP, not failure.""" + path = os.path.join(CACHE, "omz-sudo.zsh") + if os.path.exists(path) and os.path.getsize(path) > 0: + return path + if os.environ.get("OFFLINE") == "1": + return None + try: + os.makedirs(CACHE, exist_ok=True) + with urllib.request.urlopen(URL, timeout=20) as r: + data = r.read() + if not data: + return None + with open(path, "wb") as f: + f.write(data) + return path + except (urllib.error.URLError, OSError, ValueError): + return None + + +def fakes(): + """A bin dir whose sudo and vim only say what they were given.""" + d = tempfile.mkdtemp(prefix="sudo-widget-bin-") + for name, tag in (("sudo", "FAKESUDO"), ("vim", "FAKEVIM")): + p = os.path.join(d, name) + with open(p, "w") as f: + f.write('#!/bin/sh\necho "%s[$*]"\n' % tag) + os.chmod(p, os.stat(p).st_mode | stat.S_IEXEC) + return d + + +def line(t, keys, expect, name): + """Type `keys` (a list: text, then the ESC ESC widget, then Enter) and + expect `expect` from what the resulting line ran.""" + mark = len(t.out) + for k in keys: + t.send(k) + ok = t.expect(expect) and t.expect(b"P$ ") + check(name, ok, repr(t.since(mark)[-300:])) + + +def main(): + path = plugin() + if not path: + print("SKIP omz sudo plugin: no cached copy and no network") + return 0 + rc = "PS1='P$ '\nEDITOR=vim\nexport EDITOR\n. %s\n" % path + t = Tty(SHELL, rc=rc, path=fakes() + ":" + os.environ.get("PATH", "")) + check("start/prompt", t.expect(b"P$ "), repr(t.out[-300:])) + esc = b"\x1b\x1b" + line(t, ["echo a1", esc, "\r"], b"FAKESUDO[echo a1]", + "prefix/plain-command-gets-sudo") + line(t, ["sudo echo a2", esc, "\r"], b"\r\na2\r\n", + "toggle/sudo-comes-back-off") + line(t, ["vim /tmp/f3", esc, "\r"], b"FAKESUDO[-e /tmp/f3]", + "editor/becomes-sudo-e") + line(t, ["sudo -e /tmp/f4", esc, "\r"], b"FAKEVIM[/tmp/f4]", + "editor/sudo-e-goes-back-to-EDITOR") + t.send("alias v=vim\r") + t.expect(b"P$ ") + line(t, ["v /tmp/f5", esc, "\r"], b"FAKESUDO[-e /tmp/f5]", + "alias/of-the-editor-is-the-editor") + t.send("echo a6\r") + t.expect(b"a6\r\nP$ ") + line(t, [esc, "\r"], b"FAKESUDO[echo a6]", + "empty-line/recalls-the-previous-command") + check("no-bad-substitution", b"bad substitution" not in t.out, + repr(t.out[-400:])) + st = t.close() + check("exit/not-signalled", not os.WIFSIGNALED(st)) + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/prompt_command_box_test.py b/tests/prompt_command_box_test.py new file mode 100644 index 00000000..a3cabd0a --- /dev/null +++ b/tests/prompt_command_box_test.py @@ -0,0 +1,359 @@ +#!/usr/bin/env python3 +"""The boxed prompt from issue #132 keeps the cursor where the terminal has it. + +The report's configuration builds a two-row, coloured PS1 in a +PROMPT_COMMAND function on every prompt, times commands with a DEBUG trap +that runs `date` in a command substitution, and still had a coloured +RPROMPT clock from an earlier zsh attempt: + + ╭─ javiesan in ~/proyectos/Cursus/Python/PyMod00 on main* + ╰─ ✔ ❯ 20:21:37 + +Typing on the second row overwrote the prompt: the input started at +column 0, over `╰─ ✔ ❯`, and a long line ran on over rows below. The +cause was fixed in 3.1.0 (the right prompt was drawn inside the line +editor's own prompt string, which made the editor believe the prompt was +nine columns wider than it was). prompt_drift_matrix_test.py pins that +cause with a fixed PS1; this pins the report itself, with its rc as the +user had it, so that the shape that failed in the field keeps working: +a PS1 that PROMPT_COMMAND rewrites before every prompt, a DEBUG trap +that forks, a git segment, and a `✘130` / `✘127` status that changes the +prompt's width from one prompt to the next. + +A terminal emulator (pyte) is the judge, never hellish's own width model. +Both line readers are covered: in-process, and forked (HELLISH_RL_FORK=1). + +Usage: python3 prompt_command_box_test.py [/path/to/hellish] +""" +import fcntl +import os +import pty +import select +import shutil +import struct +import subprocess +import sys +import tempfile +import termios +import time + +HERE = os.path.dirname(os.path.abspath(__file__)) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(HERE, "..", "build", "bin", + "hellish")) +COLS, ROWS = 120, 30 +FAILS = [] + +try: + import pyte + + class Screen(pyte.Screen): + """pyte dispatches CSI with parameters; save/restore take none.""" + + def save_cursor(self, *_): + super().save_cursor() + + def restore_cursor(self, *_): + super().restore_cursor() + + pyte.Stream.csi = dict(pyte.Stream.csi, s="save_cursor", + u="restore_cursor") +except ImportError: # pragma: no cover + pyte = None + +# The issue's ~/.hellish/rc.d/50-prompt.hsh, verbatim but for the +# comments, followed by the RPROMPT line its earlier zsh block had left. +RC = r""" +HX_CMD_START=0 +HX_CMD_DUR="" + +hx_preexec() { + [[ "$BASH_COMMAND" == "$PROMPT_COMMAND" ]] && return + HX_CMD_START=$(date +%s%N) +} +trap 'hx_preexec' DEBUG + +hx_git_branch() { + local b + b=$(git symbolic-ref --short HEAD 2>/dev/null) || return + if git diff --quiet --ignore-submodules HEAD 2>/dev/null; then + printf '%s' "$b" + else + printf '%s*' "$b" + fi +} + +hx_precmd() { + local exit=$? + + HX_CMD_DUR="" + if (( HX_CMD_START > 0 )); then + local elapsed=$(( ( $(date +%s%N) - HX_CMD_START ) / 1000000000 )) + (( elapsed >= 1 )) && HX_CMD_DUR=" took ${elapsed}s" + fi + HX_CMD_START=0 + + local branch + branch=$(hx_git_branch) + [[ -n "$branch" ]] && branch=" on \[\e[35m\]${branch}\[\e[0m\]" + + local icon color + if [[ $exit -eq 0 ]]; then icon="✔"; color="\[\e[32m\]" + else icon="✘$exit"; color="\[\e[31m\]"; fi + + PS1="\[\e[90m\]╭─\[\e[0m\] \[\e[32m\]\u\[\e[0m\] in \[\e[36m\]\w\[\e[0m\]${branch}${HX_CMD_DUR} \[\e[90m\]\A\[\e[0m\] +\[\e[90m\]╰─\[\e[0m\] ${color}${icon}\[\e[0m\] ${color}❯\[\e[0m\] " +} + +PROMPT_COMMAND="hx_precmd" +RPROMPT='%F{240}%*%f' +""" + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +class Session: + def __init__(self, env, cwd): + self.screen = Screen(COLS, ROWS) + self.stream = pyte.ByteStream(self.screen) + self.buf = bytearray() + self.pid, self.fd = pty.fork() + if self.pid == 0: + os.chdir(cwd) + os.environ.clear() + os.environ.update(env) + os.execv(SHELL, [SHELL, "-i"]) + os._exit(127) + fcntl.ioctl(self.fd, termios.TIOCSWINSZ, + struct.pack("HHHH", ROWS, COLS, 0, 0)) + + def drain(self, cap=3.0, quiet=0.25): + """Read until the pty has been quiet for `quiet` s; the new bytes.""" + start = len(self.buf) + last = time.monotonic() + end = last + cap + while time.monotonic() < end: + r, _, _ = select.select([self.fd], [], [], 0.02) + if r: + try: + d = os.read(self.fd, 65536) + except OSError: + break + if not d: + break + self.buf.extend(d) + self.stream.feed(d) + last = time.monotonic() + elif time.monotonic() - last > quiet: + break + return bytes(self.buf[start:]) + + def wait_for(self, text, cap=8.0): + """Drain until `text` is on screen (a prompt the hook built).""" + end = time.monotonic() + cap + while time.monotonic() < end: + if any(text in r for r in self.rows()): + self.drain(1.0, 0.15) + return True + self.drain(0.2, 0.05) + return False + + def until(self, pred, cap=8.0): + """Drain until pred() holds or `cap` s pass; pred()'s last value. + A loaded runner may pause mid-redraw, so a check waits for the + state it expects instead of trusting a quiet gap.""" + end = time.monotonic() + cap + while not pred() and time.monotonic() < end: + self.drain(0.2, 0.05) + return pred() + + def type(self, text): + """One key at a time, the way a person types.""" + for ch in text: + os.write(self.fd, ch.encode()) + self.drain(1.0, 0.01) + self.drain(1.0, 0.15) + + def send(self, data): + os.write(self.fd, data) + return self.drain() + + def rows(self): + return [r.rstrip() for r in self.screen.display] + + def cursor(self): + return self.screen.cursor.y, self.screen.cursor.x + + def close(self): + try: + os.write(self.fd, b"\x15exit\r") + self.drain(1.0) + os.kill(self.pid, 9) + except OSError: + pass + os.waitpid(self.pid, 0) + + +def cell(label, home, repo, fork): + env = {"HOME": home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "xterm-256color", "LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", + "USER": "tester", "HELLISH_NO_BANNER": "1", + "HELLISH_NO_UPDATE_CHECK": "1", "HELLISH_NO_ANIM": "1", + "ASAN_OPTIONS": "detect_leaks=0"} + if fork: + env["HELLISH_RL_FORK"] = "1" + s = Session(env, repo) + try: + run_cell(label, s) + finally: + s.close() + + +def prompt_ok(s, y, icon): + """Row y-1 is the box's top, row y starts with the status row.""" + rows = s.rows() + top = rows[y - 1] if y > 0 else "" + want = "╰─ %s ❯ " % icon + return (top.startswith("╭─ tester in ~/r on main*") + and rows[y].startswith(want)), top, rows[y] + + +def run_cell(label, s): + ok = s.wait_for("╰─ ✔ ❯") + check("%s: first prompt drawn" % label, ok, "rows=%r" % s.rows()[:4]) + if not ok: + return + y, x = s.cursor() + good, top, row = prompt_ok(s, y, "✔") + check("%s: the box is two rows, cursor after ❯" % label, + good and x == 7, "cursor=%d,%d top=%r row=%r" % (y, x, top, row)) + check("%s: RPROMPT clock at the right of the status row" % label, + row.rstrip()[-8:].count(":") == 2 and len(row.rstrip()) > 100, + "row=%r" % row) + + # Typing starts after the prompt, never over it (the report). + s.type("bbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaa") + s.until(lambda: s.cursor() == (y, 45)) + y2, x2 = s.cursor() + row = s.rows()[y2] + check("%s: typed text follows the prompt on its row" % label, + y2 == y and row.startswith("╰─ ✔ ❯ bbbbbbbbbbbbaaaa") and x2 == 45, + "cursor=%d,%d row=%r" % (y2, x2, row)) + + # Ctrl-A goes back to the prompt's end on the same row. + start = len(s.buf) + s.send(b"\x01") + s.until(lambda: s.cursor() == (y, 7), 3.0) + out = bytes(s.buf[start:]) + y3, x3 = s.cursor() + check("%s: Ctrl-A lands just after ❯, same row" % label, + (y3, x3) == (y, 7) and b"\x1b[A" not in out, + "cursor=%d,%d out=%r" % (y3, x3, out[:80])) + + # A line longer than the terminal wraps under the prompt: the prompt + # row is intact and the rest continues at column 0 of the next row. + s.send(b"\x05") + s.type("d" * 110) + total = 7 + 38 + 110 + s.until(lambda: s.cursor() == (y + 1, total - COLS)) + rows = s.rows() + y4, x4 = s.cursor() + check("%s: a wrapped line keeps the prompt and wraps once" % label, + rows[y].startswith("╰─ ✔ ❯ bbbb") and y4 == y + 1 + and x4 == total - COLS and rows[y + 1] == "d" * (total - COLS), + "cursor=%d,%d rows=%r" % (y4, x4, rows[y:y + 3])) + + # ^C: the status becomes ✘130, a wider prompt, drawn on fresh rows. + s.send(b"\x03") + ok = s.wait_for("╰─ ✘130 ❯") + s.until(lambda: s.cursor()[1] == 10, 3.0) + y5, x5 = s.cursor() + good, top, row = prompt_ok(s, y5, "✘130") + check("%s: after ^C the ✘130 prompt is whole" % label, + ok and good and x5 == 10 and y5 >= y + 3, + "cursor=%d,%d top=%r row=%r" % (y5, x5, top, row)) + s.type("echo " + "z" * 20) + s.until(lambda: s.cursor() == (y5, 35)) + row = s.rows()[y5] + check("%s: typing after ✘130 follows the wider prompt" % label, + row.startswith("╰─ ✘130 ❯ echo zzzz") and s.cursor() == (y5, 35), + "cursor=%s row=%r" % (s.cursor(), row)) + + # Run it, then a missing command: ✘127, still whole. + s.send(b"\r") + s.wait_for("╰─ ✔ ❯") + s.type("ghhgg") + s.send(b"\r") + ok = s.wait_for("╰─ ✘127 ❯") + s.until(lambda: s.cursor()[1] == 10, 3.0) + y6, x6 = s.cursor() + good, top, row = prompt_ok(s, y6, "✘127") + check("%s: after a missing command the ✘127 prompt is whole" % label, + ok and good and x6 == 10, + "cursor=%d,%d top=%r row=%r" % (y6, x6, top, row)) + + # Up recalls the last command after the prompt; Down comes back to the + # empty line with the screen as it was. + before = (s.cursor(), s.rows()) + s.send(b"\x1b[A") + s.until(lambda: s.cursor() == (y6, 15), 3.0) + row = s.rows()[y6] + check("%s: Up recalls the command after the prompt" % label, + row.startswith("╰─ ✘127 ❯ ghhgg") and s.cursor() == (y6, 15), + "cursor=%s row=%r" % (s.cursor(), row)) + s.send(b"\x1b[B") + s.until(lambda: s.cursor() == before[0], 3.0) + after = (s.cursor(), s.rows()) + check("%s: Down restores the screen exactly" % label, + before[0] == after[0] and before[1][y6 - 1:y6 + 2] + == after[1][y6 - 1:y6 + 2], + "before=%r after=%r" % (before[1][y6], after[1][y6])) + + # Left alone, nothing is repainted anywhere else. + time.sleep(1.2) + s.drain(0.5, 0.2) + check("%s: idle, the prompt stays put" % label, + s.cursor() == after[0] and s.rows()[y6 - 1:y6 + 1] + == after[1][y6 - 1:y6 + 1], + "cursor=%s rows=%r" % (s.cursor(), s.rows()[y6 - 1:y6 + 2])) + + +def main(): + if not os.path.exists(SHELL): + print("no shell at", SHELL) + return 1 + if pyte is None: + print("skip: needs pyte") + return 0 + if not shutil.which("git"): + print("skip: needs git") + return 0 + home = tempfile.mkdtemp(prefix="pcbox-") + try: + with open(os.path.join(home, ".hellishrc"), "w") as f: + f.write(RC) + repo = os.path.join(home, "r") + os.mkdir(repo) + env = dict(os.environ, HOME=home, GIT_CONFIG_NOSYSTEM="1") + for argv in (["git", "init", "-q", "-b", "main", repo], + ["git", "-C", repo, "-c", "user.name=t", + "-c", "user.email=t@t", "commit", "-q", + "--allow-empty", "-m", "i"]): + subprocess.run(argv, check=True, env=env) + with open(os.path.join(repo, ".dirty"), "w") as f: + f.write("x\n") + subprocess.run(["git", "-C", repo, "add", ".dirty"], check=True, + env=env) + cell("inproc", home, repo, False) + cell("fork", home, repo, True) + finally: + shutil.rmtree(home, ignore_errors=True) + print("\n%d failed" % len(FAILS) if FAILS else "\nall passed") + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/prompt_doc_example_test.py b/tests/prompt_doc_example_test.py new file mode 100644 index 00000000..8c5f2fba --- /dev/null +++ b/tests/prompt_doc_example_test.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 +"""The prompt example the docs give is the one that works (issue #134, 2, 8). + +USER_DOC.md and hellishrc.example both say that `$(...)` in a prompt is +never run, and show the way to get a computed segment instead: a variable +set by a HELLISH_PRECMD_FUNCS hook. Documentation that no longer matches +the binary is what section 8 of the issue was about, so this runs the +snippet exactly as USER_DOC.md prints it -- read from the file, not +copied here -- and checks that: + * hellishrc.example carries the same snippet; + * the hook's segment shows up in the prompt, and follows a `cd` into a + git repository on the next prompt; + * `$(...)` and backquotes in PS1 stay on screen as written. + +Usage: python3 prompt_doc_example_test.py [/path/to/hellish] +""" +import os +import re +import shutil +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] +T = 8.0 + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def doc_snippet(): + """The ```sh block that follows 'For a computed segment' in USER_DOC.""" + with open(os.path.join(ROOT, "USER_DOC.md")) as f: + text = f.read() + m = re.search(r"For a computed segment.*?```sh\n(.*?)```", text, re.S) + return m.group(1) if m else None + + +def example_snippet(): + """The same lines as hellishrc.example comments them: '# code'.""" + with open(os.path.join(ROOT, "hellishrc.example")) as f: + lines = f.read().split("\n") + out = [ln[6:] for ln in lines if ln.startswith("# ")] + return "\n".join(out) + + +def main(): + if not os.path.exists(SHELL): + print("no shell at", SHELL) + return 1 + snip = doc_snippet() + check("doc/USER_DOC-has-the-snippet", snip is not None) + if snip is None: + return 1 + ex = example_snippet() + check("doc/hellishrc.example-has-the-same-snippet", + all(ln in ex for ln in snip.strip().split("\n")), + "snippet=%r" % snip) + if not shutil.which("git"): + print("skip: no git") + return 1 if FAILS else 0 + t = Tty(SHELL, rc=snip + "PS2='> '\n") + try: + ok = t.expect(b"~ ", T) + check("prompt/renders-outside-a-repo", ok, "tail=%r" % t.out[-120:]) + repo = os.path.join(t.home, "r") + os.mkdir(repo) + subprocess.run(["git", "init", "-q", repo], check=True) + subprocess.run(["git", "-C", repo, "checkout", "-q", "-b", "topic"], + check=True) + t.send("cd r\r") + check("prompt/hook-segment-follows-cd", t.expect(b"(topic) ~/r ", T), + "tail=%r" % t.out[-160:]) + t.send("PS1='[$(echo CS)] [`echo BQ`] '\r") + check("prompt/command-substitution-is-not-run", + t.expect(b"[$(echo CS)] [`echo BQ`] ", T), + "tail=%r" % t.out[-160:]) + finally: + t.close() + print("\n%d failed" % len(FAILS) if FAILS else "\nall passed") + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/prompt_expand_p_test.py b/tests/prompt_expand_p_test.py new file mode 100644 index 00000000..aa80b5a2 --- /dev/null +++ b/tests/prompt_expand_p_test.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""${PS1@P} is the prompt the shell actually shows (issue #134, section 1). + +`prompt preview` rendered PS1 themes with `print -rP`. print -P is zsh's +and, as in zsh, scans what a variable expanded to for escapes again; the +PS1 renderer does not (bash's order). So a theme segment holding +`100%done` previewed as `100/home/...one`, a prompt that never appears. +${var@P}, bash's prompt-expansion operator, now runs the PS1 rule itself +and the preview uses it. + +What this pins, at a real prompt: + * the live PS1 and ${PS1@P} are the same bytes, in hellish's bilingual + PS1 (a `%~` escape and a `\\W` escape side by side) and under + `set -o zsh`, where PS1 is zsh's PROMPT and a variable's `%` IS read + again -- @P follows the mode the prompt follows; + * a variable's value is left as it is in the bilingual PS1; + * `prompt preview` prints that same line for a PS1 theme. + +The part of @P hellish shares with bash is graded against bash in +tests/scripts/61_prompt_expand_P.sh. + +Cells: the in-process reader, and HELLISH_RL_FORK=1. + +Usage: python3 prompt_expand_p_test.py [/path/to/hellish] +""" +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +SWITCH = os.path.join(ROOT, "share", "rc.d", "40-prompt-switch.hsh") +FAILS = [] +T = 8.0 +ROOT_USER = os.geteuid() == 0 + +# The value is what a theme segment gets from a hook: text with a percent +# and a backslash in it. +VALUE = r"100%done \w %d" +RC = ("V='%s'\n" % VALUE + + "PS1='[${V}|%~|\\W|$((6*7))]\\$ '\n") +# HOME is the pty's temp dir and the shell starts in it: %~ and \W are ~. +LIVE = ("[%s|~|~|42]%s " % (VALUE, "#" if ROOT_USER else "$")).encode() + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def at_p(t, var): + """Print ${var@P} between markers; return it, or None on a timeout. + The markers are typed split (B''EG, E''ND) so their echo cannot be + mistaken for their output, and the slice ends where the END marker + was consumed (t.pos): the read that delivers it may also carry the + next prompt, which must not leak into the value.""" + t.send("printf 'B''EG%%sE''ND\\n' \"${%s@P}\"\r" % var) + if not t.expect(b"\nBEG", T): + return None + mark = t.pos + if not t.expect(b"END\r\n", T): + return None + return t.out[mark:t.pos - 5] + + +def live_prompt(t, suffix): + """Run a command and return the prompt printed after it, through + `suffix` (the prompt's known last bytes). The marker is typed as + MA''RK so that its echo cannot be mistaken for its output.""" + t.send("echo MA''RK\r") + if not t.expect(b"\nMARK\r\n", T): + return None + mark = t.pos + if not t.expect(suffix, T): + return None + return t.out[mark:t.pos] + + +def bilingual_cell(cell, env): + t = Tty(SHELL, rc=RC, env=env) + try: + ok = t.expect(LIVE, T) + check("%s/live-prompt-leaves-the-value" % cell, ok, + "tail=%r" % t.out[-160:]) + got = at_p(t, "PS1") + check("%s/PS1@P-is-the-live-prompt" % cell, got == LIVE, + "got=%r want=%r" % (got, LIVE)) + # A later change to the variable shows in both, the same way. + t.send("V='50%off \\e'\r") + want = LIVE.replace(VALUE.encode(), b"50%off \\e") + live = live_prompt(t, want[-4:]) + check("%s/live-prompt-follows-the-variable" % cell, live == want, + "live=%r want=%r" % (live, want)) + got = at_p(t, "PS1") + check("%s/PS1@P-follows-the-variable" % cell, got == want, + "got=%r want=%r" % (got, want)) + finally: + t.close() + + +def zsh_cell(cell, env): + """Under set -o zsh, PS1 is zsh's PROMPT: the value is substituted + first and its `%` IS an escape (zsh's PROMPT_SUBST order). @P follows + the prompt there too.""" + rc = "V='50%~x'\nset -o zsh\nPS1='[${V}]%# '\n" + t = Tty(SHELL, rc=rc, env=env) + try: + suffix = b"]# " if ROOT_USER else b"]% " + live = live_prompt(t, suffix) + check("%s/zsh-live-prompt-reads-the-value" % cell, + live == b"[50~x" + suffix, + "live=%r" % live) + got = at_p(t, "PS1") + check("%s/zsh-PS1@P-is-the-live-prompt" % cell, got == live, + "got=%r live=%r" % (got, live)) + finally: + t.close() + + +def preview_cell(cell, env): + """prompt preview prints, for a PS1 theme, the line the prompt shows + once the theme is active.""" + t = Tty(SHELL, rc=RC, env=env) + tdir = os.path.join(t.home, "themes") + os.mkdir(tdir) + with open(os.path.join(tdir, "pct.hsh"), "w") as f: + f.write("PS1='[${V}|%~|\\W|$((6*7))]\\$ '\n") + try: + t.expect(LIVE, T) + t.send("HELLISH_THEMES=%s; . %s; prompt preview; echo DO''NE\r" + % (tdir, SWITCH)) + mark = len(t.out) + ok = t.expect(b"\nDONE\r\n", T) + out = t.since(mark) + line = b"\npct " + LIVE + b"\n" + check("%s/preview-is-the-live-prompt" % cell, ok and line in out, + "out=%r want=%r" % (out[-200:], line)) + finally: + t.close() + + +def main(): + if not os.path.exists(SHELL): + print("no shell at", SHELL) + return 1 + for cell, env in (("inproc", {}), ("fork", {"HELLISH_RL_FORK": "1"})): + bilingual_cell(cell, env) + zsh_cell(cell, env) + preview_cell(cell, env) + print("\n%d failed" % len(FAILS) if FAILS else "\nall passed") + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/prompt_row_writes_test.py b/tests/prompt_row_writes_test.py new file mode 100644 index 00000000..2686cb30 --- /dev/null +++ b/tests/prompt_row_writes_test.py @@ -0,0 +1,222 @@ +#!/usr/bin/env python3 +"""Regression gate: no escape sequence reaches the tty split across write()s. + +prompt_atomic_test.py hammers a live prompt with type-ahead and counts the +colour escapes it finds broken. That is the symptom, and it is a race: on +an idle machine it passes, on a loaded CI runner it fails now and then -- +it failed on develop at a2b7f1b, in the prompt's LAST row, the one row +rl_prerow.c's fix could not account for. + +This file checks the cause instead, and the cause is not timing. Linux's +line discipline keeps the echo of a key it could not put out yet (the +reader was behind) and emits it at the start of the NEXT write() on the +tty -- whatever ECHO is by then (n_tty_write -> process_echoes). So every +write() boundary is a place a typed key can appear, and a boundary inside +`\\e[38;2;152;195;121m` is a place it splits the escape: every letter is a +CSI final byte, and the tail prints as text (issues #10, #19; #5 for a +split UTF-8 glyph). + +readline drew the prompt through an unbuffered stderr, one putc per byte +-- 26 write()s for `\\e[38;2;152;195;121m> \\e[0m` -- so its row was all +boundaries. bash line-buffers stderr (shell_initialize) and its row goes +out in one write(). The invariant pinned here: + + no write() boundary falls inside an escape sequence or a UTF-8 character + anywhere in what the shell sends the terminal, over several prompts, a + typed line, and a redraw. + +It holds on any machine or none: the answer is in strace's record of the +write() calls, not in a race. Skips (exit 0, says so) without strace; CI +installs it. + +Usage: python3 prompt_row_writes_test.py /path/to/hellish +""" +import os +import pty +import re +import select +import shutil +import sys +import tempfile +import time + +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else "../build/bin/hellish") +STRACE = shutil.which("strace") +FAILS = [] +GREEN = "\x1b[38;2;152;195;121m" +# two rows, like the shipped default prompt: truecolour escapes and box +# glyphs above, and a last row -- readline's -- with a colour and a +# multibyte glyph of its own +PS1 = ("\\[\x1b[38;2;122;162;247m\\]\u256d\u2500\u2500 row\\[\x1b[0m\\]\\n" + "\\[" + GREEN + "\\]\u276f \\[\x1b[0m\\]") +MARK = "\u276f ".encode() +# a whole CSI sequence, and a whole OSC/charset/two-byte ESC sequence +ESC_SEQ = re.compile(rb"\x1b(?:\[[0-9;?<=>!]*[ -/]*[@-~]|[()][0-9A-Za-z]" + rb"|\][^\x07\x1b]*(?:\x07|\x1b\\)|[ -~])") +WRITE = re.compile(r'^(\d+)\s+write\(2, "((?:\\x[0-9a-f]{2})*)", \d+\)\s+=\s+(\d+)') + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def utf8_len(b): + if b >= 0xf0: + return 4 + if b >= 0xe0: + return 3 + if b >= 0xc0: + return 2 + return 1 + + +def atoms(stream): + """(start, end) of every escape sequence and multibyte character.""" + out = [] + i = 0 + while i < len(stream): + if stream[i] == 0x1b: + m = ESC_SEQ.match(stream, i) + n = (m.end() - i) if m else 1 + else: + n = utf8_len(stream[i]) + if n > 1: + out.append((i, i + n)) + i += n + return out + + +def split_atoms(writes): + """The atoms some write() boundary falls strictly inside.""" + stream = b"".join(writes) + cuts = set() + at = 0 + for w in writes[:-1]: + at += len(w) + cuts.add(at) + bad = [] + for a, b in atoms(stream): + if any(a < c < b for c in range(a + 1, b) if c in cuts): + bad.append(stream[a:b]) + return bad + + +class Traced: + def __init__(self, home, trace): + env = { + "HOME": home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "xterm-256color", "LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", + "PS1": PS1, "HELLISH_NO_BANNER": "1", "HELLISH_BANNER": "0", + "HELLISH_NO_UPDATE_CHECK": "1", "HELLISH_NO_ANIM": "1", + "ASAN_OPTIONS": "detect_leaks=0", + } + self.buf = bytearray() + self.pid, self.fd = pty.fork() + if self.pid == 0: + os.chdir(home) + os.execve(STRACE, [STRACE, "-f", "-xx", "-s", "65535", + "-e", "trace=write", "-o", trace, + SHELL, "--norc"], env) + os._exit(127) + + def drain(self, secs, quiet=0.3): + end = time.monotonic() + secs + last = time.monotonic() + while time.monotonic() < end: + r, _, _ = select.select([self.fd], [], [], 0.05) + if r: + try: + d = os.read(self.fd, 65536) + except OSError: + return + if not d: + return + self.buf.extend(d) + last = time.monotonic() + elif time.monotonic() - last > quiet and self.buf: + return + + def prompts(self): + return self.buf.count(MARK) + + def wait_prompts(self, n, cap=15.0): + end = time.monotonic() + cap + while self.prompts() < n and time.monotonic() < end: + self.drain(0.5, quiet=0.1) + return self.prompts() >= n + + def send(self, data): + os.write(self.fd, data) + + def close(self): + try: + os.write(self.fd, b"exit\r") + self.drain(2.0) + except OSError: + pass + try: + os.kill(self.pid, 9) + except OSError: + pass + try: + os.waitpid(self.pid, 0) + except ChildProcessError: + pass + + +def tty_writes(trace): + """Every complete write() to fd 2, per pid, in order.""" + per = {} + with open(trace, errors="replace") as f: + for line in f: + m = WRITE.match(line) + if m: + data = bytes.fromhex(m.group(2).replace("\\x", "")) + per.setdefault(m.group(1), []).append(data[:int(m.group(3))]) + return per + + +def main(): + if not STRACE: + print("skip: strace not installed (apt-get install strace)") + return 0 + base = tempfile.mkdtemp(prefix="hellish_rowwrites_") + home = os.path.join(base, "home") + os.makedirs(home) + trace = os.path.join(base, "trace") + s = Traced(home, trace) + check("the prompt appeared under strace", s.wait_prompts(1), + repr(bytes(s.buf[-300:]))) + for i in range(4): + s.send(b"\r") + s.wait_prompts(2 + i) + s.send(b"echo \xe2\x94\x80ok") + s.drain(1.5) + s.send(b"\x0c") # ^L: readline redraws the whole prompt + s.drain(1.5) + s.send(b"\r") + ok = s.wait_prompts(7) + s.close() + check("every prompt was drawn (5 Enters, a line, ^L)", ok, + "saw %d" % s.prompts()) + writes = max(tty_writes(trace).values(), key=len, default=[]) + stream = b"".join(writes) + rows = stream.count(GREEN.encode()) + check("the trace holds the prompt's last row", rows >= 6, + "found it %d times in %d writes" % (rows, len(writes))) + whole = sum(1 for w in writes if GREEN.encode() + MARK in w) + check("readline's row reaches the tty in one write() each time", + whole >= rows > 0, + "%d of %d rows were whole inside a single write()" % (whole, rows)) + bad = split_atoms(writes) + check("no escape sequence or UTF-8 character is split across write()s", + not bad, "%d split, first: %r" % (len(bad), bad[:4])) + shutil.rmtree(base, ignore_errors=True) + print("%d failed (%d writes)" % (len(FAILS), len(writes))) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/prompt_themes_test.py b/tests/prompt_themes_test.py index d2414d42..114dda9f 100644 --- a/tests/prompt_themes_test.py +++ b/tests/prompt_themes_test.py @@ -123,16 +123,17 @@ def visible_width(s): def render(name, env=None): - """Render one theme the way the shell does. `print -rP` runs the real - engine, so this cannot drift from what a user sees; the percent-doubling - is the same one the switcher's preview does, and for the same reason -- - print -P enters through the zsh frontend, where a bare % is an escape.""" + """Render one theme the way the shell does, and the way the switcher's + preview does: a PROMPT theme through `print -rP`, a PS1 theme through + `${PS1@P}`, which is the PS1 rule itself (a variable's value is not + read again for escapes; print -P's zsh order would), so this cannot + drift from what a user sees.""" script = (". %s\n" % SWITCH + "prompt %s >/dev/null 2>&1\n" % name + 'if [ -n "${PROMPT:-}" ]; then print -rP "$PROMPT"; ' - + 'else print -rP "${PS1//%/%%}"; fi\n') - # print -P strips the \001/\002 width guards by default, because zsh's - # print -P emits none (measured; the parity suite pins it). This test + + "else printf '%s\\n' \"${PS1@P}\"; fi\n") + # print -P and, outside a line editor, @P strip the \001/\002 width + # guards, as zsh's print -P and bash's @P do (measured). This test # exists to SEE those bytes, so it asks for them by name. e = {"HELLISH_THEMES": THEMES, "HELLISH_DBG_PROMPT_MARKS": "1"} if env: diff --git a/tests/ps2_continuation_test.py b/tests/ps2_continuation_test.py new file mode 100644 index 00000000..7a1bffff --- /dev/null +++ b/tests/ps2_continuation_test.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 +"""PS2 is the prompt of every continuation line (issue #134, section 4). + + $ PS2='CUSTOM2> ' + $ echo one \\ + > two # still '> ', not CUSTOM2> + +PS2 was read only for an unfinished compound command (`if`, a function +body, a pipe). The other continuation reads handed readline a fixed +label -- `> ` after a backslash-newline, `dquote> ` / `squote> ` in an +open quote, `subshell> ` / `bquote> ` in an open substitution, +`heredoc> ` for a here-document body -- so a PS2 set in an rc looked +ignored. In bash PS2 is the one continuation prompt; prompt_ps2 now +decides it for every one of those reads. + +What each part pins, at a real prompt: + * with PS2 set, every kind of continuation line shows it, rendered + (a variable and an arithmetic expansion in it), and the command still + runs; a change to the variable shows on the next continuation line; + * `unset PS2` brings the built-in labels back; + * no right prompt on a continuation row: the lexer reads skipped the + reset the compound path did, so RPROMPT was painted after `dquote> `. + +That PS2 is never expanded when no prompt is shown (a script, a pipe) is +graded against bash in tests/scripts/59_ps2_not_expanded.sh. + +Cells: the in-process reader, and HELLISH_RL_FORK=1. Waits on output: a +missing prompt is a timeout and a named failure, not a sleep. + +Usage: python3 ps2_continuation_test.py [/path/to/hellish] +""" +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "helpers")) +from ptyexpect import Tty # noqa: E402 + +ROOT = os.path.dirname(HERE) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] +T = 8.0 + +# name, the lines typed, what the command prints. Every line but the last +# is left open, so each one must be answered by a continuation prompt. +CASES = [ + ("backslash-newline", ["echo B1 \\", "B2"], b"B1 B2\r\n"), + ("double quote", ['echo "D1', 'D2"'], b"D1\r\nD2\r\n"), + ("single quote", ["echo 'S1", "S2'"], b"S1\r\nS2\r\n"), + ("heredoc body", ["cat < "), + ("double quote", b"dquote> "), + ("heredoc body", b"heredoc> "), + ("if", b"if> "), +] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def run(t, lines, cont, out, cell, name): + """Type `lines`; every line but the last must get `cont` back. A miss + is reported and the rest is still typed, so the shell gets back to + its primary prompt and the next case runs. Returns the offsets (first + continuation prompt, output), or None when anything was missed.""" + first = None + shown = True + for i, line in enumerate(lines): + t.send(line + "\r") + if i == len(lines) - 1 or not shown: + continue + shown = t.expect(cont, T) + if first is None: + first = t.pos + check("%s: %s: continuation lines show %r" % (cell, name, cont), shown, + repr(t.since(max(0, len(t.out) - 160)))) + at = t.pos + ok = t.expect(out, T) + check("%s: %s: runs" % (cell, name), ok, + repr(t.since(at)[-160:])) + end = t.pos - len(out) + ok = t.expect(b"P$ ", T) and ok + check("%s: %s: back at the primary prompt" % (cell, name), ok) + return (first, end) if ok and shown else None + + +def ps2_cell(cell, env): + t = Tty(SHELL, rc="PS1='P$ '\nK=c2\nPS2='${K}$((1+1))> '\n", env=env) + try: + if not t.expect(b"P$ ", T): + return check(cell + ": first prompt", False) + for name, lines, out in CASES: + run(t, lines, b"c22> ", out, cell, name) + t.send("K=later\r") + t.expect(b"P$ ", T) + run(t, ['echo "L1', 'L2"'], b"later2> ", b"L1\r\nL2\r\n", cell, + "PS2 re-rendered for each continuation line") + t.send("unset PS2\r") + t.expect(b"P$ ", T) + for name, label in LABELS: + lines, out = next((c[1], c[2]) for c in CASES if c[0] == name) + run(t, lines, label, out, cell, "unset PS2: " + name + + " is " + label.decode().strip()) + finally: + t.close() + + +def rprompt_cell(cell, env): + env = dict(env, TERM="xterm") + t = Tty(SHELL, rc="PS1='P$ '\nRPROMPT=RPX\n", env=env) + try: + if not t.expect(b"P$ ", T): + return check(cell + ": rprompt: first prompt", False) + check(cell + ": rprompt: painted on the primary prompt", + t.expect(b"RPX", T)) + for name, cont in (("double quote", b"dquote> "), + ("backslash-newline", b"> "), + ("heredoc body", b"heredoc> "), + ("if", b"if> ")): + lines, out = next((c[1], c[2]) for c in CASES if c[0] == name) + span = run(t, lines, cont, out, cell, "rprompt: " + name) + if span is None: + continue + row = t.out[span[0]:span[1]] + check("%s: rprompt: none on a %s continuation row" + % (cell, name), b"RPX" not in row, repr(row[-160:])) + finally: + t.close() + + +def main(): + if not os.access(SHELL, os.X_OK): + print("hellish not built at " + SHELL) + return 1 + for cell, env in (("inproc", {}), ("fork", {"HELLISH_RL_FORK": "1"})): + ps2_cell(cell, env) + rprompt_cell(cell, env) + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/rl_inproc_test.py b/tests/rl_inproc_test.py index dcb97616..2a31dbab 100644 --- a/tests/rl_inproc_test.py +++ b/tests/rl_inproc_test.py @@ -389,6 +389,8 @@ def case_completion_sandbox(): def case_history_revert(): s = Session() + # Pins plain Up/Down walking (edit, then Down): prefix search is off. + s.send(b"HELLISH_HISTORY_PREFIX=0\r") s.send(b"echo one\r") s.send(b"echo two\r") s.typed(b"\x1b[A\x1b[AXX") diff --git a/tests/rl_read_race_test.py b/tests/rl_read_race_test.py new file mode 100644 index 00000000..a6e06447 --- /dev/null +++ b/tests/rl_read_race_test.py @@ -0,0 +1,298 @@ +#!/usr/bin/env python3 +"""Regression gate: a ^C between "a key is ready" and read() loses no key. + +prompt_interrupt_test.py failed now and then under load: ^C inside a ^R +search, then `echo ISR_$((1+1))` ran as `cho ISR_2` -- the new line's +first key had gone to the search the ^C was meant to end, five seconds +after that ^C, which had seemingly done nothing. + +The reader (rl_getc.c) waits in pselect, which lets signals in +atomically, and reads the key once pselect reports one. A ^C can be +processed by the line discipline between the two, and then two things +happen in this order: SIGINT is sent -- readline's handler runs before +read() is entered, so no EINTR is coming -- and the input queue is +flushed, taking the key pselect saw with it. A BLOCKING read() then slept +on an empty queue until the next key arrived, and handed that key to the +line the ^C had ended. + +Timing makes it rare; this file makes it certain. A small LD_PRELOAD shim +stalls the shell's read() right there -- after pselect, before the real +read -- for the one key that follows a chosen byte, and says so through a +marker file; the ^C is sent only once the shell is known to be in the +gap. Before the fix every scenario below hangs until the next key and +eats it; after it, the ^C ends the line at once and the next command runs +whole. + + prompt a half-typed line, ^C in the gap, then a command + isearch ^C in the gap inside a ^R search (the reported failure) + +Each runs with the in-process reader and with the forked one +(HELLISH_RL_FORK=1), whose child inherits the same read. + +The fix reads the keys from a second, non-blocking open of the terminal +(rl_keyfd.c), so two things about it are checked too, with no shim: + + hygiene no program the shell runs inherits that descriptor + nodelay fd 0 left O_NONBLOCK by a program is blocking again at the + next command, as bash makes it before every readline() + +The shim scenarios skip (exit 0 for them, says so) where the shim cannot +take effect: no C compiler, or a binary whose own read() comes first +(clang links ASan's interceptors into the executable). CI's pty job +builds with gcc. Linux only: /proc and the line discipline under test. + +Usage: python3 rl_read_race_test.py /path/to/hellish +""" +import os +import pty +import select +import shutil +import subprocess +import sys +import tempfile +import time + +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else "../build/bin/hellish") +FAILS = [] +MARK = b"#> " +STALL = 1.5 +SHIM_C = r""" +#define _GNU_SOURCE +#include +#include +#include +#include +#include + +/* Stall the one 1-byte terminal read that follows a read of the byte + HX_STALL_AFTER, after creating HX_STALL_MARK, for 1.5s (STALL) -- the + whole of it, a signal in the middle included. */ +static ssize_t (*real_read)(int, void *, size_t); +static int armed; + +static void stall(void) +{ + struct timespec ts; + const char *mark; + + armed = 0; + mark = getenv("HX_STALL_MARK"); + if (mark) + close(open(mark, O_WRONLY | O_CREAT, 0644)); + ts.tv_sec = 1; + ts.tv_nsec = 500000000L; + while (nanosleep(&ts, &ts) < 0) + ; +} + +ssize_t read(int fd, void *buf, size_t n) +{ + const char *after; + ssize_t r; + + if (!real_read) + real_read = (ssize_t (*)(int, void *, size_t))dlsym(RTLD_NEXT, + "read"); + if (armed && n == 1 && isatty(fd)) + stall(); + r = real_read(fd, buf, n); + after = getenv("HX_STALL_AFTER"); + if (after && r == 1 && n == 1 && isatty(fd) + && ((unsigned char *)buf)[0] == (unsigned char)atoi(after)) + armed = 1; + return (r); +} +""" + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def build_shim(base): + cc = shutil.which("cc") or shutil.which("gcc") or shutil.which("clang") + if not cc: + return None + src = os.path.join(base, "stall.c") + lib = os.path.join(base, "stall.so") + with open(src, "w") as f: + f.write(SHIM_C) + r = subprocess.run([cc, "-shared", "-fPIC", "-o", lib, src, "-ldl"], + capture_output=True) + return lib if r.returncode == 0 else None + + +class Session: + def __init__(self, home, shim, mark, extra): + env = { + "HOME": home, "PATH": os.environ.get("PATH", "/usr/bin:/bin"), + "TERM": "xterm-256color", "LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", + "PS1": "P#> ", "HISTFILE": os.path.join(home, "hist"), + "HELLISH_BANNER": "0", "HELLISH_NO_BANNER": "1", + "HELLISH_NO_UPDATE_CHECK": "1", "HELLISH_NO_ANIM": "1", + "LD_PRELOAD": shim, "HX_STALL_MARK": mark, + "ASAN_OPTIONS": "detect_leaks=0:verify_asan_link_order=0", + } + env.update(extra) + self.mark = mark + self.buf = bytearray() + self.at = 0 + self.pid, self.fd = pty.fork() + if self.pid == 0: + os.chdir(home) + os.execve(SHELL, [SHELL, "--norc"], env) + os._exit(127) + self.prompt(10.0) + + def pump(self, secs): + r, _, _ = select.select([self.fd], [], [], secs) + if r: + try: + d = os.read(self.fd, 65536) + except OSError: + return + self.buf.extend(d) + + def until(self, pred, cap): + end = time.monotonic() + cap + while not pred() and time.monotonic() < end: + self.pump(0.05) + return pred() + + def prompt(self, cap): + """Wait for a prompt drawn after self.at; move self.at past it.""" + ok = self.until(lambda: MARK in self.buf[self.at:], cap) + if ok: + self.at = self.buf.index(MARK, self.at) + len(MARK) + return ok + + def send(self, data): + os.write(self.fd, data) + + def run(self, line, token, cap=5.0): + start = self.at + self.send(line) + ok = self.until(lambda: token in self.buf[start:], cap) + if ok: + self.at = self.buf.index(token, start) + len(token) + self.prompt(cap) + return ok + + def in_gap(self, cap=5.0): + end = time.monotonic() + cap + while time.monotonic() < end: + if os.path.exists(self.mark): + return True + self.pump(0.02) + return False + + def tail(self): + return bytes(self.buf[-200:]) + + def close(self): + try: + os.kill(self.pid, 9) + except OSError: + pass + try: + os.waitpid(self.pid, 0) + except ChildProcessError: + pass + + +def descriptors(base): + """The second descriptor stays the shell's own, and fd 0 is put back + to blocking before a read, as bash does.""" + home = tempfile.mkdtemp(dir=base) + s = Session(home, "", os.path.join(home, "unused"), {}) + try: + s.run(b"python3 -c 'import os; d = \"/proc/self/fd/\"; print(" + b"\"TTYFD_%s_\" % [f for f in os.listdir(d) if int(f) > 2 and" + b" os.path.realpath(d + f) == os.ttyname(0)])'\r", b"_\r\n") + got = bytes(s.buf[s.buf.rindex(b"TTYFD_"):]).split(b"_\r")[0] + check("hygiene: a program the shell runs sees the terminal on 0-2 only", + got == b"TTYFD_[]", repr(got)) + s.run(b"python3 -c 'import os,fcntl; fcntl.fcntl(0, fcntl.F_SETFL," + b" fcntl.fcntl(0, fcntl.F_GETFL) | os.O_NONBLOCK)';" + b" echo SET_$((2+2))\r", b"SET_4") + s.run(b"python3 -c 'import os,fcntl; print(\"NB_%d\" % bool(fcntl." + b"fcntl(0, fcntl.F_GETFL) & os.O_NONBLOCK))'\r", b"NB_") + after = bytes(s.buf[s.buf.rindex(b"NB_"):][:4]) + check("nodelay: fd 0 left non-blocking is blocking at the next command", + after == b"NB_0", repr(after)) + finally: + s.close() + + +def scenario(label, shim, base, extra, trigger, keys, what): + """`keys` is typed in one write; the shell stalls in the gap before + the key after `trigger`, and ^C is sent while it is there.""" + home = tempfile.mkdtemp(dir=base) + mark = os.path.join(home, "in-gap") + env = dict(extra, HX_STALL_AFTER=str(trigger)) + s = Session(home, shim, mark, env) + try: + s.run(b"echo one_$((0+1))\r", b"one_1") + start = s.at + s.send(keys) + if not s.in_gap(): + return None + t0 = time.monotonic() + s.send(b"\x03") + # the shim holds the shell for STALL; without the fix nothing + # comes after it either, until another key is typed + prompt = s.prompt(STALL + 1.5) + dt = time.monotonic() - t0 + check("%s: %s: ^C in the gap ends the line, no key needed" + % (label, what), prompt, + "no new prompt %.1fs after ^C: %r" % (dt, s.tail())) + check("%s: %s: ^C is echoed once" % (label, what), + s.buf[start:].count(b"^C") == 1, repr(s.tail())) + ok = s.run(b"echo NEXT_$((2+3))\r", b"NEXT_5") + check("%s: %s: the next command runs whole" % (label, what), + ok and b"command not found" not in s.buf[start:], + repr(s.tail())) + check("%s: %s: nothing typed before ^C ran" % (label, what), + b"LEAK_42" not in s.buf[start:], repr(s.tail())) + return True + finally: + s.close() + + +def main(): + if not os.path.exists(SHELL): + print("FAIL shell not found: " + SHELL) + return 1 + if not sys.platform.startswith("linux"): + print("skip: Linux only (/proc, and the line discipline under test)") + return 0 + base = tempfile.mkdtemp(prefix="hellish_rl_read_race_") + descriptors(base) + shim = build_shim(base) + if not shim: + print("skip: no C compiler: the stall shim cannot be built") + return finish(base) + cells = (("in-process", {}), ("forked", {"HELLISH_RL_FORK": "1"})) + for label, extra in cells: + for trigger, keys, what in ( + (ord("x"), b"x; echo LEAK_$((6*7))", "prompt"), + (0x12, b"\x12ech", "isearch")): + r = scenario(label, shim, base, extra, trigger, keys, what) + if r is None: + print("skip: the shell never stalled in the shim -- its own " + "read() comes first (clang ASan?), so the gap cannot " + "be held open") + return finish(base) + return finish(base) + + +def finish(base): + shutil.rmtree(base, ignore_errors=True) + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/scripts/52_heredoc_logical_line.sh b/tests/scripts/52_heredoc_logical_line.sh new file mode 100644 index 00000000..32df25c1 --- /dev/null +++ b/tests/scripts/52_heredoc_logical_line.sh @@ -0,0 +1,188 @@ +# Issue #139: `nvm install 22` printed "syntax error near unexpected token +# `newline'" twice and found no version. nvm quotes a multi-line awk program +# right before a `< sourced_nvm_shape.sh <<'SRC' +g() { + L='v1 x +v2 y' + V="$( { command awk '{ + print $1 + }' \ + | grep v2 \ + | cat; } << EOF +$L +EOF +)" + echo "sourced=[$V]" +} +SRC +. ./sourced_nvm_shape.sh +g +rm -f sourced_nvm_shape.sh +eval "{ echo 'e +val' \\ + | cat \\ + | cat; } << EOF +x +EOF" + +# --- eval and source read their own text, not the caller's -------------------- +eval 'cat </dev/null +eval-body +EOF +echo "after eval" diff --git a/tests/scripts/53_exec_diagnostics.sh b/tests/scripts/53_exec_diagnostics.sh new file mode 100644 index 00000000..da9d172a --- /dev/null +++ b/tests/scripts/53_exec_diagnostics.sh @@ -0,0 +1,68 @@ +# Issue #133: `./life` answered "No such file or directory" -- for a file +# that was there. Running a path that cannot be run said the same thing +# whatever the reason: +# +# * the direct-path check forced errno to ENOENT before printing, so a +# file without +x, a directory, `file/x` (ENOTDIR) and a symlink loop +# (ELOOP) all read "No such file or directory", and the last two came +# back 127 where bash says 126; +# * once execve itself had failed, only EACCES was reported: a script +# whose `#!` interpreter is missing (or a program whose loader is -- +# execve's ENOENT on a file that exists) printed NOTHING and exited 127; +# * a binary execve would not load was handed to /bin/sh, which printed +# its own confusion about "line 1" -- and ran the rest of the file when +# the NUL came late enough. +# +# Each case prints its diagnostic and status on stdout, so the corpus diff +# against bash grades both. What precedes the message is the shell's name +# and `line N:`; that is stripped, so the text after it is what is compared. +# The fixtures live in a directory of their own: verify_alloc.sh runs the +# corpus from tests/scripts itself. + +T=$(mktemp -d) || exit 1 +cd "$T" || exit 1 + +diag() { + "$@" 2>err.txt + rc=$? + sed -e 's|^.*: line [0-9]*: ||' -e "s|^$0: ||" -e "s|$PWD|PWD|g" err.txt + echo "$1 rc=$rc" +} + +mkdir d pbin +printf 'int main(void){return 0;}\n' > notexec +printf '#!/nonexistent/interp\necho ran\n' > nointerp +printf '#!/bin/sh\r\necho ran\r\n' > crlf +printf '#!/\necho ran\n' > dirinterp +printf '\177ELFnot-a-real-header\n' > elfjunk +printf 'ab\000cd\necho ran-after-nul\n' > nulfirst +printf '#!/bin/sh\n\000\necho ran\n' > nulsecond +printf 'echo text-script-ran\n' > noshebang +: > empty +ln -s loop2 loop1 +ln -s loop1 loop2 +chmod +x nointerp crlf dirinterp elfjunk nulfirst nulsecond noshebang empty +cp nointerp pbin/onpath + +diag ./notexec +diag ./d +diag ./nope +diag ./notexec/x +diag ./loop1 +diag ./nointerp +diag ./crlf +diag ./dirinterp +diag ./elfjunk +diag ./nulfirst +diag ./nulsecond +diag ./noshebang +diag ./empty +PATH="$PWD/pbin:$PATH" diag onpath + +# The same failures one level down, where the status is all that is seen. +./notexec/x 2>/dev/null; echo "enotdir $?" +./loop1 2>/dev/null; echo "eloop $?" +(./elfjunk) 2>/dev/null; echo "subshell binary $?" +x=$(./nointerp 2>&1); echo "cmdsub nointerp $? [${x##*: }]" + +cd / && rm -rf "$T" diff --git a/tests/scripts/54_error_line_numbers.sh b/tests/scripts/54_error_line_numbers.sh new file mode 100644 index 00000000..1c9d6b14 --- /dev/null +++ b/tests/scripts/54_error_line_numbers.sh @@ -0,0 +1,78 @@ +# A runtime error names the line it happened on -- and inside a sourced +# file, that file and its line. +# +# ctx, the "script: line N:" every runtime error starts with, was set only +# by the READER. Input arrives in batches -- a whole small script at once +# -- so by the time a command ran, the reader stood past the batch and every +# error named the line after the last one: +# +# printf 'true\n./nope\n' > t.sh; hellish t.sh +# t.sh: line 3: ./nope: No such file or directory (bash: line 2) +# +# Inside a sourced file the error named the sourcing script, at a line of +# its own, and $LINENO was the `.` line's. Bash names the sourced file. +# +# Errors are folded into stdout here, so the corpus diff grades their text; +# only the script's own path is replaced, since the corpus hands the same +# path to both shells anyway. Missing commands are ./paths: their message +# is the same everywhere, unlike a PATH miss's. + +T=$(mktemp -d) || exit 1 +say() { sed "s|$0|SCRIPT|; s|$T|T|g"; } + +{ + true + ./no-such-1 + echo "LINENO=$LINENO" +} 2>&1 | say + +./no-such-2 2>&1 | say + +cat > "$T/inc.sh" <<'EOF' +echo "inc LINENO=$LINENO" +./no-such-in-inc + +f_in_inc() { echo "f_in_inc LINENO=$LINENO"; } +for i in 1 2; do + ./no-such-in-loop-$i +done +echo "inc end LINENO=$LINENO" +EOF + +cat > "$T/outer.sh" <<'EOF' +./no-such-before-nested +. "$T_DIR/inc.sh" +./no-such-after-nested +EOF + +T_DIR=$T +. "$T/inc.sh" 2>&1 | say +./no-such-after-source 2>&1 | say +. "$T/outer.sh" 2>&1 | say +./no-such-after-outer 2>&1 | say + +# eval inside a sourced file reports from the eval's line in the file +cat > "$T/ev.sh" <<'EOF' +true +eval './no-such-in-eval' +EOF +. "$T/ev.sh" 2>&1 | say + +# a heredoc before the error does not shift the count +cat > "$T/hd.sh" <<'EOF' +cat <&1 | say + +# many lines of a big script: the count comes from the start, not a cache +i=0 +while [ "$i" -lt 3 ]; do + i=$((i + 1)) + ./no-such-in-while-$i 2>&1 | say +done +echo "after while LINENO=$LINENO" + +rm -rf "$T" diff --git a/tests/scripts/55_exec_builtin.sh b/tests/scripts/55_exec_builtin.sh new file mode 100644 index 00000000..9d26e8e2 --- /dev/null +++ b/tests/scripts/55_exec_builtin.sh @@ -0,0 +1,78 @@ +# `exec NAME` for a NAME that is not there used find_cmd_path, the lookup +# a forked child makes just before it exits: on a miss that prints +# "command not found" and frees the whole shell. exec runs in the shell +# itself, so it went on to print its own message through the freed state +# (a heap-use-after-free under ASan, garbage or a crash without it), and +# a subshell's EXIT trap ran on freed memory too. +# +# bash's rules, which each case below checks against it: +# * a NAME found nowhere: "exec: NAME: not found", status 127; +# * a file execve refuses: "PATH: ", 126 -- 127 when it is not +# there -- a directory reads "Is a directory"; PATH is the file's full +# name, a relative one joined to the working directory less one +# leading "./" (bash's full_pathname); +# * a file on PATH without execute permission is still the one picked, +# and fails "Permission denied"; a directory on PATH is skipped; +# * `--` ends exec's options; +# * a shell that is not interactive (this script, any subshell) exits +# with that status after running its EXIT trap; nothing after it runs. +# +# Each case prints its diagnostic and status on stdout, so the corpus diff +# grades both. The shell's name and `line N:` before a message are +# stripped. Paths are absolute and the fixture directory is printed as T. +# The fixtures live in a directory of their own: verify_alloc.sh runs the +# corpus from tests/scripts itself. + +T=$(mktemp -d) || exit 1 +cd "$T" || exit 1 + +run() { + ( eval "$1" ) 2>err.txt + rc=$? + sed -e 's|^.*: line [0-9]*: ||' -e "s|^$0: ||" -e "s|$T|T|g" err.txt + echo "[$1] rc=$rc" +} + +mkdir d pdir pdir/dircmd +printf 'echo ran\n' > notexec +printf '#!/bin/sh\necho "noxcmd ran"\n' > pdir/noxcmd +printf '#!/bin/sh\necho "args:$*"\n' > pdir/showargs +chmod +x pdir/showargs + +run 'exec nosuch_cmd_q' +run 'exec -- nosuch_cmd_q' +run 'exec nosuch_cmd_q out.txt' +run 'exec nosuch_cmd_q; echo not-reached' +run 'exec "$T/notexec"' +run 'exec "$T/d"' +run 'exec "$T/nope"' +run 'exec "$T/notexec/x"' +run 'exec ./notexec' +run 'exec ./d' +run 'exec nope/../notexec' +run 'exec .//notexec' +run 'cd d && exec ../notexec' +run 'PATH=pdir; exec noxcmd' +run 'PATH=.; cd pdir && exec noxcmd' +run 'PATH="$T/pdir:$PATH"; exec noxcmd' +run 'PATH="$T/pdir:$PATH"; exec dircmd' +run 'PATH="$T/pdir:$PATH"; exec showargs a "b c"' +run 'PATH="$T/pdir:$PATH"; exec -- showargs -- x' +run 'exec echo via-exec' +run 'exec -- echo dashdash' +run 'exec --' +run 'exec 3>fd3.txt; echo kept >&3' +cat fd3.txt +run 'trap "echo exit-trap-ran" EXIT; exec nosuch_cmd_q; echo not-reached' +run 'f() { exec nosuch_cmd_q; echo after-f; }; f; echo after-call' +run 'exec nosuch_cmd_q | cat; echo after-pipeline' + +x=$(exec nosuch_cmd_q 2>/dev/null; echo after) +echo "cmdsub [$x] rc=$?" +( exec nosuch_cmd_q ) 2>/dev/null | cat +echo "pipeline rc=$?" + +cd / && rm -rf "$T" +trap 'echo final-exit-trap' EXIT +exec nosuch_cmd_q 2>/dev/null +echo "not reached: a script ends at a failed exec" diff --git a/tests/scripts/56_quoted_at_no_glob.sh b/tests/scripts/56_quoted_at_no_glob.sh new file mode 100644 index 00000000..383fa84f --- /dev/null +++ b/tests/scripts/56_quoted_at_no_glob.sh @@ -0,0 +1,70 @@ +# A quoted "$@" or "${a[@]}" expands to one field per element, and quoting +# means no pathname expansion: an element that is `*` stays `*`. +# +# hellish globbed every element, because the fields were built as unquoted +# expansion tokens. `f() { printf '[%s]' "$@"; }; f '*'` printed every file +# in the directory, and any wrapper of the form `rm -- "$@"` deleted them. +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT +cd "$work" || exit 1 +: > aa; : > ab; : > b1; mkdir dd + +show() { printf '[%s]' "$@"; echo; } + +echo "-- positionals" +set -- '*' 'a?' '[ab]*' 'dd/*' '' 'x y' +show "$@" +show "${@}" +show "${@:1}" +show "${@:2:2}" +for i in "$@"; do printf '<%s>' "$i"; done; echo +show pre"$@"post +show "x$@" +show "$@y" +count() { echo "$#"; } +count "$@" + +echo "-- a function's own arguments" +fwd() { show "$@"; } +fwd '*' '?b' +wrap() { local a="$1"; shift; show "$a" "$@"; } +wrap '*' '*' 'a*' + +echo "-- arrays" +a=('*' 'a?' '' '[ab]*') +show "${a[@]}" +for e in "${a[@]}"; do printf '<%s>' "$e"; done; echo +show "p${a[@]}s" + +echo "-- associative arrays" +declare -A h +h['*']='a*' +show "${h[@]}" +show "${!h[@]}" + +echo "-- still unquoted, still globbed" +set -- '*' 'b?' +show $@ +show ${a[0]} +b=('a*') +show ${b[@]} + +echo "-- set -f turns globbing off for both" +set -f +show $@ +set +f + +echo "-- a quoted element next to an unquoted glob: only the glob matches" +: > 'a*z' +set -- a '*' 'a*' +show "$@"* +show *"$@" +show "${@:2}"* +c=(a 'a*' b) +show "${c[@]}"* +rm -f 'a*z' + +echo "-- the destructive case" +del() { rm -f -- "$@"; } +del '*' +ls diff --git a/tests/scripts/58_read_eof_readonly.sh b/tests/scripts/58_read_eof_readonly.sh new file mode 100644 index 00000000..98e87c4f --- /dev/null +++ b/tests/scripts/58_read_eof_readonly.sh @@ -0,0 +1,53 @@ +# read assigns its variables even at end of input, and never a readonly one. +# +# At EOF with nothing read, hellish returned 1 without touching the +# variables, so they kept their old values. That made the common loop +# while IFS= read -r line || [ -n "$line" ]; do ...; done +# (which also takes a last line with no newline) run forever: $line still +# held the last line. bash assigns them empty. +# +# And read wrote readonly variables: `readonly r=1; echo x | read r` +# changed r. bash refuses with "r: readonly variable" and a status. +msg() { sed 's/^.*: line [0-9]*: //'; } + +echo "-- EOF with nothing read empties the variables" +x=old; y=old; REPLY=old; arr=(o l d) +printf '' | { read -r x y; echo "rc=$? [${x-unset}] [${y-unset}]"; } +printf '' | { read -r; echo "rc=$? [${REPLY-unset}]"; } +printf '' | { read -r -a arr; echo "rc=$? n=${#arr[@]}"; } +printf 'a\n' | { read -r x; read -r x; echo "rc=$? [$x]"; } +printf '' | { read -r -d : x; echo "rc=$? [$x]"; } +printf '' | { read -r -n 3 x; echo "rc=$? [$x]"; } +printf '' | { read -r -N 3 x; echo "rc=$? [$x]"; } + +echo "-- a last line with no newline, then EOF" +printf 'last' | { read -r x y; echo "rc=$? [$x] [$y]"; } +printf 'l1\nl2' | { + n=0 + while IFS= read -r line || [ -n "$line" ]; do + n=$((n + 1)); echo "got [$line]" + [ "$n" -gt 5 ] && { echo "runaway loop"; break; } + done +} +printf 'k=v\nlast=1' | { + n=0 + while IFS= read -r line || [ -n "$line" ]; do + n=$((n + 1)); echo "line: $line" + [ "$n" -gt 5 ] && { echo "runaway loop"; break; } + done + echo "done, line=[$line]" +} + +echo "-- a readonly variable is refused, the others follow bash" +readonly ro=keep +echo x | { read -r ro 2>&1 | msg; } +echo x | { read -r ro 2>/dev/null; echo "only var: rc=$? [$ro]"; } +printf '' | { read -r ro 2>/dev/null; echo "only var at EOF: rc=$? [$ro]"; } +echo "a b" | { read -r ro y 2>/dev/null; echo "first of two: rc=$? [$ro] [${y-unset}]"; } +echo "a b" | { read -r y ro 2>/dev/null; echo "last of two: rc=$? [$ro] [$y]"; } +echo "a b c" | { y=; z=old; read -r y ro z 2>/dev/null; echo "middle: rc=$? [$ro] [$y] [$z]"; } +echo a | { readonly REPLY; read -r 2>/dev/null; echo "REPLY: rc=$?"; } +ra=(1 2); readonly ra +echo "p q" | { read -r -a ra 2>&1 | msg; } +echo "p q" | { read -r -a ra 2>/dev/null; echo "array: rc=$? [${ra[*]}]"; } +echo x | { read -r ro 2>/dev/null; echo "the shell goes on"; } diff --git a/tests/scripts/59_ps2_not_expanded.sh b/tests/scripts/59_ps2_not_expanded.sh new file mode 100644 index 00000000..c6fb79d0 --- /dev/null +++ b/tests/scripts/59_ps2_not_expanded.sh @@ -0,0 +1,48 @@ +# PS2 is the prompt for a continuation line, and a shell that is not +# interactive shows no prompt, so it never expands PS2. hellish renders +# PS2 for every kind of continuation line (an open quote, a backslash- +# newline, a heredoc body, a command substitution, an unfinished +# compound); this pins that it still does so only at a prompt. The probe +# is a PS2 with a side effect: each render would count in $n. +PS2='$((n+=1))> ' +echo "a +b" +echo 'c +d' +echo one \ +two +cat < "$d/src" +. "$d/src" +rm -rf "$d" +echo "n=${n-unset}" diff --git a/tests/scripts/60_epochrealtime_srandom.sh b/tests/scripts/60_epochrealtime_srandom.sh new file mode 100644 index 00000000..19656d2c --- /dev/null +++ b/tests/scripts/60_epochrealtime_srandom.sh @@ -0,0 +1,74 @@ +# $EPOCHREALTIME and $SRANDOM (bash 5.1). Their values change on every +# read, so this prints properties, never the values: the format, where +# they sit relative to $EPOCHSECONDS, that they move, and that $SRANDOM +# has 32 bits and cannot be seeded through $RANDOM. + +echo "set: ${EPOCHREALTIME:+yes} ${SRANDOM:+yes}" + +# EPOCHREALTIME: seconds, a '.', exactly six digits of microseconds. +r=$EPOCHREALTIME +case $r in + *[!0-9.]* | *.*.* | .* | *.) echo "realtime: bad format" ;; + *.[0-9][0-9][0-9][0-9][0-9][0-9]) echo "realtime: sec.usec" ;; + *) echo "realtime: bad format" ;; +esac + +# Its seconds are EPOCHSECONDS read just before and just after. +s1=$EPOCHSECONDS +r=$EPOCHREALTIME +s2=$EPOCHSECONDS +if [ "${r%.*}" -ge "$s1" ] && [ "${r%.*}" -le "$s2" ]; then + echo "realtime: agrees with EPOCHSECONDS" +fi + +# It moves forward, in microseconds. +us() { echo $(( ${1%.*} * 1000000 + 10#${1#*.} )); } +a=$EPOCHREALTIME +sleep 0.05 +b=$EPOCHREALTIME +d=$(( $(us "$b") - $(us "$a") )) +if [ "$d" -ge 40000 ] && [ "$d" -lt 5000000 ]; then + echo "realtime: moved about 50ms" +else + echo "realtime: moved $d us" +fi + +# SRANDOM: a decimal number in 0 .. 2^32-1, new on every read. +bad=0 big=0 same=0 i=0 prev= +while [ "$i" -lt 16 ]; do + v=$SRANDOM + case $v in '' | *[!0-9]*) bad=1 ;; esac + [ "${#v}" -le 10 ] && [ "$v" -le 4294967295 ] || bad=1 + [ "$v" -gt 32767 ] && big=$((big + 1)) + [ "$v" = "$prev" ] && same=$((same + 1)) + prev=$v + i=$((i + 1)) +done +echo "srandom: bad=$bad repeats=$same wider-than-RANDOM=$([ "$big" -gt 0 ] && echo yes)" + +# Seeding $RANDOM never makes $SRANDOM repeat. (Whether RANDOM=42 reseeds +# $RANDOM itself is not asserted: hellish does not, a documented +# divergence in expand2.c.) +RANDOM=42; x1=$SRANDOM +RANDOM=42; x2=$SRANDOM +[ "$x1" != "$x2" ] && echo "RANDOM=42 does not repeat SRANDOM" + +# A subshell and a command substitution draw fresh values. +p=$SRANDOM +c=$(echo "$SRANDOM") +[ "$p" != "$c" ] && echo "srandom: fresh in a command substitution" + +# Arithmetic reads them like any variable. +echo "arith: $(( SRANDOM >= 0 && SRANDOM <= 4294967295 ))" +echo "arith: $(( ${EPOCHREALTIME%.*} >= s1 ))" + +# Assigning does not make them constant. +SRANDOM=7 +t=0; for i in 1 2 3; do [ "$SRANDOM" = 7 ] && t=$((t + 1)); done +echo "SRANDOM=7 ignored: $([ "$t" -lt 3 ] && echo yes)" +EPOCHREALTIME=1.000000 +case $EPOCHREALTIME in 1.000000) echo "EPOCHREALTIME=1 stuck" ;; + *) echo "EPOCHREALTIME=1 ignored" ;; esac + +# set -u: both are always set. +( set -u; : "$EPOCHREALTIME" "$SRANDOM"; echo "nounset: ok" ) diff --git a/tests/scripts/61_prompt_expand_P.sh b/tests/scripts/61_prompt_expand_P.sh new file mode 100644 index 00000000..3092dd48 --- /dev/null +++ b/tests/scripts/61_prompt_expand_P.sh @@ -0,0 +1,49 @@ +# ${var@P}: the value rendered as a prompt string (bash 4.4). This is the +# part hellish and bash agree on byte for byte -- the escapes both have, +# parameter and arithmetic expansion, and that what a variable expanded +# to is NOT read again for escapes. hellish's own additions (the zsh `%` +# escapes of its bilingual PS1, \A, \g) and `${PS1@P}` being exactly the +# live prompt are pinned at a real prompt, tests/prompt_expand_p_test.py. +d=$(mktemp -d) +cd "$d" || exit 1 +mkdir leaf && cd leaf || exit 1 +HOME=$d + +show() { for a; do printf '<%s>\n' "$a"; done; } + +P='\w \W \$ \\ \n.' +show "${P@P}" +P='\u' Q='\h' R='\H' +[ "${P@P}" = "$(id -un)" ] && echo "\\u is the user" +[ "${Q@P}" = "$(hostname | cut -d. -f1)" ] && echo "\\h is the host" +[ "${R@P}" = "$(hostname)" ] && echo "\\H is the full host" + +# \[ \] are readline's markers; with no line editor they are nothing. +P='a\[b\]c\e[0m' +show "${P@P}" | od -An -c | tr -s ' ' + +# \nnn octal, \D{fmt}. +P='\101\060 \D{%Y}' +[ "${P@P}" = "A0 $(date +%Y)" ] && echo "octal and \\D" + +# Expansions run after the escapes, and what they produce stays as it is. +V='100%done \w %d \$HOME $HOME' +P='[${V}] [$V] [$((6 * 7))] [${V%% *}] [${nosuch-unset}]' +show "${P@P}" + +# An empty and an unset variable both render as nothing. +E= +show "${E@P}" "${nosuch@P}" + +# Positionals and a function's arguments. +set -- '\W' '$((1+1))' +show "${1@P}" "${2@P}" +f() { show "${1@P}"; } +f 'in \W' + +# In double quotes, in an assignment, as part of a word. +P='\W' +x=${P@P} +show "$x" "pre${P@P}post" + +cd / && rm -rf "$d" diff --git a/tests/scripts/62_export_unset_attribute.sh b/tests/scripts/62_export_unset_attribute.sh new file mode 100644 index 00000000..a9108224 --- /dev/null +++ b/tests/scripts/62_export_unset_attribute.sh @@ -0,0 +1,88 @@ +# `export NAME` before NAME has a value. POSIX: the export attribute is +# set, and "subsequent" assignments are exported. Until one comes, NAME +# is still unset: no child sees it, ${NAME+x} is empty, `set` leaves it +# out, and `export -p` / `declare -p` list it without a value. + +# The idiom itself. +export A +A=1 +sh -c 'echo "A in child: [$A]"' + +# Before it is assigned: unset everywhere, listed bare. +export B +echo "B set: [${B+yes}]" +env | grep -c '^B=' || true +export -p | grep -E '^export B($|=)' +declare -p B +set | grep -c '^B=' || true +( set -u; : "$B" ) 2>/dev/null || echo "B unbound under set -u" + +# Any way of assigning it exports it. +export C D E F +read -r C </dev/null || E=from-printf +F+=appended +sh -c 'echo "C=[$C] D=[$D] E=[$E] F=[$F]"' + +# declare -x does the same. +declare -x G +G=2 +sh -c 'echo "G in child: [$G]"' +declare -p G + +# unset takes the attribute away with the variable. +export H +H=3 +unset H +H=4 +sh -c 'echo "H after unset: [${H-unset}]"' + +# export -n and declare +x take it away and leave the name unset. +export I +export -n I +I=5 +sh -c 'echo "I after export -n: [${I-unset}]"' +export J +declare +x J +J=6 +sh -c 'echo "J after declare +x: [${J-unset}]"' + +# export -p output can be fed back to a shell and means the same thing. +export K +L=val +export L +export -p | grep -E '^export (K|L)($|=)' > "${TMPDIR:-/tmp}/exp_$$" +unset K L +. "${TMPDIR:-/tmp}/exp_$$" +rm -f "${TMPDIR:-/tmp}/exp_$$" +echo "K set: [${K+yes}] L=[$L]" +K=k +sh -c 'echo "K=[$K] L=[$L]"' + +# Exporting a name that is already set changes nothing but the attribute. +M=keep +export M +sh -c 'echo "M=[$M]"' + +# cd with PWD exported but unset: OLDPWD takes PWD's state, attribute +# included, and PWD is set again. +( + unset PWD + export PWD + cd / + echo "OLDPWD set: [${OLDPWD+yes}]" + export -p | grep -E '^export OLDPWD($|=)' + echo "PWD=[$PWD]" +) + +# A readonly name exported before it has a value stays readonly. +( + export N + readonly N + declare -p N + N=1 +) 2>/dev/null +echo "readonly N: status $?" diff --git a/tests/source_diag_test.py b/tests/source_diag_test.py index 06dc5b66..cbbe9285 100644 --- a/tests/source_diag_test.py +++ b/tests/source_diag_test.py @@ -156,14 +156,18 @@ def main(): ("%s: line 4:" % late) in err, "err=%r" % err) check("chunks/still one report", n_errors(err) == 1, "err=%r" % err) - # ---- eval inside a sourced file reports against ITS text + # ---- eval inside a sourced file: the eval's line in the file. + # The eval's text is not the file's, so its own line count must + # not be added to the file's -- but bash still names the file and + # the line the eval is on ("F: eval: line 2: ..."). This used to + # say "hellish: line 1", the -c string's line, naming neither. ev = os.path.join(d, "evalin.sh") write(ev, "echo l1\neval 'echo (x)'\necho l3\n") rc, out, err = run([SHELL, "-c", "source " + ev], d) check("eval/the file goes on after a failed eval", out == "l1\nl3\n", "out=%r" % out) - check("eval/not blamed on the file's line", - ("evalin.sh: line" not in err) and n_errors(err) == 1, + check("eval/the file and the eval's own line, once", + ("%s: line 2:" % ev) in err and n_errors(err) == 1, "err=%r" % err) # ---- unterminated group: bash's line, once diff --git a/tests/tester b/tests/tester index 1531c1f8..ffea6c3a 100755 --- a/tests/tester +++ b/tests/tester @@ -48,6 +48,7 @@ if [[ ${#test_lists[@]} -eq 0 ]]; then "issue17_signal_report" "issue16_net_redir" "redir_multi_fd" + "io_number" "heredoc_fd" "func_def_redirect" "dquote_brace_nest" @@ -81,6 +82,8 @@ if [[ ${#test_lists[@]} -eq 0 ]]; then "patsub_quoting" "cmdsub_jobs" "compopt" + "chunk_status" + "bind_builtin" ) fi diff --git a/tests/zsh_nested_op_test.py b/tests/zsh_nested_op_test.py new file mode 100644 index 00000000..ae6b5be6 --- /dev/null +++ b/tests/zsh_nested_op_test.py @@ -0,0 +1,119 @@ +#!/usr/bin/env python3 +"""zsh `aliases`, flags on subscripts, and a subscript + operator after a +nested expansion -- the three expansion gaps behind issue #137. + +oh-my-zsh's sudo widget (on ESC ESC) asks "what command does this alias +run" as `"${${(Az)aliases[$cmd]}[1]:-$cmd}"`. On hellish that was a bad +substitution three ways over: + + * `aliases` read empty -- its WRITES reached the alias table (#114), + its reads fell through to a plain, empty variable; + * a flag on a subscripted name, `${(z)aliases[ll]}` or `${(U)e[2]}`, + reached the scalar engine, which has no subscript; + * a subscript WITH an operator after a nested expansion, `${${..}[1]:-x}`, + did the same -- and when it did get through, the nested value had been + joined first, so `[1]` took a character instead of a word. + +Every expectation below is what zsh 5.8 printed (zsh -f, with `unalias -a` +first: zsh -f still predefines run-help and which-command). They are +asserted here so the check runs where no zsh is installed -- CI's default +-- and when a zsh oracle IS found (ZSH_ORACLE, ~/zsh-5.9, or PATH), each +case is also run under it, so an expectation cannot drift from zsh. + +Only double-quoted forms and forms whose result is one word are pinned: +hellish's zsh dialect still field-splits unquoted expansions (zsh does not +without SH_WORD_SPLIT), which is a separate, dialect-wide difference. + +Usage: python3 zsh_nested_op_test.py [/path/to/hellish] +""" +import os +import shutil +import subprocess +import sys + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SHELL = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 + else os.path.join(ROOT, "build", "bin", "hellish")) +FAILS = [] + +PRE = ("unalias -a; alias ll='ls -l' gs='git status'; cmd=ll; c2=vim; " + "e=(p 'q r' s); x='a b c'; ") + +# label, script (after PRE), what zsh 5.8 printed +CASES = [ + ("aliases/read", r"""printf '[%s]' "${aliases[ll]}" """, "[ls -l]"), + ("aliases/read-unset", r"""printf '[%s]' "${aliases[nope]}" """, "[]"), + ("aliases/count", r"""printf '[%s]' ${#aliases}""", "[2]"), + ("aliases/keys", r"""printf '[%s]' ${(ok)aliases}""", "[gs][ll]"), + ("aliases/values", r"""printf '[%s]' "${(@ov)aliases}" """, + "[git status][ls -l]"), + ("aliases/defined-since", r"""alias zz=echo; printf '[%s]' "${aliases[zz]}" """, + "[echo]"), + ("aliases/gone-after-unalias", + r"""unalias ll; printf '[%s]' "${aliases[ll]:-gone}" """, "[gone]"), + ("widget/alias", r"""printf '[%s]' "${${(Az)aliases[$cmd]}[1]:-$cmd}" """, + "[ls]"), + ("widget/not-an-alias", + r"""printf '[%s]' "${${(Az)aliases[$c2]}[1]:-$c2}" """, "[vim]"), + ("widget/buffer-first-word", + r"""BUFFER='sudo -e f'; printf '[%s]' "${${(Az)BUFFER}[1]}" """, + "[sudo]"), + ("flag/on-array-element", r"""printf '[%s]' "${(U)e[2]}" """, "[Q R]"), + ("flag/on-alias-element", r"""printf '[%s]' "${(U)aliases[gs]}" """, + "[GIT STATUS]"), + ("nested/word-then-default", r"""printf '[%s]' "${${(z)x}[2]:-d}" """, + "[b]"), + ("nested/past-the-end", r"""printf '[%s]' "${${(z)x}[5]:-dflt}" """, + "[dflt]"), + ("nested/alternate", r"""printf '[%s]' "${${(z)x}[2]:+set}" """, + "[set]"), + ("nested/scalar-char", r"""printf '[%s]' "${${(U)x}[1]:-d}" """, "[A]"), + ("nested/quoted-array-joins", + r"""printf '[%s]' "${${(U)e}[2]:-d}" """, "[ ]"), + ("nested/unflagged", r"""printf '[%s]' "${${x}[1]:-d}" """, "[a]"), + ("nested/trim-after-subscript", + r"""printf '[%s]' "${${(z)x}[3]#c}" """, "[]"), +] + + +def check(name, ok, detail=""): + print(("ok " if ok else "FAIL ") + name + ("" if ok else " " + detail)) + if not ok: + FAILS.append(name) + + +def find_zsh(): + env = os.environ.get("ZSH_ORACLE") + if env and os.path.exists(env): + return env + home = os.path.expanduser("~/zsh-5.9/bin/zsh") + for c in (home, shutil.which("zsh"), "/usr/bin/zsh", "/bin/zsh"): + if c and os.path.exists(c): + return c + return None + + +def run(argv, script): + p = subprocess.run(argv + [script], capture_output=True, timeout=10, + env=dict(os.environ, HELLISH_NO_BANNER="1", + HELLISH_NO_UPDATE_CHECK="1")) + return p.stdout.decode(errors="replace"), p.stderr.decode(errors="replace") + + +def main(): + zsh = find_zsh() + for name, script, want in CASES: + out, err = run([SHELL, "-c"], "set -o zsh\n" + PRE + script) + check(name, out == want and "bad substitution" not in err, + "got %r want %r err=%r" % (out, want, err.strip()[:120])) + if zsh: + zout, _ = run([zsh, "-f", "-c"], PRE + script) + check("oracle/" + name, zout == want, + "zsh printed %r, the pinned expectation is %r" % (zout, want)) + if not zsh: + print("SKIP oracle/* (no zsh; set ZSH_ORACLE to cross-check)") + print("%d failed" % len(FAILS)) + return 1 if FAILS else 0 + + +sys.exit(main()) diff --git a/tests/zsh_procsub_test.py b/tests/zsh_procsub_test.py index 6bec051a..6f40104f 100644 --- a/tests/zsh_procsub_test.py +++ b/tests/zsh_procsub_test.py @@ -118,6 +118,17 @@ def temp_file_cases(): check("temp: path was printed", bool(re.match(r"^/.*", path)), True) if path: check("temp: cleaned up after exit", os.path.exists(path), False) + # A file left in /tmp by another process must never block a new one. + # The name used to be /tmp/hsh-, opened O_EXCL: a leftover + # from an earlier process that had the same pid (a =(cmd) made inside + # $( ) outlives its subshell) failed the open, the word vanished, and + # `cat` read its stdin instead -- no error, status 0. It surfaced as + # zsh_glob_test's eqsub/reads-back failing about once in 150 runs. + out, _ = run_zsh(': > /tmp/hsh$$-0; : > /tmp/hsh$$-1\n' + 'cat =(echo hi) `, `heredoc> `, `if> `). - **ANSI- and multibyte-aware prompt** rendering, so colorful prompts and wide characters don't desync the cursor. - **Ctrl-C / Ctrl-D / signal handling** behaves like a real login shell, including inside heredocs. @@ -97,19 +99,35 @@ The fish/zsh-grade layer (menus, descriptions, fuzzy matching) is on the roadmap **The default is zsh's own**: `hostname% `, plus the self-spacing `⬆` update badge — a first prompt that looks like the shell you already know. The rich two-row theme is one command away: `prompt` lists 29 themes, `prompt ` switches, `prompt save ` persists, and -`PS1='\B'` is the old default by name. +`PS1='\B'` is the old default by name. `prompt` is a shell function, not a builtin: the installer +seeds it into `~/.config/hellish/rc.d/40-prompt-switch.hsh`, so `type prompt` says whether it is +loaded, and a shell run from a checkout loads it with +`HELLISH_THEMES=share/themes; . share/rc.d/40-prompt-switch.hsh`. **PS1 is bilingual.** Both escape languages render in a plain PS1 — paste `%n@%m %~ %#` or `\u@\h \w \$` and either works, while `100% `, csh-style `%> ` and the strftime percents inside `$(date +%H:%M)` or `\D{%M}` survive literally. `PROMPT` and `print -P` keep exact zsh semantics (measured against zsh 5.9), and under `set -o zsh`, PS1 *is* PROMPT, as in zsh. +**What is live, and what is not.** `$VAR`, `${VAR…}`, `$?` and `$((…))` are expanded every time +the prompt is drawn. Command substitution is not: `$(…)` and backquotes stay on screen as the +text you wrote, because they would fork on every redraw. A computed segment is a variable set by +a `HELLISH_PRECMD_FUNCS` hook (below), which runs once per prompt. + Bash's escape set is implemented — `\u \h \H \w \W \t \d \D{fmt} \T \@ \! \# \j \l \r \s \v \V \n \e \a \$ \\ \[ \] \nnn` — plus hellish's own: `\g` git branch, `\S` failure badge, `\p` duration, `\J` jobs, `\U` pending update, `\B` the built-in prompt, `\I` the file being sourced. +`${PS1@P}` prints what PS1 renders to. It is bash's prompt-expansion operator, and here it runs +the PS1 renderer itself, both escape languages included, so it cannot disagree with the prompt: +`prompt preview` draws PS1 themes with it. (`print -P` is zsh's, and like zsh's it reads a +variable's value for `%` escapes again; a PS1 does not.) + **`\A` is the one deliberate divergence.** In bash it is the 24-hour clock; in hellish it is the -animation frame, and it shipped first. `\D{%H:%M}` gives you bash's meaning. +animation frame, and it shipped first. `\D{%H:%M}` gives you bash's meaning. A bash PS1 pasted in +with `\A` shows the glyph, or nothing while the animation is off, which is the default: the idle +repaint that animates it has been reported to corrupt the prompt on some terminals and could not +be reproduced on demand (`hellishrc.example`, "ANIMATION"). Two hook arrays run around every command — `HELLISH_PRECMD_FUNCS` before each prompt, `HELLISH_PREEXEC_FUNCS` before the typed line, with the line as `$1`. They are arrays of function diff --git a/wiki/manual.md b/wiki/manual.md index 310e115b..963ad359 100644 --- a/wiki/manual.md +++ b/wiki/manual.md @@ -117,6 +117,31 @@ git's own `git-completion.bash` works. The default stays off where bash has it on, for a measured reason recorded in the [interactive page](interactive.md#completion-). +### make targets + +`make ` / `make cl` completes target names from the Makefile in the +current directory (also `gmake`, `/usr/bin/make`, and behind `sudo`, `env X=1`, +`&&`, `(`; a wrapper option that takes an argument, as in `sudo -u root make`, +falls back to filenames). The file is parsed in the shell itself; make is never run. + +- Files read: the first regular file of `GNUmakefile`, `makefile`, `Makefile` + (GNU make's order, symlinks followed), at most 4 MiB. No such file, a + directory, or an unreadable file means plain filename completion. +- Offered: the names on `target:` / `a b c:` / `a::` lines. Not offered: + variable assignments, directives, recipe lines, `define` blocks, names with + `%` or `$`, names starting with `.`; text after the colon is ignored. +- `include` / `-include` / `sinclude` of literal file names are followed ONE + level deep (what an included file includes is ignored). Changing only an + included file does not refresh the cache; touch the Makefile. +- Cache: the targets are reused while the Makefile's name, inode, mtime and + size are unchanged (one `stat` per TAB). +- A word starting with `-` or containing `=` or `/` keeps filename completion, + and so does any `make` line with `-C`, `--directory`, `-f`, `--file` or + `--makefile` before the cursor, or a prefix that matches no target + (`make out.txt`). +- A `complete` spec for `make` (under `shopt -s progcomp`) wins over all of + this; `complete -r make` brings the targets back. + ## JOB CONTROL `&`, `jobs`, `fg`, `bg`, `wait`, `kill` with job specs, `$!`, Ctrl-Z, and @@ -147,6 +172,10 @@ inside the shell: `help` for the grouped list, `help NAME` for one, | variable | effect | |---|---| | `HELLISH_BANNER=0\|1` | force the welcome panel off / on | +| `HELLISH_HISTORY_PREFIX=0\|off` | Up/Down walk plain history; default on = search by the text before the cursor; read at every prompt | +| `HELLISH_AUTOSUGGEST=1\|on` | inline suggestion (off by default): the newest history line starting with the typed text, dim after the cursor; Right / End / Ctrl-F insert it; never a line with control characters or a newline; read at every prompt | +| `HELLISH_SUGGEST_MAX=N` | history lines the suggestion scans, newest first (default 1000; non-numeric or below 1 = default); read at every prompt | +| `POSTDISPLAY` | zle widgets only: the text painted after the cursor, i.e. the suggestion; assign it to show your own until the buffer next changes | | `HELLISH_NO_UPDATE_CHECK=1` | never check for releases; no badge | | `HELLISH_NO_ANIM=1` | skip the startup animation | | `HELLISH_ANIM=spinner\|pulse\|ember` | opt into a prompt animation |