Skip to content

feat(parser): read instance strings in one pass, with clear errors - #16

Open
Trosper3 wants to merge 45 commits into
ReduxISU:mainfrom
Trosper3:fix/2-instance-scanner
Open

Trosper3 wants to merge 45 commits into
ReduxISU:mainfrom
Trosper3:fix/2-instance-scanner

Conversation

@Trosper3

@Trosper3 Trosper3 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Instance strings are now read in a single pass that ignores whitespace around structure, reports bad input with a clear message and position, and can't be slowed down or crashed by large or deeply nested input. Every bad-input case now throws one exception type, SpadeParseException.

Stacked on #15. Only the last two commits belong to this PR.

Merge order

#11, #13, #15 and #16 are stacked. Merge them into main in this order:

  1. ci: run the Redux Build System (rbs) report, soft for now #31: rbs CI report and dotnet format (the new base)
  2. fix(collections): hash sets independently of element order #11 (for Equal sets hash differently, so sets of sets give wrong answers #9): set hashing; adds the test project the others use
  3. fix(grammar): accept set edges in unorderedcross again #13 (for fix(grammar): unorderedcross rejects every edge since d7e982e #12): unorderedcross edges
  4. fix(grammar): make is int, is set and is list check their type #15 (for is int and is set constraints don't check their type #10): is int / is set / is list type checks
  5. feat(parser): read instance strings in one pass, with clear errors #16 (for Ignore whitespace that isn't part of a value #2, Clear parse errors #4, Parsing is slow on large input and crashes the process on deep nesting #7): one-pass instance reader and clear errors (this PR)
  6. feat(collections): add multisets #17 (for Add multisets #1): multisets. Optional for the next release: no Redux definition uses it until Redux opts in.

After each merge, the next PR shows only its own commits. If one is squash-merged, I'll rebase the next one onto main before it's reviewed.

Releasing for Redux: publish a new version only after all four are merged. Redux's grammar fixes (ReduxISU/Redux#628, ReduxISU/Redux#629, ReduxISU/Redux#630) work with 0.1.2 and can land any time before the upgrade; only ReduxISU/Redux#630's test changes go in the same Redux PR that upgrades the SPADE package. Tracked in ReduxISU/Redux#627.

What changed

Commit 1, feat(parser): read instance strings in one pass (for #2 and #7, part of #4)

  • New InstanceScanner replaces the substring-based reader in UtilCollection(string).
  • Whitespace (Ignore whitespace that isn't part of a value #2): spaces, tabs and line breaks around { } ( ) , and at either end are skipped. Inside a value, whitespace next to [ ] ; : = is dropped, plain spaces are kept (New York), and a run with a tab or line break becomes one space (New\nYork reads as New York). These are the same rules Redux applies before parsing today, so that layer can be removed.
  • Syntax errors (Clear parse errors #4): malformed input throws SpadeParseException with Position (from 1), Expected and Found, e.g. expected , or } at position 5, found end of input for {1,2. Nothing is guessed any more: {1,2 used to read as {1}, and {1,2}junk as {1,2}jun}.
  • Time and depth (Parsing is slow on large input and crashes the process on deep nesting #7): parsing is linear in the input's length, and ToString builds its text in one buffer. Nesting is limited to 256 levels, and deeper input is rejected instead of overflowing the stack.
  • New SpadeParseException (a FormatException).
  • Docs: "Whitespace" and "Errors" sections in Documentation/InstanceString.md.

Commit 2, fix(grammar): report shape and constraint failures as SpadeParseException (for #4)

  • A format such as (N,E) checks it got a list of exactly that many elements. A 3-element list used to be accepted with the extra element dropped. A set or value used to crash with InvalidOperationException.
  • Constraint failures quote the constraint as written instead of PLACEHOLDER, e.g. Constraint E subset N unorderedcross N failed: {1,9} is not a member. Long values are shortened to keep messages short.
  • subset on a plain value, and cross on anything that isn't a pair, fail the constraint instead of crashing.
  • Docs: "Handling bad input" in Documentation/example.md.

Behavior changes to be aware of

  • new UtilCollection(string) is stricter for every caller, not only through StringParser: empty or blank strings, stray braces, empty elements ({1,,2}, {1,}) and text after the closing brace now throw.
  • Values are trimmed, and whitespace beside [ ] ; : = is removed. "{x1 : True}" now holds x1:True.
  • Exception types change from System.Exception, InvalidOperationException and others to SpadeParseException. It derives from Exception, so existing catch (Exception) blocks still catch it.
  • Grammar mistakes, such as a name that's never defined, still throw other exception types. That's tracked in fix(grammar): x is <set> and undefined names check nothing #14.

Testing

Closes #2
Closes #4
Closes #7

Equal sets such as {1,2} and {2,1} used to get different hash codes,
because GetHashCode hashed ToString(), which lists elements in the
order they were written. Any set of sets could then miss a lookup:
{{1,2}}.Contains({2,1}) was False and {{1,2},{2,1}} kept both.

Sets now combine their element hashes with an order-independent sum,
lists hash their elements in order, and values hash their text.

Equals now also checks that both sides are the same kind (value, list
or set). Before, an empty set compared equal to an empty list and to
any plain value.

Adds a test project (tests/SPADE.Tests, xUnit) with tests for both.

Closes ReduxISU#9
Since d7e982e, an unorderedcross membership check rejected every
normal undirected edge such as {a,b}. The added line checked whether
each node inside the edge was a list, and plain nodes never are.

The check now looks at the edge itself: a plain value is not an
edge, so it fails the constraint instead of crashing while being
enumerated. List edges such as (a,b) are still accepted, as in the
released 0.1.2, because existing grammars for directed graphs rely
on it.

Closes ReduxISU#12
An unorderedcross member is an unordered pair, so it must be a set
such as {a,b}. A list such as (a,b) is an ordered pair, which is what
cross describes, so it is now rejected here too. This is what
d7e982e set out to do.

Grammars for directed graphs that use unorderedcross with list edges
need to switch to cross.

Refs ReduxISU#12
"N is int" checked nothing at all: int reaches the constraint as a set
descriptor, and the "is" branch only handled the set, list and int
keywords. So abc, {1} and (1,2) all passed. It now requires a single
value that parses as an integer.

"S is set" accepted a plain value, because a value is not ordered. It
now requires an unordered collection.

"is list" already rejected values and sets; it is now covered by tests.

Closes ReduxISU#10
Replaces the substring-based reader in UtilCollection(string) with
InstanceScanner, which walks the input once.

Whitespace (ReduxISU#2): spaces, tabs and line breaks around { } ( ) , and at
either end of the input are skipped, so {2, 3} now holds 3 and a
trailing new line no longer breaks an instance. Inside a value,
whitespace next to [ ] ; : = is dropped, plain spaces are kept (New
York), and a run with a tab or line break becomes one space. These
match the rules Redux applies before parsing today.

Errors (ReduxISU#4): malformed input throws SpadeParseException with the
position, what was expected and what was found, e.g. "expected , or }
at position 5, found end of input" for {1,2. Input that used to be
misread without an error, such as {1,2 or {1,2}junk, is now rejected.

Time and depth (ReduxISU#7): parsing is linear in the input's length, and
ToString builds its text in one buffer. Nesting deeper than 256 levels
is rejected instead of overflowing the stack and crashing the process.
40,000 elements took 104 seconds before; 100,000 now take well under
a second.

Closes ReduxISU#2
Closes ReduxISU#7
Refs ReduxISU#4
…tion

Input that is well formed but doesn't fit the definition string threw
whatever happened to fail: InvalidOperationException ("Cannot index
into a set"), ArgumentOutOfRangeException, or a bare Exception whose
message said PLACEHOLDER. All of these now throw SpadeParseException
with a message that names what failed:

- A format such as (N,E) checks that it got a list of exactly that
  many elements. A longer list used to be accepted and the extra
  elements dropped.
- Constraint failures quote the constraint as written, e.g.
  "Constraint E subset N unorderedcross N failed: {1,9} is not a
  member". Long values are shortened.
- subset on a plain value, and cross on anything that isn't a pair,
  fail the constraint instead of crashing.

Closes ReduxISU#4
@Trosper3 Trosper3 added this to the 0.2.0 milestone Oct 13, 2026
@norkish norkish self-assigned this Oct 15, 2026
rbs runs audit, format-check, lint (dotnet build) and unit tests with a
coverage floor inside the repo's devcontainer, the same way it does for
Redux. It reports without failing (soft) while ci.yml stays the gate;
publish.yml keeps doing the NuGet release, since rbs has no NuGet push.

- SPADE.slnx: the dotnet engine needs a solution. It names only
  DiscreteParser.csproj for now; the test project joins it with the first
  fix PR. ci.yml and publish.yml name DiscreteParser.csproj explicitly, so
  the MSB1011 a bare `dotnet build` now gives does not affect them.
- rbs.toml: coverage floor 70; the goal is 80%+.
- .devcontainer: SDK 10 (reads .slnx, builds the net6.0 library) plus the
  .NET 8 runtime the tests run on.
- .gitignore: rbs's .rbs/ work directory and report.md.
Whitespace only: dotnet format SPADE.slnx, so rbs's format-check starts clean. Brace placement, indentation and blank lines; no code changes.
SPADE.slnx now names tests/SPADE.Tests, so rbs's unit-test stage runs it.
coverlet.collector writes the cobertura report the coverage floor reads;
without it the project gets no coverage gate at all. Same version as
Redux's test project.
Whitespace only: dotnet format SPADE.slnx on this branch's code.
The tests move to net10.0, so the container no longer needs the .NET 8
runtime feature next to the SDK 10 image. SDK 10 still reads SPADE.slnx
and builds the net6.0 library.
The test project moves from net8.0 to net10.0, and ci.yml and publish.yml
install only the 10.0.x SDK, which still builds the net6.0 library.
DiscreteParser.csproj stays on net6.0, the NuGet package's target. The
test packages (xunit.v3 3.2.2, xunit.runner.visualstudio 4.0.0,
Microsoft.NET.Test.Sdk 18.9.0, coverlet.collector 10.1.0) run on net10.0
unchanged. development.md now says the tests need the .NET 10 SDK.
The measured figure changes with every PR in the stack, so the comment
states only the floor and the goal; the rbs report shows the current number.
@Trosper3

Copy link
Copy Markdown
Contributor Author

Rebased the stack on #31 by merging (no force-push): this branch now has the rbs report, dotnet format (whitespace only; it's a separate style: commit), and tests on .NET 10 from #11. There are no code changes beyond that.

# Conflicts:
#	Documentation/InstanceString.md
#	Documentation/example.md
A HashSet or Dictionary files each element under the hash it had when it
went in. UtilCollection is mutable, so changing a collection (or anything
nested in it) after it has been hashed left it under a stale hash:
Contains and equality lookups missed it and duplicates slipped in, with no
error.

The first GetHashCode() call now freezes the collection. A list's or set's
hash is built from its elements' GetHashCode(), so they freeze too, all the
way down. Add and the indexer setter throw InvalidOperationException on a
frozen collection. The hash is computed once and cached.

The List<UtilCollection> and HashSet<UtilCollection> constructors copy
their argument instead of keeping the caller's reference, and ToList() on
a list returns a copy, so a frozen collection can't be changed from
outside either.
GetHashCode writes the hash and then sets frozen. Without a barrier,
another thread could see frozen before the hash and return 0. frozen is
now volatile, so a thread that sees it also sees the hash; two threads
computing it at once get the same value. A frozen collection can't
change, so it is safe to share between threads; building one still
isn't, as with List and HashSet. Documented, with a test that races
threads to hash a fresh nested set.
… threads

Freezing used to happen only as a side effect of hashing, so a list that
was built but never hashed could still be changed while other threads
read it. Freeze() freezes a collection and everything inside it on
purpose and returns it; IsFrozen() reports it. Documentation/threads.md
says what is safe to share (frozen collections, sets) and what isn't
(a StringParser while it parses, a collection still being built), with
the parse-once, freeze, then share pattern.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parsing is slow on large input and crashes the process on deep nesting Clear parse errors Ignore whitespace that isn't part of a value

2 participants