Skip to content

feat(collections): add multisets - #17

Open
Trosper3 wants to merge 56 commits into
ReduxISU:mainfrom
Trosper3:feat/1-multisets
Open

Trosper3 wants to merge 56 commits into
ReduxISU:mainfrom
Trosper3:feat/1-multisets

Conversation

@Trosper3

@Trosper3 Trosper3 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds multisets, following the design in #1. A definition string can mark a collection with is multiset, and its repeated elements are kept instead of dropped: {N | N is multiset, N subset int} reads {3,3,2,2} as four numbers. Nothing changes for definitions or code that don't use it.

Draft: this stays a draft until @Jetison333 has answered the open questions in #1, unless you're happy to merge it with the choices below. Each is a small change if you'd prefer the other option.

Stacked on #16. Only the last commit, feat(collections): add multisets, belongs to this PR.

Merge order

#11, #13, #15 and #16 are stacked, and this PR builds on them:

  1. ci: run the Redux Build System (rbs) report, soft for now #31: rbs CI report and dotnet format (the new base)
  2. fix(collections): hash sets independently of element order #11 (for Equal sets hash differently, so sets of sets give wrong answers #9): set hashing; adds the test project the others use
  3. fix(grammar): accept set edges in unorderedcross again #13 (for fix(grammar): unorderedcross rejects every edge since d7e982e #12): unorderedcross edges
  4. fix(grammar): make is int, is set and is list check their type #15 (for is int and is set constraints don't check their type #10): is int / is set / is list type checks
  5. feat(parser): read instance strings in one pass, with clear errors #16 (for Ignore whitespace that isn't part of a value #2, Clear parse errors #4, Parsing is slow on large input and crashes the process on deep nesting #7): one-pass instance reader and clear errors
  6. feat(collections): add multisets #17 (for Add multisets #1): multisets (this PR)

This PR is optional for the next release: no Redux definition uses is multiset until Redux opts in, so releasing with or without it doesn't change Redux's results.

Choices for the open questions in #1

Question Chosen here Alternative
Keyword is multiset e.g. is bag
Duplicates in a field marked is set still dropped without an error make them an error
ToString() order for a multiset input order, e.g. {3,2,3,2} a canonical order, with #8
Union of multisets larger count of each element sum of the counts

What changed

  • UtilCollection
    • A set read from an instance string keeps its elements as written, and AsMultiset() turns them into a multiset. This is how is multiset gets the duplicates back after the reader has built a set.
    • Multisets count every occurrence in Count(), enumeration and ToString(). They compare by count in any order, and their hash codes are order-independent. Union and Intersect use the larger and smaller count.
    • New IsMultiset(), AsMultiset(), CountOf(x) and IsSubsetOf(other). IsSubsetOf compares counts, so {7,7} is in {7,7,1} but not in {7,1}.
    • A set and a multiset are never equal, and mixing them in Union or Intersect throws, the same as mixing a set and a list.
    • Set behavior, including set hash codes, is unchanged.
  • Definition strings
    • multiset is added to the BUILT_IN keywords in coll.g4. The lexer is regenerated with ANTLR 4.13.2, the version that generated the existing files: regenerating the unchanged grammar reproduces them exactly. Only collLexer.cs and collLexer.interp change.
    • N is multiset requires N to be written with braces, and runs before the other constraints wherever it's written.
    • is set now rejects a multiset.
  • Docs: is multiset in DefinitionString.md, a "Multisets" section in InstanceString.md, and the new methods in utilcollection.md.

Limitation: is multiset applies to a field of the definition (N, or S in (S,T)). Elements nested inside a field, such as multisets inside a set comprehension, stay sets. Partition, Subset Sum and Knapsack don't need that.

Testing

Closes #1

Equal sets such as {1,2} and {2,1} used to get different hash codes,
because GetHashCode hashed ToString(), which lists elements in the
order they were written. Any set of sets could then miss a lookup:
{{1,2}}.Contains({2,1}) was False and {{1,2},{2,1}} kept both.

Sets now combine their element hashes with an order-independent sum,
lists hash their elements in order, and values hash their text.

Equals now also checks that both sides are the same kind (value, list
or set). Before, an empty set compared equal to an empty list and to
any plain value.

Adds a test project (tests/SPADE.Tests, xUnit) with tests for both.

Closes ReduxISU#9
Since d7e982e, an unorderedcross membership check rejected every
normal undirected edge such as {a,b}. The added line checked whether
each node inside the edge was a list, and plain nodes never are.

The check now looks at the edge itself: a plain value is not an
edge, so it fails the constraint instead of crashing while being
enumerated. List edges such as (a,b) are still accepted, as in the
released 0.1.2, because existing grammars for directed graphs rely
on it.

Closes ReduxISU#12
An unorderedcross member is an unordered pair, so it must be a set
such as {a,b}. A list such as (a,b) is an ordered pair, which is what
cross describes, so it is now rejected here too. This is what
d7e982e set out to do.

Grammars for directed graphs that use unorderedcross with list edges
need to switch to cross.

Refs ReduxISU#12
"N is int" checked nothing at all: int reaches the constraint as a set
descriptor, and the "is" branch only handled the set, list and int
keywords. So abc, {1} and (1,2) all passed. It now requires a single
value that parses as an integer.

"S is set" accepted a plain value, because a value is not ordered. It
now requires an unordered collection.

"is list" already rejected values and sets; it is now covered by tests.

Closes ReduxISU#10
Replaces the substring-based reader in UtilCollection(string) with
InstanceScanner, which walks the input once.

Whitespace (ReduxISU#2): spaces, tabs and line breaks around { } ( ) , and at
either end of the input are skipped, so {2, 3} now holds 3 and a
trailing new line no longer breaks an instance. Inside a value,
whitespace next to [ ] ; : = is dropped, plain spaces are kept (New
York), and a run with a tab or line break becomes one space. These
match the rules Redux applies before parsing today.

Errors (ReduxISU#4): malformed input throws SpadeParseException with the
position, what was expected and what was found, e.g. "expected , or }
at position 5, found end of input" for {1,2. Input that used to be
misread without an error, such as {1,2 or {1,2}junk, is now rejected.

Time and depth (ReduxISU#7): parsing is linear in the input's length, and
ToString builds its text in one buffer. Nesting deeper than 256 levels
is rejected instead of overflowing the stack and crashing the process.
40,000 elements took 104 seconds before; 100,000 now take well under
a second.

Closes ReduxISU#2
Closes ReduxISU#7
Refs ReduxISU#4
…tion

Input that is well formed but doesn't fit the definition string threw
whatever happened to fail: InvalidOperationException ("Cannot index
into a set"), ArgumentOutOfRangeException, or a bare Exception whose
message said PLACEHOLDER. All of these now throw SpadeParseException
with a message that names what failed:

- A format such as (N,E) checks that it got a list of exactly that
  many elements. A longer list used to be accepted and the extra
  elements dropped.
- Constraint failures quote the constraint as written, e.g.
  "Constraint E subset N unorderedcross N failed: {1,9} is not a
  member". Long values are shortened.
- subset on a plain value, and cross on anything that isn't a pair,
  fail the constraint instead of crashing.

Closes ReduxISU#4
A set drops repeated elements, so {3,3,2,2} was read as {3,2}. Problems
defined on multisets, such as Partition and Subset Sum, then solved a
different instance from the one written.

A definition string can now mark a collection with "is multiset":

    {N | N is multiset, N subset int}

reads {3,3,2,2} as four numbers. It applies before the other
constraints wherever it is written. Without it, braces still mean a
set, so nothing changes for existing definitions or callers.

To make this possible, a set read from an instance string keeps its
elements as written, and AsMultiset() turns them into a multiset.
Multisets count every occurrence in Count(), enumeration and
ToString(), compare by count in any order, and have order-independent
hash codes. Union and Intersect use the larger and smaller count. New
CountOf() and IsSubsetOf() compare by count, so {7,7} is in {7,7,1}
but not in {7,1}. A set and a multiset are never equal, and "is set"
rejects a multiset.

Adds the multiset keyword to coll.g4; the lexer is regenerated with
ANTLR 4.13.2, the version that generated the existing files.

Closes ReduxISU#1
rbs runs audit, format-check, lint (dotnet build) and unit tests with a
coverage floor inside the repo's devcontainer, the same way it does for
Redux. It reports without failing (soft) while ci.yml stays the gate;
publish.yml keeps doing the NuGet release, since rbs has no NuGet push.

- SPADE.slnx: the dotnet engine needs a solution. It names only
  DiscreteParser.csproj for now; the test project joins it with the first
  fix PR. ci.yml and publish.yml name DiscreteParser.csproj explicitly, so
  the MSB1011 a bare `dotnet build` now gives does not affect them.
- rbs.toml: coverage floor 70; the goal is 80%+.
- .devcontainer: SDK 10 (reads .slnx, builds the net6.0 library) plus the
  .NET 8 runtime the tests run on.
- .gitignore: rbs's .rbs/ work directory and report.md.
Whitespace only: dotnet format SPADE.slnx, so rbs's format-check starts clean. Brace placement, indentation and blank lines; no code changes.
@Trosper3

Copy link
Copy Markdown
Contributor Author

Rebased the stack on #31 by merging (no force-push): this branch now has the rbs report, dotnet format (whitespace only; it's a separate style: commit), and tests on .NET 10 from #11. There are no code changes beyond that.

# Conflicts:
#	Documentation/InstanceString.md
#	Documentation/example.md
# Conflicts:
#	Documentation/DefinitionString.md
A HashSet or Dictionary files each element under the hash it had when it
went in. UtilCollection is mutable, so changing a collection (or anything
nested in it) after it has been hashed left it under a stale hash:
Contains and equality lookups missed it and duplicates slipped in, with no
error.

The first GetHashCode() call now freezes the collection. A list's or set's
hash is built from its elements' GetHashCode(), so they freeze too, all the
way down. Add and the indexer setter throw InvalidOperationException on a
frozen collection. The hash is computed once and cached.

The List<UtilCollection> and HashSet<UtilCollection> constructors copy
their argument instead of keeping the caller's reference, and ToList() on
a list returns a copy, so a frozen collection can't be changed from
outside either.
Multisets go through the same Add guard, so a hashed multiset can't
change its occurrences or counts. FromSetItems now copies the item list it
keeps as the set's occurrences instead of holding the caller's list.

Tests cover a hashed multiset refusing Add, its elements freezing as they
go in, AsMultiset still working on a frozen set, and unhashed multisets
changing freely.
GetHashCode writes the hash and then sets frozen. Without a barrier,
another thread could see frozen before the hash and return 0. frozen is
now volatile, so a thread that sees it also sees the hash; two threads
computing it at once get the same value. A frozen collection can't
change, so it is safe to share between threads; building one still
isn't, as with List and HashSet. Documented, with a test that races
threads to hash a fresh nested set.
# Conflicts:
#	Documentation/utilcollection.md
… threads

Freezing used to happen only as a side effect of hashing, so a list that
was built but never hashed could still be changed while other threads
read it. Freeze() freezes a collection and everything inside it on
purpose and returns it; IsFrozen() reports it. Documentation/threads.md
says what is safe to share (frozen collections, sets) and what isn't
(a StringParser while it parses, a collection still being built), with
the parse-once, freeze, then share pattern.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add multisets

2 participants