Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
0a2b9b2
Bump the analyzers group with 2 updates
dependabot[bot] Jul 30, 2026
f5d1164
Bump Microsoft.Testing.Extensions.TrxReport from 2.3.2 to 2.3.3
dependabot[bot] Jul 30, 2026
4f2dc4b
Merge pull request #48 from GabrielMarquezMatte/dependabot/nuget/deve…
GabrielMarquezMatte Jul 31, 2026
a7feec9
Merge pull request #49 from GabrielMarquezMatte/dependabot/nuget/test…
GabrielMarquezMatte Jul 31, 2026
fe496cd
Bump CsCheck from 4.7.0 to 4.9.1
dependabot[bot] Sep 24, 2026
45a74ef
Bump Microsoft.Testing.Extensions.CodeCoverage from 18.9.0 to 18.12.0
dependabot[bot] Oct 8, 2026
70b58f8
Bump the test group with 2 updates
dependabot[bot] Oct 15, 2026
17c0baa
Bump System.IO.Pipelines from 10.0.10 to 10.0.13
dependabot[bot] Oct 15, 2026
0a42254
Merge pull request #77 from GabrielMarquezMatte/dependabot/nuget/src/…
GabrielMarquezMatte Oct 15, 2026
6929c9c
Merge pull request #76 from GabrielMarquezMatte/dependabot/nuget/test…
GabrielMarquezMatte Oct 15, 2026
66ff2d6
Merge pull request #73 from GabrielMarquezMatte/dependabot/nuget/test…
GabrielMarquezMatte Oct 15, 2026
e87af5c
Merge pull request #70 from GabrielMarquezMatte/dependabot/nuget/test…
GabrielMarquezMatte Oct 15, 2026
5c1ea64
feat!: read and write bytes only, configured through options
gabriel-matte-daemon Oct 15, 2026
21872ab
ci: validate the package against the last release on every pull request
gabriel-matte-daemon Oct 15, 2026
84c11f1
docs: document the 2.0 API and split the changelog by release
gabriel-matte-daemon Oct 15, 2026
4fce9bb
perf(writer): format numbers straight into the column and write plain…
Oct 16, 2026
759aafc
refactor(generator): read one column in its own method
Oct 16, 2026
868d528
build: align .editorconfig with the style guide and pin LF line endings
Oct 16, 2026
b6f25b2
docs: add a style guide, an architecture map and the writer measurements
Oct 16, 2026
a800a14
feat(reader): aggregate a file in parallel
Oct 16, 2026
a32b1d5
feat(reader): aggregate a stream in parallel
Oct 16, 2026
d36b958
feat(reader)!: drop PipeReader reading in favour of AsStream
Oct 16, 2026
ff98723
feat(layout): build attribute layouts as FixedWidthLayout<T> objects
Oct 24, 2026
0eb52a7
feat(reader): read and aggregate through a FixedWidthLayout<T>
Oct 24, 2026
ae1ca4a
feat(writer): write generated and layout models through FixedWidth
Oct 24, 2026
1b09d6e
feat(layout): describe a layout fluently with FixedWidthLayout.Build
Oct 24, 2026
b640bcb
feat!: make FixedWidth the only entry point and drop the runtime classes
Oct 24, 2026
02da8af
test: publish a fluent-layout app with NativeAOT and record the read/…
Oct 24, 2026
9b017fc
docs: document FixedWidthLayout and the single FixedWidth API
Oct 24, 2026
c57d36a
test: take the best of three allocation measurements per guard
Oct 24, 2026
cff0c39
feat(layout): add nullable decoded and nullable converted fluent columns
Oct 24, 2026
bcfc200
refactor!: move user-facing types to the root namespace and plumbing …
Oct 24, 2026
9eee4ac
refactor!: name parameters the same way across the API
Oct 24, 2026
cb5ee86
refactor!: fold FixedWidthParallelOptions into FixedWidthReaderOptions
Oct 24, 2026
18c1545
refactor!: pass per-column parse and format settings as extensible st…
Oct 24, 2026
55c74ea
refactor: hide generated-code plumbing from IntelliSense
Oct 24, 2026
c556c08
docs: describe the settled public surface and the 2.x compatibility r…
Oct 24, 2026
487d889
feat: accept ref struct models in the readers and aggregates
Oct 24, 2026
6a1504b
refactor: run parallel aggregation over an internal line sink and source
Oct 24, 2026
b291855
feat(native): scaffold the NativeAOT library
Oct 24, 2026
7b8d214
feat(native): build layouts from C column specs
Oct 24, 2026
030629c
feat(native): read fixed-width files into a columnar table
Oct 24, 2026
903894d
feat(native): read in parallel by byte range
Oct 24, 2026
e1aee57
feat(native): read in batches
Oct 24, 2026
7608ee7
feat(native): export reads through the Arrow C Data Interface
Oct 24, 2026
57029c1
feat(native): write columnar tables
Oct 24, 2026
7629e84
feat(native): C header and smoke test
Oct 24, 2026
5e3ac88
feat(python): fixedwidth package over the native library
Oct 24, 2026
60c45c0
docs: benchmark the Python reader against pandas.read_fwf
Oct 24, 2026
9d30a6e
feat(cpp): header-only C++ wrapper
Oct 24, 2026
d3cefe1
ci: build and test the native library and its bindings
Oct 24, 2026
c130f5c
fix(native): valid UTF-8, size limits and UTC timestamps
Oct 24, 2026
251d786
docs(native): document UTF-8 replacement and UTC timestamps in the he…
Oct 24, 2026
2b94e05
fix: add IDE1006 to WarningsNotAsErrors in Directory.Build.props
Oct 24, 2026
b7baca0
ci: publish and run the NativeAOT smoke app
Oct 24, 2026
68b4b8b
fix(python): exact, range-checked decimal writes
Oct 24, 2026
3c55352
fix: merge stream aggregation results incrementally with bounded pend…
Oct 24, 2026
e9a5ad8
fix(generator): include inherited mapped columns
Oct 24, 2026
26e5672
fix: return the rented buffer when strict decoding throws
Oct 24, 2026
3cf7130
fix(cpp): import library for the target architecture
Oct 24, 2026
1236724
refactor(ci): clean up comments and improve workflow clarity
Oct 24, 2026
6ec4caa
perf: parse doubles with csFastFloat in comma cultures
Oct 24, 2026
6b40b40
perf: cache UTF-8 columns in front of StringPool
Oct 24, 2026
ef287ed
feat: enhance documentation for FixedWidthMapper and add example usage
Oct 24, 2026
a03957c
docs: split the README into a usage guide and a benchmarks page
Oct 24, 2026
6268482
docs: bring the docs up to date with the code
Oct 24, 2026
b7db52d
build: turn off package validation while 2.0 is in development
Oct 24, 2026
5241a19
perf: replace csFastFloat with ports of ExcelReader's FastDouble and …
Oct 24, 2026
8da9f7a
refactor: collapse duplicated parse, format and read helpers
Oct 24, 2026
4fdcdcf
refactor(native): share column constants and the invalid-argument helper
Oct 24, 2026
d230e3c
refactor(generator): make ColumnInfo a record struct
Oct 24, 2026
ecbe0cf
perf: parse digit-only dates and integers straight from the bytes
Oct 24, 2026
fff8a5a
perf(native): cut page faults and serial work from parallel reads
Oct 24, 2026
d6f30d5
bench: add a Python write benchmark against savetxt and polars
Oct 24, 2026
b4d22cf
perf: write integers and digit-only dates directly, and write files i…
Oct 24, 2026
a313504
refactor: reorganize import statements for clarity
Oct 24, 2026
c8d1d5a
perf(native): parse partitions straight into the final columns
Oct 25, 2026
3f3b55a
ci(release): publish Python wheels to PyPI as fixedwidthparser
Oct 25, 2026
b34af7a
Merge develop
Oct 25, 2026
7f47469
ci: run each workflow only when its paths change
Oct 25, 2026
2a387c4
Merge pull request #79 from GabrielMarquezMatte/feat/v2-api
GabrielMarquezMatte Oct 25, 2026
0d18452
Bump Microsoft.Testing.Extensions.TrxReport from 2.3.3 to 2.5.1
dependabot[bot] Oct 25, 2026
e632b5a
Bump Microsoft.Testing.Extensions.TrxReport from 2.3.3 to 2.5.1
dependabot[bot] Oct 25, 2026
a8a2c9a
ci: add result job to enforce binding checks in workflow
Oct 25, 2026
43152fa
Merge branch 'feat/v2-api' into develop
Oct 25, 2026
3e3dfe3
feat(cpp): install as a CMake package and download the release's nati…
Oct 25, 2026
3ac88c1
feat: parse and write enum columns
Oct 25, 2026
e06bc71
feat(native): read batches and Arrow streams from a forward-only stream
Oct 25, 2026
3b873ab
docs: refresh the Python benchmark numbers and document thread safety
Oct 25, 2026
48b567c
test(native): compare random partitioned reads with the sequential read
Oct 25, 2026
b504234
perf(native): lend STRING offsets to the partitions too
Oct 25, 2026
20a42bf
feat(native): dictionary-encode STRING columns for Arrow
Oct 25, 2026
70948ae
feat(native): write to a sink callback as well as to a file or memory
Oct 25, 2026
a123dd5
Merge pull request #81 from GabrielMarquezMatte/dependabot/nuget/test…
GabrielMarquezMatte Oct 26, 2026
ee3a4cd
Merge pull request #80 from GabrielMarquezMatte/dependabot/nuget/test…
GabrielMarquezMatte Oct 26, 2026
85fd766
chore: update package references and suppress additional warnings
Oct 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 52 additions & 31 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -1,72 +1,93 @@
# EditorConfig — https://editorconfig.org
# Top-most file; stops the search here.
root = true

##########################################
# All files
##########################################
[*]
charset = utf-8
# lf, not crlf: every tracked file is committed LF. Demanding crlf here would only hold on Windows,
# where core.autocrlf converts on checkout and hides the mismatch. See .gitattributes, which pins
# eol=lf so no contributor's autocrlf setting can change what lands in a blob.
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
indent_style = space
indent_size = 4

[*.{yml,yaml,json,md}]
[*.{yml,yaml,json,md,csproj,props,targets,slnx}]
indent_size = 2

# Two trailing spaces are a hard line break in Markdown.
[*.md]
trim_trailing_whitespace = false # trailing spaces are significant (hard line breaks)
trim_trailing_whitespace = false

[*.{csproj,props,targets,slnx}]
indent_size = 2

##########################################
# C# files
##########################################
[*.cs]
indent_size = 4
max_line_length = 120

# --- Language conventions ---------------------------------------------------
# The code is block-scoped throughout; switching styles file by file would make every diff a reindent.
csharp_style_namespace_declarations = block_scoped:suggestion
csharp_using_directive_placement = outside_namespace:suggestion
dotnet_sort_system_directives_first = true
dotnet_separate_import_directive_groups = false

# var usage — prefer explicit types except when obvious from the right-hand side.
csharp_style_var_for_built_in_types = false:suggestion
csharp_style_var_when_type_is_apparent = true:suggestion
csharp_style_var_elsewhere = false:suggestion

# Expression-bodied members.
csharp_style_expression_bodied_methods = false:suggestion
# STYLEGUIDE.md: "Every method ... must use a block body. Expression bodies (=>) are forbidden on
# methods, no matter how short." Properties are exempted here (project preference overrides the
# guide for the common one-line forwarding getter).
csharp_style_expression_bodied_methods = false:warning
csharp_style_expression_bodied_constructors = false:warning
csharp_style_expression_bodied_operators = false:warning
csharp_style_expression_bodied_local_functions = false:warning
csharp_style_expression_bodied_properties = true:suggestion
csharp_style_expression_bodied_accessors = false:suggestion
# Accessors (get/set/init) are part of the property exemption: either body form is fine, so IDE0027
# reports nothing.
dotnet_diagnostic.IDE0027.severity = none

# STYLEGUIDE.md: "Always use braces on `if`, even single-line bodies. The linter enforces this."
csharp_prefer_braces = true:warning

# STYLEGUIDE.md ternary rule ("nested ternary" / "branches contain logic" forbidden) has no direct
# analyzer id; IDE0045/IDE0046 (convert if to/from ternary) would fight that rule, so both are off.
dotnet_diagnostic.IDE0045.severity = none
dotnet_diagnostic.IDE0046.severity = none

# Modern C# preferences.
csharp_prefer_braces = true:suggestion
csharp_style_prefer_pattern_matching = true:suggestion
csharp_style_prefer_switch_expression = true:suggestion
csharp_style_throw_expression = true:suggestion
csharp_prefer_simple_using_statement = true:suggestion
csharp_style_prefer_range_operator = true:suggestion
csharp_style_prefer_index_operator = true:suggestion

# --- Naming: private fields use _camelCase --------------------------------
dotnet_naming_rule.private_fields_underscore.severity = suggestion
dotnet_naming_rule.private_fields_underscore.symbols = private_fields
dotnet_naming_rule.private_fields_underscore.style = underscore_prefix
dotnet_style_qualification_for_field = false:suggestion
dotnet_style_qualification_for_property = false:suggestion
dotnet_style_qualification_for_method = false:suggestion
dotnet_style_qualification_for_event = false:suggestion

# Naming: private fields _camelCase, private consts PascalCase, everything else PascalCase/camelCase
# per STYLEGUIDE.md.
dotnet_naming_style.underscore_camel_case.capitalization = camel_case
dotnet_naming_style.underscore_camel_case.required_prefix = _
dotnet_naming_style.pascal_case.capitalization = pascal_case

# Consts first — more specific than the general private-field rule below, so it must win. Rule order
# matters: the first matching rule applies.
dotnet_naming_symbols.private_consts.applicable_kinds = field
dotnet_naming_symbols.private_consts.applicable_accessibilities = private
dotnet_naming_symbols.private_consts.required_modifiers = const

dotnet_naming_rule.private_consts_pascal_case.symbols = private_consts
dotnet_naming_rule.private_consts_pascal_case.style = pascal_case
dotnet_naming_rule.private_consts_pascal_case.severity = warning

dotnet_naming_symbols.private_fields.applicable_kinds = field
dotnet_naming_symbols.private_fields.applicable_accessibilities = private
dotnet_naming_symbols.private_fields.required_modifiers =

dotnet_naming_style.underscore_prefix.required_prefix = _
dotnet_naming_style.underscore_prefix.capitalization = camel_case
dotnet_naming_rule.private_fields_underscore_camel_case.symbols = private_fields
dotnet_naming_rule.private_fields_underscore_camel_case.style = underscore_camel_case
dotnet_naming_rule.private_fields_underscore_camel_case.severity = warning

# --- 'this.' qualification --------------------------------------------------
dotnet_style_qualification_for_field = false:suggestion
dotnet_style_qualification_for_property = false:suggestion
dotnet_style_qualification_for_method = false:suggestion
dotnet_style_qualification_for_event = false:suggestion
dotnet_diagnostic.MA0051.severity = warning
MA0051.maximum_lines_per_method = 90
MA0051.maximum_statements_per_method = 50
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Line endings are pinned here rather than left to each contributor's core.autocrlf, so what lands in
# a blob never depends on who committed it. Every tracked text file is already LF, and
# .editorconfig's end_of_line matches.
* text=auto eol=lf
11 changes: 9 additions & 2 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,15 @@ updates:
test:
patterns:
- "xunit*"
- "Microsoft.NET.Test.Sdk"
- "coverlet.*"
- "Microsoft.Testing.*"
- "CsCheck"
# The libraries the comparison benchmarks measure against.
benchmarks:
patterns:
- "BenchmarkDotNet"
- "FileHelpers"
- "FlatFiles"
- "RecordParser"
labels:
- dependencies

Expand Down
4 changes: 3 additions & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@
- [ ] Tests added/updated for the change
- [ ] `dotnet build --configuration Release` is warning-clean
- [ ] `dotnet test --configuration Release` passes
- [ ] Public API changes are documented (XML docs + README if relevant)
- [ ] Public API changes are documented (XML docs, plus `docs/guide/` if relevant) and recorded in `PublicAPI.Unshipped.txt`
- [ ] User-visible changes are in `CHANGELOG.md`
- [ ] Native, Python or C++ changes: the Python, C and C++ tests pass (see CONTRIBUTING.md)
- [ ] For performance-sensitive changes: benchmark numbers included below

## Benchmark impact (if applicable)
Expand Down
83 changes: 83 additions & 0 deletions .github/scripts/promote_public_api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""Moves every entry out of each PublicAPI.Unshipped.txt into its PublicAPI.Shipped.txt.

Run after a release ships: everything that was "unshipped" at release time is now part of
the public API contract. Operates on every src/*/PublicAPI.Unshipped.txt.

Exit codes: 0 = promoted something, 2 = nothing needed promoting (not an error — the caller
skips the rest of the job), 1 = an actual failure (an uncaught exception, same as Python's
default for any unhandled error). 2 is deliberately not 1 - an uncaught exception used
to exit 1 exactly like the intentional "nothing to promote" case, so release.yml's `if
python3 ...; then changed=true; else changed=false; fi` silently treated a crash (missing
file, permission error) the same as a no-op success, disarming the next release's
breaking-change detection instead of failing the job.
"""

import glob
import os
import sys

HEADER = "#nullable enable"


def read_entries(path):
with open(path, encoding="utf-8") as f:
return [line.strip() for line in f if line.strip() and line.strip() != HEADER]


def write_entries(path, entries):
with open(path, "w", encoding="utf-8", newline="\n") as f:
f.write(HEADER + "\n")
f.writelines(entry + "\n" for entry in sorted(entries))


REMOVED_PREFIX = "*REMOVED*"


def promote_one(unshipped_path, shipped_path):
"""Applies unshipped_path's entries onto shipped_path and clears unshipped_path.

A plain entry is added to Shipped. A `*REMOVED*Foo` entry deletes the plain `Foo` entry
from Shipped (an API removal is meaningful only for one release cycle) and is itself
discarded rather than copied - RS0024 forbids a *REMOVED* line inside Shipped.txt, since
Shipped is the current API surface, not a removal history.

Returns the number of entries read from unshipped_path (0 if there was nothing to do).
"""
new_entries = read_entries(unshipped_path)
if not new_entries:
return 0

shipped_entries = set(read_entries(shipped_path)) if os.path.exists(shipped_path) else set()
for entry in new_entries:
if entry.startswith(REMOVED_PREFIX):
shipped_entries.discard(entry[len(REMOVED_PREFIX):])
else:
shipped_entries.add(entry)

write_entries(shipped_path, shipped_entries)
write_entries(unshipped_path, [])
return len(new_entries)


def main():
unshipped_files = sorted(glob.glob("src/*/PublicAPI.Unshipped.txt"))
if not unshipped_files:
print("No src/*/PublicAPI.Unshipped.txt files found.")
return 2

promoted_any = False
for unshipped_path in unshipped_files:
shipped_path = unshipped_path.replace("Unshipped.txt", "Shipped.txt")
count = promote_one(unshipped_path, shipped_path)
if count == 0:
print(f"{unshipped_path}: nothing to promote.")
continue
print(f"{unshipped_path}: promoted {count} entries to {shipped_path}.")
promoted_any = True

return 0 if promoted_any else 2


if __name__ == "__main__":
sys.exit(main())
77 changes: 77 additions & 0 deletions .github/scripts/test_promote_public_api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#!/usr/bin/env python3
"""Unit tests for promote_public_api.py.

Run: python3 .github/scripts/test_promote_public_api.py
"""

import os
import tempfile
import unittest

import promote_public_api as promote


class PromoteTests(unittest.TestCase):
def _write(self, path, lines):
with open(path, "w", encoding="utf-8", newline="\n") as f:
f.write(promote.HEADER + "\n")
f.writelines(line + "\n" for line in lines)

def _read(self, path):
with open(path, encoding="utf-8") as f:
return {line.strip() for line in f if line.strip() and line.strip() != promote.HEADER}

def setUp(self):
self._tmp = tempfile.TemporaryDirectory()
self.shipped = os.path.join(self._tmp.name, "PublicAPI.Shipped.txt")
self.unshipped = os.path.join(self._tmp.name, "PublicAPI.Unshipped.txt")

def tearDown(self):
self._tmp.cleanup()

def test_plain_entry_is_added_to_shipped(self):
self._write(self.shipped, ["Foo.Bar() -> void"])
self._write(self.unshipped, ["Foo.Baz() -> void"])

promote.promote_one(self.unshipped, self.shipped)

self.assertEqual(self._read(self.shipped), {"Foo.Bar() -> void", "Foo.Baz() -> void"})
self.assertEqual(self._read(self.unshipped), set())

def test_removed_entry_deletes_the_plain_entry_from_shipped(self):
self._write(self.shipped, ["Foo.Bar() -> void", "Foo.Baz() -> void"])
self._write(self.unshipped, ["*REMOVED*Foo.Bar() -> void"])

promote.promote_one(self.unshipped, self.shipped)

self.assertEqual(self._read(self.shipped), {"Foo.Baz() -> void"})
self.assertEqual(self._read(self.unshipped), set())

def test_removed_entry_is_never_written_to_shipped(self):
self._write(self.shipped, ["Foo.Bar() -> void"])
self._write(self.unshipped, ["*REMOVED*Foo.Bar() -> void"])

promote.promote_one(self.unshipped, self.shipped)

for entry in self._read(self.shipped):
self.assertFalse(entry.startswith("*REMOVED*"), entry)

def test_removed_entry_with_no_matching_shipped_entry_is_a_noop_delete(self):
self._write(self.shipped, ["Foo.Bar() -> void"])
self._write(self.unshipped, ["*REMOVED*Foo.NeverShipped() -> void"])

promote.promote_one(self.unshipped, self.shipped)

self.assertEqual(self._read(self.shipped), {"Foo.Bar() -> void"})

def test_mixed_add_and_remove_in_the_same_promotion(self):
self._write(self.shipped, ["Foo.Old() -> void"])
self._write(self.unshipped, ["*REMOVED*Foo.Old() -> void", "Foo.New() -> void"])

promote.promote_one(self.unshipped, self.shipped)

self.assertEqual(self._read(self.shipped), {"Foo.New() -> void"})


if __name__ == "__main__":
unittest.main()
48 changes: 48 additions & 0 deletions .github/workflows/audit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Dependency audit

# A newly published advisory flags a package without any change here, so the audit runs weekly as well as when
# the dependencies change.
on:
pull_request:
branches: [ master, develop ]
paths:
- '**/*.csproj'
- 'Directory.Build.props'
- 'global.json'
- '.github/workflows/audit.yml'
push:
branches: [ master ]
paths:
- '**/*.csproj'
- 'Directory.Build.props'
- 'global.json'
- '.github/workflows/audit.yml'
schedule:
- cron: '0 6 * * 1'
workflow_dispatch:

permissions:
contents: read

jobs:
audit:
name: Dependency audit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7

- uses: actions/setup-dotnet@v6
with:
global-json-file: global.json

- name: Restore
run: dotnet restore FixedWidthParser.slnx

- name: Check for vulnerable packages
run: |
output=$(dotnet list FixedWidthParser.slnx package --vulnerable --include-transitive)
echo "$output"
if echo "$output" | grep -q -E '\b(High|Critical)\b'; then
echo "::error::High or Critical severity vulnerability found in dependencies."
exit 1
fi
Loading
Loading