[alxsimon]

Closes #2467
Uses the .add_note method of exception, introduced in Python 3.11. Also hardcodes the file name in
the raised from exceptions for HDF5 or zip.
Until 3.10 is EOF, needs the sys module.

Description

Would fix #2467

I'll tidy up (changelog and linting) if you think this change is ok.

PR Checklist:

• Tests that fully cover new/changed functionality.
• Documentation including tutorial content if appropriate.
• Changelogs, if there are API changes.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.68%. Comparing base (49eff38) to head (1467142).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3366   +/-   ##
=======================================
  Coverage   91.68%   91.68%           
=======================================
  Files          38       38           
  Lines       32241    32242    +1     
  Branches     5155     5155           
=======================================
+ Hits        29559    29560    +1     
  Misses       2346     2346           
  Partials      336      336           

Flag	Coverage Δ
C	82.26% <ø> (ø)
c-python	77.52% <ø> (ø)
python-tests	96.44% <100.00%> (+<0.01%)	⬆️
python-tests-no-jit	33.29% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Python API	98.73% <100.00%> (+<0.01%)	⬆️
Python C interface	91.25% <ø> (ø)
C library	91.22% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jeromekelleher]

LGTM. Can we add a unit test please, checking that the filename goes into the exception as expected, and validating what happens when we use something that doesn't have a name? (Although, it's not obvious to me how you could have a seekable stream that doesn't have a name, but we need to do due diligence here.)

You can pytest.skip for Python < 310.

@benjeffery shall we just ditch Python 3.10 altogether?

[alxsimon]

LGTM. Can we add a unit test please, checking that the filename goes into the exception as expected, and validating what happens when we use something that doesn't have a name? (Although, it's not obvious to me how you could have a seekable stream that doesn't have a name, but we need to do due diligence here.)

You can pytest.skip for Python < 310.

@benjeffery shall we just ditch Python 3.10 altogether?

Thanks for taking a look, I've trying modifying the existing tests in test_utils.py.
I'm not sure if you'd prefer some dedicated tests that don't touch the existing ones? Especially if we consider keeping some skip for python 3.11 (which I guess would skip each test with the modification for this version of Python).

I could use some pointers on how to test the stream side of things, where should it go and how do you generally do this?

[jeromekelleher]

I think we're going to drop 3.10 ASAP, so let's not worry about the failures on 3.10 CI (and we can remove the conditional at the top). The tests you have look good, updating the existing ones here is fine (although may not need to check the filename on every downstream check for FileFormatError (as that's not what most of the tests are about).

Have a look at tests/test_fileobj.py for what happens in the stream case and see what kind of error messages we get.

[alxsimon]

Thanks, reduced the number of tests of FileFormatError.

In the TestBadStream, the error_text for matching does not seem to be used as modifying it to something that shouldn't be in the error does not fail the test.

For instance:

    def test_bad_stream(self, tmp_path):
        fifo_path = tmp_path / "fifo"
        bad_file_path = tmp_path / "bad_file"
        bad_file_path.write_bytes(b"bad data")
        write_and_read_from_fifo(
            fifo_path, bad_file_path, tskit.FileFormatError, "foo"
        )

Still passes

[jeromekelleher]

I've had a look at this and unfortunately it's really not worth bothering with until we drop Python 3.10 (which will be later this year when it's EOL'd by Python), as it's hard to emulate add_note and we'd have to add a whole bunch of conditionals in the testing code. I'll add a note to the issue for dropping 3.10 to pick this PR up again.

[alxsimon]

thanks for investigating

[hyanwong]

Needs the regex in the test suite to escape windows filenames that often contain backslashes, e.g.

with pytest.raises(
    ...,
    match=f"{filename}...",

should be

with pytest.raises(
    ...,
    match=f"{re.escape(filename)}...",

[alxsimon]

@hyanwong Thanks I've just fixed it for the windows tests.
But I don't know how to fix the codecov.

[petrelharp]

The codecov failure is nothing related to this; it's safe to ignore it. (It's showing missing lines in the C code; I'm not sure why that's showing up here, but weird things happen sometimes.)

[petrelharp]

Okay, I think we can merge! Mind rebasing this to main, @alxsimon, then I'll hit 'merge'?

[alxsimon]

Okay, I think we can merge! Mind rebasing this to main, @alxsimon, then I'll hit 'merge'?

@petrelharp should be good to go, I forgot I was working on a fork so didn't understand why I was not up to date with main... (I'm used to the AGit workflow of codeberg now). This also explains why codecov was behind before.

[petrelharp]

LGTM!