Bump dependencies for NumPy 2 compatibility #511

cbrnr · 2024-10-23T07:31:15Z

All tests pass and demo.ipynb works without any errors or warnings using NumPy >= 2.

cbrnr · 2024-10-23T08:12:32Z

OK, I'm struggling a bit with dependencies here. If you want to keep Python >= 3.9 and at the same time also support 3.13, scipy >= 1.13.1 works only for Python <= 3.12. Strangely, uv does not use scipy 1.14.1 (there is no binary wheel for scipy 1.13.1 for Python 3.13) for Python 3.13, but insists on using 1.13.1...

bemoody · 2024-10-24T19:28:10Z

Sure, seems like the problem is that "uv run --extra dev pytest" is doing something bizarre.

I don't know exactly what that command is doing or trying to do, but it seems to be trying to install the oldest versions of packages that satisfy our dependencies. That is not what we want (that's not what any normal user does) and is not likely to lead to a good solution.

cbrnr · 2024-10-24T19:51:27Z

It might actually be something at the core of how uv resolves dependencies, yes. I've opened an issue here: astral-sh/uv#8492

YmirKhang · 2024-10-24T20:19:45Z

Hells yeah!

cbrnr · 2024-11-18T15:17:00Z

Hmmm, the Windows 3.13 job crashed, but this might be unrelated. @bemoody or @tompollard could you please restart this job (I don't have the permissions to do that myself)?

cbrnr · 2024-11-19T08:05:03Z

We're hitting another issue, this time related to CPython (python/cpython#125235). We gotta wait for 3.13.1 for the fix.

cbrnr · 2024-11-19T09:37:51Z

This is ready for review. Everything works with NumPy >= 2. I've removed support for Python 3.8 and added Python 3.12. Python 3.13 can be added as soon as 3.13.1 is released, but I would do that in a separate PR. It would be great if you could push out a new release to PyPI once this PR gets merged.

cbrnr · 2024-11-19T10:01:32Z

Note that we could also force specific NumPy versions for each Python version as shown e.g. here (they are testing the oldest supported NumPy version for each Python release).

The NumPy >= 2 requirement is currently only necessary for the test, because otherwise these would use 1.26.4.

WDYT?

bemoody · 2024-11-21T17:51:35Z

I appreciate your efforts to try to get to the bottom of these issues!

However, I don't think wfdb package should require numpy 2 at this stage, and I'm not inclined to change that just to appease uv.

Furthermore, before removing the "numpy < 2" requirement from pyproject.toml, I would like to see that the test suite is able to run on numpy 2, with NPY_PROMOTION_STATE set to "weak_and_warn", and with no warnings generated. Currently that's not the case.

That aside, we really need to have a CI workflow that tests whether the package can be installed using pip. If uv doesn't install exactly the same packages that pip would have installed, then uv is not an effective substitute.

cbrnr · 2024-11-21T18:26:40Z

However, I don't think wfdb package should require numpy 2 at this stage, and I'm not inclined to change that just to appease uv.

Of course, that's why I suggested to pin several NumPy versions (e.g., the oldest supported NumPy version per Python version).

Furthermore, before removing the "numpy < 2" requirement from pyproject.toml, I would like to see that the test suite is able to run on numpy 2, with NPY_PROMOTION_STATE set to "weak_and_warn", and with no warnings generated. Currently that's not the case.

Sure, I can do that!

That aside, we really need to have a CI workflow that tests whether the package can be installed using pip. If uv doesn't install exactly the same packages that pip would have installed, then uv is not an effective substitute.

Yes, we can use the uv pip interface instead, which behaves identically to pip. Although Astral devs are currently working hard to implement a resolver option to uv run/sync, which also behaves like pip, but I don't know how long it will take them. We can immediately switch to uv pip though.

cbrnr · 2024-11-21T18:27:32Z

If you are completely unhappy with uv, I can also get rid of it again and replace it with pip, just let me know.

bemoody · 2024-11-21T21:47:18Z

Thanks!

Of course, that's why I suggested to pin several NumPy versions (e.g., the oldest supported NumPy version per Python version).

Yeah, that seems like a reasonable workaround as long as it's not too much work.

Yes, we can use the uv pip interface instead, which behaves identically to pip.

Sounds like that's the best thing for now.

cbrnr · 2024-11-26T11:33:22Z

OK, I'd really appreciate if someone took a look. I've now enabled weak_and_warn, and the Python 3.9 job on Ubuntu uses numpy==1.26.4 (all other jobs use numpy>=2). One test fails, which I cannot reproduce locally (neither on macOS nor on Linux).

The Windows jobs all emit a warning related to tk (see here), but I think these warnings have always been there (not a result of this PR).

Another question that came up: is the Debian i386 job still needed? It runs the tests with Python 3.7, which is not supported anymore.

Finally, and this is a question for when all other issues have been resolved, the test matrix is rather large. Is it really necessary to test on all combinations of OSs and Python versions?

cbrnr · 2024-12-17T07:57:40Z

Since uv has a new solver which now by default behaves like pip (astral-sh/uv#9868), I'll revert to using uv sync and uv run again (will take a little time though). I'd appreciate any responses to my questions, because these are not affected by the resolution strategy.

cbrnr · 2025-01-15T09:16:47Z

Alright, this seems to be working now. @bemoody @tompollard it would be great if you could take a look and merge if you're happy with the changes. A quick follow-up release with these changes would then be super useful too! Thanks!

tompollard

Thanks @cbrnr, just added a few quick comments.

wfdb/processing/peaks.py

tompollard · 2025-01-17T18:32:31Z

wfdb/io/convert/edf.py

@@ -399,7 +399,7 @@ def read_edf(
        }

    sig_data = np.empty((sig_len, n_sig))
-    temp_sig_data = np.fromfile(edf_file, dtype=np.int16)
+    temp_sig_data = np.fromfile(edf_file, dtype=np.int16).astype("int64")


edf.py was updated in #527 so this change is not needed.

.github/workflows/test.yml

tompollard · 2025-01-21T13:45:16Z

Looks great, thanks again for your help @cbrnr!

wfdb/processing/peaks.py

@ajadczaksunriselabs

This pull request adds a changelog for `v4.2.0`. The changelog is based on the following auto-generated summary of merge commits generated by GitHub: ``` ## What's Changed * bug-fix: Numpy ValueError when cheking empty list equality by @ajadczaksunriselabs in #459 * bug-fix: Pandas set indexing error by @ajadczaksunriselabs in #460 * fix for /issues/452 by @tecamenz in #465 * Use numpydoc to render documentation by @SnoopJ in #472 * build(deps): bump readthedocs-sphinx-search from 0.1.1 to 0.3.2 in /docs by @dependabot in #477 * Update style by @bemoody in #482 * Fix NaN handling in Record.adc, and other fixes by @bemoody in #481 * Set upper bound on Numpy version (numpy = ">=1.10.1,<2.0.0"). Ref #493. by @tompollard in #494 * Update actions to use actions/checkout@v3 and actions/setup-python@v4. by @tompollard in #495 * Fix: Indent code to ensure 'j' is within for-loop in GQRS algorithm by @tompollard in #499 * Add write_dir argument to csv_to_wfdb. Fixes #67. by @tompollard in #492 * Fix warnings by @cbrnr in #502 * README improvements by @bemoody in #503 * Change in type promotion. Fixes to annotation.py by @tompollard in #506 * Use uv by @cbrnr in #504 * Change in type promotion. Fixes to _signal.py by @tompollard in #507 * Test round-trip write/read of supported binary formats by @bemoody in #509 * Corrected typo and extended allowed types for MultiSegmentRecord by @agent3gatech in #514 * Allow expanded physical signal in `calc_adc_params` by @briangow in #512 * Add capability to write signal with unique `samps_per_frame` to `wfdb.io.wrsamp` by @briangow in #510 * Fix selection of channels when converting to EDF by @SamJelfs in #519 * Change in type promotion introduced in Numpy 2.0. Fixes to edf.py. by @tompollard in #527 * Bump dependencies for NumPy 2 compatibility by @cbrnr in #511 * Bump version to v4.2.0 and update notes on creating new releases by @tompollard in #497 ## New Contributors * @ajadczaksunriselabs made their first contribution in #459 * @tecamenz made their first contribution in #465 * @SnoopJ made their first contribution in #472 * @dependabot made their first contribution in #477 * @agent3gatech made their first contribution in #514 * @SamJelfs made their first contribution in #519 **Full Changelog**: v4.1.2...v4.2.0 ```

tompollard · 2025-01-21T21:59:51Z

v4.2.0 is now on PyPI: https://pypi.org/project/wfdb/

cbrnr mentioned this pull request Oct 23, 2024

OverflowError: Python integer 256 out of bounds for uint8 #493

Open

cbrnr marked this pull request as draft October 23, 2024 08:09

cbrnr force-pushed the numpy2 branch from aa63348 to 3445bef Compare November 18, 2024 12:32

cbrnr mentioned this pull request Nov 18, 2024

Introduce a --multi-version preference mode astral-sh/uv#8686

Closed

cbrnr marked this pull request as ready for review November 19, 2024 09:34

cbrnr mentioned this pull request Dec 5, 2024

Use uv for project management xdf-modules/pyxdf#125

Closed

briangow mentioned this pull request Dec 19, 2024

wrsamp: IndexError: Channel 0 contains values outside the allowed range [-32768, 32767] for fmt 16 #521

Open

larsoner mentioned this pull request Jan 14, 2025

Python 3.13 roadmap cbrnr/sleepecg#235

Open

3 tasks

cbrnr mentioned this pull request Jan 17, 2025

Add support for Python 3.13 cbrnr/sleepecg#241

Open

tompollard mentioned this pull request Jan 17, 2025

Change in type promotion introduced in Numpy 2.0. Fixes to edf.py. #527

Merged

tompollard reviewed Jan 17, 2025

View reviewed changes

cbrnr added 4 commits January 18, 2025 09:23

Bump dependencies for NumPy 2 compatibility

bdc7c7f

Fix CI

a7ab06f

Bump scipy for Python 3.13

03be479

Bump Python to 3.10

ebfa756

cbrnr added 22 commits January 18, 2025 09:23

Test single OS

3acea30

Fix name

cb7b529

Remove quotes

96c2e3b

Forgot $

14eaa44

Try single command

100aaed

Simplify matrix

556a4d3

Add OS name

b3a5498

Improve name

25b75c1

Revert

802fd3a

Try another variant

9e8136d

OK forget it

a9c7b74

Only test NumPy 1.26.4 on Python 3.9

e2692f2

Try with condition

c8ba35e

Use weak_and_warn

ab64860

Don't fail fast

ed25100

Fix syntax

96a52bb

Fix invalid escape sequence

ba4e1ab

Fix dtype promotion warning

102e60c

Try again

0d2a505

Maybe better

99c7f7b

Use uv run again

fd12ec1

Better name

397c09d

cbrnr force-pushed the numpy2 branch from a33a06c to 397c09d Compare January 18, 2025 08:24

Revert

ec5e24f

tompollard approved these changes Jan 21, 2025

View reviewed changes

wfdb/processing/peaks.py Show resolved Hide resolved

tompollard merged commit ca9c7f3 into MIT-LCP:main Jan 21, 2025
18 checks passed

cbrnr deleted the numpy2 branch January 21, 2025 14:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump dependencies for NumPy 2 compatibility #511

Bump dependencies for NumPy 2 compatibility #511

cbrnr commented Oct 23, 2024

cbrnr commented Oct 23, 2024

bemoody commented Oct 24, 2024

cbrnr commented Oct 24, 2024 •

edited

Loading

YmirKhang commented Oct 24, 2024

cbrnr commented Nov 18, 2024

cbrnr commented Nov 19, 2024 •

edited

Loading

cbrnr commented Nov 19, 2024

cbrnr commented Nov 19, 2024 •

edited

Loading

bemoody commented Nov 21, 2024

cbrnr commented Nov 21, 2024

cbrnr commented Nov 21, 2024

bemoody commented Nov 21, 2024

cbrnr commented Nov 26, 2024 •

edited

Loading

cbrnr commented Dec 17, 2024 •

edited

Loading

cbrnr commented Jan 15, 2025

tompollard left a comment

tompollard Jan 17, 2025

cbrnr Jan 18, 2025

tompollard commented Jan 21, 2025

tompollard commented Jan 21, 2025

Bump dependencies for NumPy 2 compatibility #511

Bump dependencies for NumPy 2 compatibility #511

Conversation

cbrnr commented Oct 23, 2024

cbrnr commented Oct 23, 2024

bemoody commented Oct 24, 2024

cbrnr commented Oct 24, 2024 • edited Loading

YmirKhang commented Oct 24, 2024

cbrnr commented Nov 18, 2024

cbrnr commented Nov 19, 2024 • edited Loading

cbrnr commented Nov 19, 2024

cbrnr commented Nov 19, 2024 • edited Loading

bemoody commented Nov 21, 2024

cbrnr commented Nov 21, 2024

cbrnr commented Nov 21, 2024

bemoody commented Nov 21, 2024

cbrnr commented Nov 26, 2024 • edited Loading

cbrnr commented Dec 17, 2024 • edited Loading

cbrnr commented Jan 15, 2025

tompollard left a comment

Choose a reason for hiding this comment

tompollard Jan 17, 2025

Choose a reason for hiding this comment

cbrnr Jan 18, 2025

Choose a reason for hiding this comment

tompollard commented Jan 21, 2025

tompollard commented Jan 21, 2025

cbrnr commented Oct 24, 2024 •

edited

Loading

cbrnr commented Nov 19, 2024 •

edited

Loading

cbrnr commented Nov 19, 2024 •

edited

Loading

cbrnr commented Nov 26, 2024 •

edited

Loading

cbrnr commented Dec 17, 2024 •

edited

Loading