✅ Fix tests broken by #2058

[shnizzedy]

Fixes

Fixes #2058 by @shnizzedy
Related to #2048 by @nx10

Description

• Fixes CPAC/utils/__init__.py, which was broken by pre-commit modifications
• Restores TSV and CSV files, which were broken by pre-commit modifications
• Fixes CPAC/nuisance/utils/__init__.py, which was broken by pre-commit modifications
• In autoversioning script, wait for git lock when git adding changed files
• Move entrypoint scripts (run.py and run-with-freesurfer.sh) from /dev/docker_data to CPAC/_entrypoints
• Upgrades print statements to log statements where such statements were relevant to tests
• Moves

  C-PAC/CPAC/utils/utils.py

  Lines 591 to 612 in 2e368b9

  	def check_random_state(seed):
  	"""
  	Turn seed into a np.random.RandomState instance
  	Code from scikit-learn (https://github.com/scikit-learn/scikit-learn).
  	Parameters
  	----------
  	seed : None | int | instance of RandomState
  	If seed is None, return the RandomState singleton used by np.random.
  	If seed is an int, return a new RandomState instance seeded with seed.
  	If seed is already a RandomState instance, return it.
  	Otherwise raise ValueError.
  	"""
  	if seed is None or seed is np.random:
  	return np.random.mtrand._rand
  	if isinstance(seed, (numbers.Integral, np.integer)):
  	return np.random.RandomState(seed)
  	if isinstance(seed, np.random.RandomState):
  	return seed
  	raise ValueError(
  	"%r cannot be used to seed a numpy.random.RandomState" " instance" % seed
  	)

  (which was lifted from scikit-learn) to

  C-PAC/CPAC/utils/sklearn.py

  Lines 40 to 58 in de50ef3

  	def check_random_state(seed: Union[None, int, RandomState]) -> RandomState:
  	"""Turn seed into a np.random.RandomState instance.
  	Parameters
  	----------
  	seed : None | int | instance of RandomState
  	If seed is None, return the RandomState singleton used by np.random.
  	If seed is an int, return a new RandomState instance seeded with seed.
  	If seed is already a RandomState instance, return it.
  	Otherwise raise ValueError.
  	"""
  	if seed is None or seed is np.random:
  	return np.random.mtrand._rand
  	if isinstance(seed, (numbers.Integral, np.integer)):
  	return np.random.RandomState(seed)
  	if isinstance(seed, np.random.RandomState):
  	return seed
  	msg = f"{seed!r} cannot be used to seed a numpy.random.RandomState instance"
  	raise ValueError(msg)

  for clearer licensing and provenance
• Replaced sets.Set with set, which has been built-in since Python 2.4 (November 30, 2004)!
• Deprecates repickle
• Linted cwas files involved in what were failing tests
• Style changes
  • Switches order_by_type from true to false as suggested here and re-runs pre-commit run --all-files as @ChildMindInstituteCNL to apply the sorting change
  • Turns on ruff's autofixable flake8-errmsg (EM) rules

Technical details

CPAC/utils/__init__.py

• Adds used public symbols to CPAC.utils.__init__.__all__
• Removes unused public symbols from CPAC.utils
• Deprecates repickle and lints CPAC.utils.docs while modifying those sections of code

TSVs and CSVs

• Restores TSVs and CSVs from before pre-commit modifications
• Excludes TSVs from trailing-whitespace hook
• Excludes CSVs, pickles and TSVs from end-of-file-fixer hook

CPAC/nuisance/utils/__init__.py

• Moves class and function definitions from CPAC/nuisance/utils/__init__.py to CPAC/nuisance/utils/utils.py
• Imports used public symbols in CPAC/nuisance/utils/__init__.py and adds them to CPAC.utils.__init__.__all__
• Lints both of those files

print statements → log statements

• upgraded in CPAC/_entrypoints/run.py
• upgraded in CPAC/utils/build_data_config.py and linted that file
• updates tests that look for printed statements to look for logged statements
• I'll post a separate PR to do this to untested print statements that were removed by ruff (🎨 print statements → log statements #2062)

Deprecates repickle

• Instead of linting this function and upgrading its print statements, I thought we could just deprecate it, for the reason given in the deprecation message:

  C-PAC/CPAC/utils/utils.py

  Lines 1327 to 1331 in 4650f4f

  	@deprecated(
  	"1.8.7",
  	"Python 2's end of life was over 4 years prior to this release. A user jumping from a C-PAC version that used Python 2 can use this function in any C-PAC version from 1.6.2 up until its removal in an upcoming version.",
  	)
  	def repickle(directory): # noqa: T20

  If we don't want this function deprecated, we can upgrade the function and its tests before we cut 1.8.7.

• TODO: 📝 Document SOP for deprecating code fcp-indi.github.io#320

Linting some cwas files

• The relevant change that was breaking the tests was ruff turned

  C-PAC/CPAC/cwas/cwas.py

  Line 194 in 7bd4afa

  volume[np.where(mask_data == True)] = data

  into

  C-PAC/CPAC/cwas/cwas.py

  Line 203 in 71c9e6f

  volume[np.where(mask_data is True)] = data

  , but np.True_ (array True from NumPy) ≠ True (scalar True from Python), so that broke the reshaping.

  C-PAC/CPAC/cwas/cwas.py

  Line 229 in 4650f4f

  volume[mask_data] = data

  is more Pythonic and equivalent to the original, so that's what is in this PR
• Linted this file and the two test files that caught this error
• Will replace other == True / == False / is True / is Falses in a subsequent PR

Style changes

order_by_type

• I think order_by_type was set to true because I messed up translating my style suggestions here
• If there's opposition to this style change, we can just revert 6c9febb...3b1bacf

flake8-errmsg (EM), f-string-missing-placeholders (F541), f-string (UP032)

• Exception must not use (a string literal|an f-string literal|a .format() string directly), assign to variable first

• f-string without any placeholders

• Use f-string instead of format call

• If there's opposition to this style change, we can just revert 9847b2c...2e368b9

Tests

All the tests that were failing in #2058 are passing here

Checklist

• My pull request has a descriptive title (not a vague title like Update index.md).
• My pull request targets the intial-run/ruff branch of the repository.
• My commit messages follow best practices.
• My code follows the established code style of the repository.
• I added tests for the changes I made (n/a).
• I updated the changelog (n/a).
• I added or updated documentation.
• I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

[shnizzedy]

These weren't used anywhere, so I deleted them.

[shnizzedy]

I think if we want to resurrect this functionality, we can rewrite it at least as easily as we could revive it from the commented-out code, so I deleted the commented-out code.

[shnizzedy]

This function isn't used anywhere except a commented-out section so I deleted both this function and the commented-out code.