Debian patched python 3.12 (or stock python 3.14.0a2) fails testing #514
Comments
|
I stumbled on the problem too and other Debian users too: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087455 @yarikoptic you seem to imply that the patches in Debian Python packaging make a difference, did you test on other systems with the same Python version to confirm? (I'll try to see if I can access other systems to confirm too) |
|
Not 100% since I don't remember if I tested against vanilla Python release. But we run tests regularly against 3.12 in conda and related test did not fail until I looked into updating Debian package. We had another test/code also triggered by a patch in Debian package which was picked up from some 3.14a IIRC |
FWIW -- with vanilla build of 3.13.0 no such error -- just 1 failing test❯ ./test/test.py -f
...................................usage: test.py [-h] --foo FOO {bar}
test.py: error: too few arguments
usage: test.py [-h] --foo FOO {bar}
test.py: error: argument --foo is required
...usage: test.py foo [-h] {bar}
test.py foo: error: too few arguments
......x...x.......................x...x.........F
======================================================================
FAIL: test_redirection_completion (__main__.TestBashGlobal.test_redirection_completion)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 1342, in test_redirection_completion
self.assertEqual(msg, "success\n")
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
AssertionError: 'failure\n' != 'success\n'
- failure
+ success
----------------------------------------------------------------------
Ran 87 tests in 85.974s
FAILED (failures=1, expected failures=4)
./test/test.py -f 51.82s user 4.85s system 65% cpu 1:26.85 total
❯ which python
/home/yoh/proj/misc/argcomplete/venv/3.13.0/bin/python
❯ python --version
Python 3.13.0
similarly with fresh vanilla build of 3.12.7fi
./python -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform
The necessary bits to build these optional modules were not found:
_dbm _gdbm _tkinter
nis
To find the necessary bits, look in configure.ac and config.log.
Checked 111 modules (31 built-in, 75 shared, 1 n/a on linux-x86_64, 0 disabled, 4 missing, 0 failed on import)
make -j 8 107.41s user 7.11s system 547% cpu 20.927 total
❯
❯ ln -s python ./python3.12.7
❯ ln -s python ./python3.12
❯ ln -s python ./python3
❯ export PATH=/home/yoh/proj/misc/cpython-3.12.7:$PATH
❯ cd -2
~/proj/misc/argcomplete
Authors.rst LICENSE.rst Makefile README.rst argcomplete/ common.mk docs/ setup.cfg test/
Changes.rst MANIFEST.in NOTICE SECURITY.md argcomplete.egg-info/ contrib/ pyproject.toml setup.py* venv/
❯ py=3.12.7; d=venv/$py; python$py -m venv $d && source $d/bin/activate && pip install -e . && pip install pexpect && ./test/test.py -f
Obtaining file:///home/yoh/proj/misc/argcomplete
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: argcomplete
Building editable for argcomplete (pyproject.toml) ... done
Created wheel for argcomplete: filename=argcomplete-3.5.1-0.editable-py3-none-any.whl size=12819 sha256=b774e0b7d0fbe9a9392a050a5762858d6b801c83e949c536176af32d62ecdd13
Stored in directory: /home/yoh/.tmp/pip-ephem-wheel-cache-4j3avk04/wheels/06/87/cf/8a90c55a9e84ae1b6cb791c26deb6b39eb9518210d3a5e12e4
/home/yoh/proj/misc/argcomplete/venv/3.12.7/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
return cls(files, info_location)
Successfully built argcomplete
Installing collected packages: argcomplete
Successfully installed argcomplete-3.5.1
[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
Collecting pexpect
Using cached pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting ptyprocess>=0.5 (from pexpect)
Using cached ptyprocess-0.7.0-py2.py3-none-any.whl.metadata (1.3 kB)
Using cached pexpect-4.9.0-py2.py3-none-any.whl (63 kB)
Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
/home/yoh/proj/misc/argcomplete/venv/3.12.7/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
return cls(files, info_location)
Installing collected packages: ptyprocess, pexpect
/home/yoh/proj/misc/argcomplete/venv/3.12.7/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
return cls(files, info_location)
Successfully installed pexpect-4.9.0 ptyprocess-0.7.0
[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
...................................usage: test.py [-h] --foo FOO {bar}
test.py: error: too few arguments
usage: test.py [-h] --foo FOO {bar}
test.py: error: argument --foo is required
...usage: test.py foo [-h] {bar}
test.py foo: error: too few arguments
......x...x.......................x...x.........F
======================================================================
FAIL: test_redirection_completion (__main__.TestBashGlobal.test_redirection_completion)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 1342, in test_redirection_completion
self.assertEqual(msg, "success\n")
AssertionError: 'failure\n' != 'success\n'
- failure
+ success
----------------------------------------------------------------------
Ran 87 tests in 65.577s
FAILED (failures=1, expected failures=4)
./test/test.py -f 32.20s user 4.25s system 54% cpu 1:06.37 total
and when I try current main of cpython (v3.14.0a2-70-ga4d4c1ede21) built with "nogil" support -- I get this error. So it is, as I hinted, some patches in debian picked up from current "main" of cpythonChecked 112 modules (34 built-in, 74 shared, 1 n/a on linux-x86_64, 0 disabled, 3 missing, 0 failed on import)
make -j 10 269.65s user 29.03s system 1221% cpu 24.444 total
❯ ln -s python ./python3
❯ ln -s python ./python3.14.0a2+nogil
❯ ./python --version
Python 3.14.0a2+
❯ cd -
❯ export PATH=/home/yoh/proj/misc/cpython-3.14.0a2+70+nogil:$PATH
❯ cd ../argcomplete
Authors.rst LICENSE.rst Makefile README.rst argcomplete/ common.mk docs/ setup.cfg test/
Changes.rst MANIFEST.in NOTICE SECURITY.md argcomplete.egg-info/ contrib/ pyproject.toml setup.py* venv/
❯ py=3.14.0a2+nogil; d=venv/$py; python$py -m venv $d && source $d/bin/activate && pip install -e . && pip install pexpect && ./test/test.py -f
python$py -m venv $d 6.66s user 0.13s system 99% cpu 6.798 total
Obtaining file:///home/yoh/proj/misc/argcomplete
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: argcomplete
Building editable for argcomplete (pyproject.toml) ... done
Created wheel for argcomplete: filename=argcomplete-3.5.1-0.editable-py3-none-any.whl size=12819 sha256=95ac218a96599214a1b2f48fa3355b3c0dd01fe19ce3bff3999ab866ae239470
Stored in directory: /home/yoh/.tmp/pip-ephem-wheel-cache-48al5mba/wheels/ac/3d/d9/e3290233d2be26ecfe28f8109144c11fdc602a93e9a7b7adb5
/home/yoh/proj/misc/argcomplete/venv/3.14.0a2+nogil/lib/python3.14t/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
return cls(files, info_location)
Successfully built argcomplete
Installing collected packages: argcomplete
Successfully installed argcomplete-3.5.1
pip install -e . 7.44s user 0.23s system 95% cpu 8.007 total
Collecting pexpect
Using cached pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting ptyprocess>=0.5 (from pexpect)
Using cached ptyprocess-0.7.0-py2.py3-none-any.whl.metadata (1.3 kB)
Using cached pexpect-4.9.0-py2.py3-none-any.whl (63 kB)
Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
/home/yoh/proj/misc/argcomplete/venv/3.14.0a2+nogil/lib/python3.14t/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
return cls(files, info_location)
Installing collected packages: ptyprocess, pexpect
/home/yoh/proj/misc/argcomplete/venv/3.14.0a2+nogil/lib/python3.14t/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
return cls(files, info_location)
Successfully installed pexpect-4.9.0 ptyprocess-0.7.0
E
======================================================================
ERROR: test_action_activation (__main__.TestArgcomplete.test_action_activation)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 250, in test_action_activation
self.assertEqual(set(self.run_completer(make_parser(), cmd)), set(output))
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 139, in run_completer
completer(parser, output_stream=t, exit_method=sys.exit, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 174, in __call__
completions = self._get_completions(comp_words, cword_prefix, cword_prequote, last_wordbreak_pos)
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 223, in _get_completions
completions = self.collect_completions(active_parsers, parsed_args, cword_prefix)
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 440, in collect_completions
completions += self._get_option_completions(active_parser, cword_prefix)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 326, in _get_option_completions
if not self._action_allowed(action, parser):
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 336, in _action_allowed
for conflict_action in parser._action_conflicts.get(action, []):
^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MonkeyPatchedIntrospectiveArgumentParser' object has no attribute '_action_conflicts'
----------------------------------------------------------------------
Ran 1 test in 0.005s
FAILED (errors=1)
|
|
Thanks for reporting this - I can keep this issue open to track solutions but I am not involved with nor have insight into Debian maintenance and patches for argcomplete, so I'm not going to be able to diagnose or resolve this issue directly. |
|
@kislyuk as I have hinted above - issue is reproducible with "stock" python 3.14.0a2 -- so there is high change that this issue will become relevant to upcoming 3.14.0 release of Python... You can diagnose by building stock python from http://github.com/python/cpython (very easy -- I will adjust title to reflect that. |
yarikoptic
changed the title
|
@yarikoptic it looks like Debian (or Ubuntu?) has chosen to patch stable Python in ways that make it incompatible with the upstream Python distribution. I don't have control over those patches and I can't support argcomplete's compatibility with them. I would be open to someone linking and explaining to me what exactly was patched and why, providing a fix, and setting up CI to regression test the fix for future use. To be clear, all of my tests with Python 3.13 (which is the latest released version) indicate that there are no errors (and the same is true for Python 3.12). If you see errors with a released version of Python, please advise and give steps to reproduce, and we can follow up. I understand I can build alpha versions of Python myself and test with those. I don't currently have the time to do that, so I prefer to wait until at least the release candidate stage. |
|
Per the discussion in #513, the patches in question were released 2 days ago by upstream Python in 3.12.8/3.13.1, so I'm going to go ahead and test with those as soon as I can. |
Fair enough! So please just treat this issue as a "pro-active awareness maker" on what to be on lookout for with 3.14rc. But maybe someone could see a quick fix and thus make argcomplete meat rc "ready". (FWIW: I personally prefer to be proactive, so e.g. we maintain https://github.com/datalad/git-annex where we build/test our datalad daily against development version of git-annex, often addressing issues which popup in git-annex, or making sure that released datalad is ready for upcoming breaking changes...) |
|
I totally get that and am open to doing that in this project; I just don't currently have the time to do it myself. There are a couple of other issues open and #513 offers an (incomplete/not fully documented) patch that I will test and merge as soon as I can. If someone wants to set up CI for this project to test against alpha releases of Python, I'm open to merging such PRs. |
|
apparently at large pre-releases are already allowed, it might be just a matter of having 3.14 not yet added |
|
Commit that introduced all relevant changes: python/cpython@759a54d |
|
OK, I released argcomplete v3.5.2 with a fix to restore compatibility. Please let me know if you see any further issues. |
|
FWIW, confirming that the issue is resolved for me with a debian python: full dump of before/after❯ ./test/test.py -f
E
======================================================================
ERROR: test_action_activation (__main__.TestArgcomplete.test_action_activation)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 250, in test_action_activation
self.assertEqual(set(self.run_completer(make_parser(), cmd)), set(output))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 139, in run_completer
completer(parser, output_stream=t, exit_method=sys.exit, **kwargs)
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 174, in __call__
completions = self._get_completions(comp_words, cword_prefix, cword_prequote, last_wordbreak_pos)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 223, in _get_completions
completions = self.collect_completions(active_parsers, parsed_args, cword_prefix)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 440, in collect_completions
completions += self._get_option_completions(active_parser, cword_prefix)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 326, in _get_option_completions
if not self._action_allowed(action, parser):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 336, in _action_allowed
for conflict_action in parser._action_conflicts.get(action, []):
^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MonkeyPatchedIntrospectiveArgumentParser' object has no attribute '_action_conflicts'
----------------------------------------------------------------------
Ran 1 test in 0.003s
FAILED (errors=1)
~/proj/misc/argcomplete develop ?1 ▓▒░─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────░▒▓ 02:37:4❯ git pff
remote: Enumerating objects: 21, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 21 (delta 5), reused 20 (delta 5), pack-reused 0 (from 0)
Unpacking objects: 100% (21/21), 426.11 KiB | 2.42 MiB/s, done.
From https://github.com/kislyuk/argcomplete
ed99165..67f559d develop -> origin/develop
+ e6cfe8a...13036f9 gh-pages -> origin/gh-pages (forced update)
* [new tag] v3.5.2 -> v3.5.2
Updating ed99165..67f559d
Fast-forward
.github/workflows/ci.yml | 2 +-
Changes.rst | 9 +++++++++
argcomplete/packages/_argparse.py | 2 +-
3 files changed, 11 insertions(+), 2 deletions(-)
❯ ./test/test.py -f
...................................usage: test.py [-h] --foo FOO {bar}
test.py: error: too few arguments
usage: test.py [-h] --foo FOO {bar}
test.py: error: argument --foo is required
...usage: test.py foo [-h] {bar}
test.py foo: error: too few arguments
......x...x.......................x...x.........F
======================================================================
FAIL: test_redirection_completion (__main__.TestBashGlobal.test_redirection_completion)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 1342, in test_redirection_completion
self.assertEqual(msg, "success\n")
AssertionError: 'failure\n' != 'success\n'
- failure
+ success
----------------------------------------------------------------------
Ran 87 tests in 71.927s
FAILED (failures=1, expected failures=4)
./test/test.py -f 34.66s user 7.49s system 58% cpu 1:12.60 total
~/proj/misc/argcomplete develop ?1 ▓▒░──────────────────────────and there is only a failing |
|
thanks, @kislyuk and @yarikoptic for sorting this out! |
There is a good number of patches (motivation is unknown to me, didn't track) are in Debian's python 3.12 package. See https://sources.debian.org/patches/python3.12/3.12.7-3/ . Testing of our https://github.com/datalad/datalad/ package started to fail on 3.12 (passed ok before IIRC), and it seems to manifest also in running argcomplete tests:
if you don't have a debian system, but podman or docker do smth like
and inside
The text was updated successfully, but these errors were encountered: