Provide Python bindings for Minpack #49
Conversation
|
Great job, thanks for pushing this! I believe SciPy exclusively uses f2py for the wrappers? So I wonder if we should just use that, in order to be consistent. @HaoZeke, what would you recommend? But to get started, I think it doesn't matter how we do it, as long as it works, and then based on the feedback from the SciPy developers, we can adjust. |
With |
|
Yes, I agree we definitely want a robust C interface (header files) and we might as well use it for the Python wrappers. I know @HaoZeke and I brainstormed this a few times, also in relation to have LFortran's In the meantime, I think doing what you are doing is fine. Let's get the wrappers working, and then we'll have something concrete in our hands, and experience with it, and we can see how to design a solution for long term. |
awvwgk
force-pushed
the
python
branch
2 times, most recently
from
6ff7f51 to
aca4395
Compare
|
Are these binding compatible with numba compiled python? Can you do this: import minpack.library
import numpy as np
from math import sqrt
import numba as nb
@nb.njit
def fcn(x, fvec) -> None:
for k in range(x.size):
tmp = (3.0 - 2.0 * x[k]) * x[k]
tmp1 = x[k - 1] if k > 0 else 0.0
tmp2 = x[k + 1] if k < len(x) - 1 else 0.0
fvec[k] = tmp - tmp1 - 2.0 * tmp2 + 1.0
x = np.array(9 * [-1.0])
fvec = np.zeros(9, dtype=np.float64)
tol = sqrt(np.finfo(np.float64).eps)
@nb.njit
def test():
res = minpack.library.hybrd1(fcn, x, fvec, tol)
test()Compatibility with numba, or some python compiler, would be an improvement over the Scipy bindings. |
awvwgk
force-pushed
the
python
branch
2 times, most recently
from
b398b09 to
37d3ef1
Compare
Seems like this is not possible right now, do you have an idea what I have to change for this to work? |
|
Here is the relevant part of the numba docs: https://numba.pydata.org/numba-doc/latest/reference/pysupported.html?highlight=cffi |
|
The Numba documentation is unfortunately not very useful due to its brevity. I did look a bit into the Numba source code trying to figure out what I'm supposed to do. I added a branch python-numba with some changes, but now I'm kind of stuck. Current error status@Nicholaswogan if you have an idea what is needed to register this function with Numba let me know. Note that I decorated the function |
|
Independent of the Numba compatibility this patch is now finished and ready for review. If we can figure the Numba compatibility out easily, I will add them here as well, but this doesn't affect the overall API design IMO. |
|
I'll play around with your branch and let you know if I figure out anything useful. |
This should be possible in |
A problem no automatic wrapping can solve (at least the one I know of) is that the wrapped functions still look and feel like they are C or Fortran which is usually not really Pythonic. I guess even However, I prefer a route via C due to the general stability of the ABI in C, which is simply not available in Fortran. This becomes especially important when distributing (e.g. when packaging the Python bindings and the Fortran library separately). The main issue in the current API I can see, might be that we have to modify the |
|
I'm confused by how to test the python binding. I'm getting a segmentation fault. I build the library: Then I copy pasted the built Then I added this directory to my runtime path The from the directory import pytest
import minpack.library
import numpy as np
from math import sqrt
def fcn(x, fvec) -> None:
for k in range(x.size):
tmp = (3.0 - 2.0 * x[k]) * x[k]
tmp1 = x[k - 1] if k > 0 else 0.0
tmp2 = x[k + 1] if k < len(x) - 1 else 0.0
fvec[k] = tmp - tmp1 - 2.0 * tmp2 + 1.0
x = np.array(9 * [-1.0])
fvec = np.zeros(9, dtype=np.float64)
tol = sqrt(np.finfo(np.float64).eps)
assert minpack.library.hybrd1(fcn, x, fvec, tol) == 1Segmentation fault occurs on the last line when minpack is called. |
|
Oh never mind. Its because i'm using a M1 mac, and |
Unfortunately that is a common problem when the bindings are not compiled (ctypes, cffi?) unlike Cython or manual C/API. One has to go in, and debug it. One approach is to get a minimal example working on your machine, then slowly adding more complexity until you hit the bug. Probably some C signature incompatibility. |
|
@awvwgk, Current python API would take some work to be compatible with numba. Numba can not call any non-compiled python. So this sort of thing doesn't work, because @nb.njit
def test():
test1()
def test1():
print('Hi')Looking at I put together a wrapper |
Even the C bindings should fail here, because GFortran for MacOS/Arm64 does not support nested function usage. If we decide on implementing #30 I can create an API which also works on MacOS/Arm64 before support in GFortran is implemented. Also, CFFI is compiled, just like Cython or building it manually in C.
Indeed, I need some Python logic to ensure that we don't by accident throw an exception throw the Fortran runtime, as it would leave the stack in an undefined state. I have checked your Edit: Looks like we are almost there, I now have to figure out how to properly register all CFFI procedures Current error status |
|
I updated the description with a bit more context on the implementation of the Python bindings. Let me know if there is anything you are not happy with, since we can refine the interface generation later, I hope nobody considers CFFI as a blocker. |
Exports Python bindings via CFFI, allows installation of Python bindings with meson and setuptools.
Implementation
-Dpython=trueand installed together with minpack library(see
python/minpack/meson.build)(see
python/setup.cfg)MINPACK_CFFIpreprocessorextern "Python"pyton/ffi-builder.py) for handling generation of extension module sourcepython/minpack/meson.build) pipeline usingconfigure_fileto run source generator(see
python/minpack/library.py)(see
python/minpack/typing.py)(see
python/minpack/exception.py)(see
python/minpack/test_library.py)https://github.com/
Provided functions in
minpackPython moduleFollow-up PRs
Known issues
See #43