Symbol not found: ___darwin_check_fd_set_overflow

[TimothyStephens]

Hi,

I am encountering the below error when trying to run higlass-manage view using the test dataset.
I am running it through command line on a Mac (Mojave V10.14.6).

$ higlass-manage view sample.mcool
Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/bin/higlass-manage", line 5, in <module>
    from higlass_manage.cli import cli
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/cli.py", line 3, in <module>
    from higlass_manage.ingest import ingest
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/ingest.py", line 2, in <module>
    import clodius.cli.aggregate as cca
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/clodius/cli/__init__.py", line 18, in <module>
    from . import (  # noqa: F401
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/clodius/cli/convert.py", line 11, in <module>
    import bbi
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/__init__.py", line 1, in <module>
    from .cbbi import (
ImportError: dlopen(/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/cbbi.cpython-37m-darwin.so, 2): Symbol not found: ___darwin_check_fd_set_overflow
  Referenced from: /Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/.dylibs/libcrypto.3.dylib (which was built for Mac OS X 11.0)
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/bbi/.dylibs/libcrypto.3.dylib

Here is my conda environment

$ conda list
# packages in environment at /Users/timothystephens/opt/miniconda2/envs/highglass-manager:
#
# Name                    Version                   Build  Channel
asciitree                 0.3.3                    pypi_0    pypi
blas                      1.0                         mkl  
bzip2                     1.0.8                h1de35cc_0  
c-ares                    1.18.1               hca72f7f_0  
ca-certificates           2021.10.26           hecd8cb5_2  
cached-property           1.5.2                      py_0  
certifi                   2021.10.8        py37hecd8cb5_2  
charset-normalizer        2.0.12                   pypi_0    pypi
click                     8.0.3                    pypi_0    pypi
clodius                   0.18.0                   pypi_0    pypi
cloudpickle               2.0.0                    pypi_0    pypi
cooler                    0.8.11                   pypi_0    pypi
curl                      7.80.0               hca72f7f_0  
cytoolz                   0.10.1                   pypi_0    pypi
dask                      2021.12.0                pypi_0    pypi
dill                      0.3.4                    pypi_0    pypi
docker                    5.0.3                    pypi_0    pypi
fsspec                    2022.1.0                 pypi_0    pypi
h5py                      3.6.0            py37h4a1dd59_0  
hdf5                      1.10.6               hdbbcd12_0  
higlass-manage            0.8.0                    pypi_0    pypi
idna                      3.3                      pypi_0    pypi
importlib-metadata        4.11.1                   pypi_0    pypi
intel-openmp              2021.4.0          hecd8cb5_3538  
krb5                      1.19.2               hcd88c3b_0  
libcurl                   7.80.0               h6dfd666_0  
libcxx                    12.0.0               h2f01273_0  
libdeflate                1.0                  h1de35cc_1    bioconda
libedit                   3.1.20210910         hca72f7f_0  
libev                     4.33                 h9ed2024_1  
libffi                    3.3                  hb1e8313_2  
libgfortran               3.0.1                h93005f0_2  
libnghttp2                1.46.0               ha29bfda_0  
libssh2                   1.9.0                ha12b0ac_1  
locket                    0.2.1                    pypi_0    pypi
mkl                       2021.4.0           hecd8cb5_637  
mkl-service               2.4.0            py37h9ed2024_0  
mkl_fft                   1.3.1            py37h4ab4a9b_0  
mkl_random                1.2.2            py37hb2f4e1b_0  
multiprocess              0.70.12.2                pypi_0    pypi
ncurses                   6.3                  hca72f7f_2  
negspy                    0.2.24                   pypi_0    pypi
nose                      1.3.7                    pypi_0    pypi
numpy                     1.21.5                   pypi_0    pypi
numpy-base                1.21.2           py37he0bd621_0  
openssl                   1.1.1m               hca72f7f_0  
packaging                 21.3                     pypi_0    pypi
pandas                    1.0.4                    pypi_0    pypi
partd                     1.2.0                    pypi_0    pypi
pip                       21.2.2           py37hecd8cb5_0  
pybbi                     0.3.2                    pypi_0    pypi
pydantic                  1.9.0                    pypi_0    pypi
pyfaidx                   0.6.4                    pypi_0    pypi
pypairix                  0.3.7                    pypi_0    pypi
pyparsing                 3.0.7                    pypi_0    pypi
pysam                     0.18.0                   pypi_0    pypi
python                    3.7.11               h88f2d9e_0  
python-dateutil           2.8.2                    pypi_0    pypi
pytz                      2021.3                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.1.2                hca72f7f_1  
requests                  2.27.1                   pypi_0    pypi
scipy                     1.7.3                    pypi_0    pypi
setuptools                58.0.4           py37hecd8cb5_0  
simplejson                3.17.6                   pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1  
slugid                    2.0.0                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
sqlite                    3.37.2               h707629a_0  
tk                        8.6.11               h7bc2e8c_0  
toolz                     0.11.2                   pypi_0    pypi
tqdm                      4.62.3                   pypi_0    pypi
typing-extensions         4.1.1                    pypi_0    pypi
urllib3                   1.26.8                   pypi_0    pypi
websocket-client          1.2.3                    pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0  
xz                        5.2.5                h1de35cc_0  
zipp                      3.7.0                    pypi_0    pypi
zlib                      1.2.11               h4dc903c_4  

Thanks for your help.
Tim.

[pkerpedjiev]

Ummm, I've never seen that before. Perhaps @nvictus has an idea since it's the pybbi package complaining?

[nvictus]

Something is up with the latest pybbi binaries for MacOS. The wheel for v0.3.2 was supposed to have been built on a 10.9 system, but the error message says that some dependencies were built on OS X 11.

Tim, could you try downgrading pybbi to version 0.3.1 or 0.3.0?

[TimothyStephens]

I downgraded pybbi to version 0.3.0 and now get a different error.

$ higlass-manage view sample.mcool
Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/view.py", line 78, in view
    temp_dir = get_temp_dir(hg_name)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/common.py", line 268, in get_temp_dir
    client = docker.from_env()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/client.py", line 101, in from_env
    **kwargs_from_env(**kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 222, in _retrieve_server_version
    f'Error while fetching server API version: {e}'
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/adapters.py", line 450, in send
    timeout=timeout
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 786, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/bin/higlass-manage", line 8, in <module>
    sys.exit(cli())
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/view.py", line 81, in view
    _start(hg_name=hg_name)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/higlass_manage/start.py", line 177, in _start
    client = docker.from_env()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/client.py", line 101, in from_env
    **kwargs_from_env(**kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/Users/timothystephens/opt/miniconda2/envs/highglass-manager/lib/python3.7/site-packages/docker/api/client.py", line 222, in _retrieve_server_version
    f'Error while fetching server API version: {e}'
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

[nvictus]

That seems to be a docker issue. Do you have docker installed and running on your Mac?

[TimothyStephens]

You are right. I didn't have docker correctly installed on my Mac. After fixing this higlass-manage now runs on my system. Thank you for all of your help.