ResourceWarning: unclosed <ssl.SSLSocket...

[jedie]

I get some warnings about unclosed SSL socket, looks like:

...
.../utilities/test_utils.py:110: in __exit__
	self.vcr_mock.__exit__(exc_type, exc_val, exc_tb)
site-packages/vcr/cassette.py:105: in __exit__
	next(self.__finish, None)
site-packages/vcr/cassette.py:65: in _patch_generator
	with contextlib.ExitStack() as exit_stack:
/usr/local/lib/python3.10/contextlib.py:561: in __exit__
	if cb(*exc_details):
site-packages/vcr/patch.py:402: in __exit__
	connection = pool.pool.get()
/usr/local/lib/python3.10/warnings.py:109: in _showwarnmsg
	sw(msg.message, msg.category, msg.filename, msg.lineno,
.../settings/warning_handler.py:47: in __call__
	frames = traceback.extract_stack()
WARNING from "test_sdk_client.py" in test_get_category_by_key():
"/usr/local/lib/python3.10/contextlib.py ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=...>"

Any advise to fix that?

[hartwork]

Hi @jedie there are many details missing from your report. Python version seems to be 3.10, version of vcrpy is unclear, user level code doing the requests is unclear, could be requests, urllib3 v1 or v2 etc, is this behavior new or does it happen across all versions of vcrpy etc. Please add more details.

[jedie]

Sorry, here Version information:

• Python 3.10
• VCR.py v4.3.1
• requests v2.31.0
• urllib3 v1.26.16

[hartwork]

Thanks! Is this behavior new or does it happen across all versions of vcrpy?
File test_sdk_client.py was making requests using requests directly or with some layer in between? If so which one?

I think vcrpy does not open SSL connections by itself so I would assume it's either a bug in vcrpy's connection pool code or the library outside of vcrpy that does open SSL connections. I'm guessing here, any help with debugging is appreciated.

[hartwork]

Three weeks without a reproducer, closing as "cannot reproduce". Happy to re-open if someone can provide a minimal producible example. Thanks!

[jedie]

Sorry, i have currently no time to investigate this further... I currently just ignore the warning ;)

[admirito]

Hi @hartwork,

This repo created by @mezhaka has a minimal reproducible example with pytest and here is an even simpler one without pytest:

import warnings
import requests
import vcr


def main():
    session = requests.Session()
    with vcr.use_cassette(path="/dev/null"):
        session.get("http://neverssl.com")
        # session.close()


if __name__ == "__main__":
    warnings.filterwarnings("always", category=ResourceWarning)
    main()

In Python 3.8, with vcrpy 5.1.0 I get a sys:1: ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.1.114', 64833), raddr=('34.223.124.45', 80)>. Even closing the session explicitly with session.close() will not stop the ResourceWarning.

[palfrey]

here is an even simpler one without pytest:

I've been trying to make this one fail locally to try and fix this (and also provide a new unit test for vcrpy) but can't seem to do so. Is there anything else that needs installing? It just seems to work with no failure for me.

[jedie]

I haven't seen this error for a while. So i assume it's fixed. But i have no more details, sorry.

EDIT: I still use vcrpy only with normal unittests (with Django) and not via pytest. Works fine...

[palfrey]

I'm able to reproduce this reliably with current vcrpy master. Adding the connection.close() line from #758 resolves the issue.

Now trying to isolate a test case that isn't my entire closed-source app...

[palfrey]

Reproduction steps

1. Checkout current vcrpy master
2. pip install pytest-vcr
3. Dump the following into "test.py"

import warnings
import pytest
import requests

@pytest.mark.vcr
def test_main():
    warnings.filterwarnings("error", category=ResourceWarning)
    session = requests.Session()
    session.get("http://neverssl.com")

4. rm -Rf cassettes && pytest test.py -vvv -s

That last step reliably gets the following

platform linux -- Python 3.8.14, pytest-7.4.0, pluggy-1.2.0 -- /home/palfrey/.virtualenvs/vcrpy/bin/python
cachedir: .pytest_cache
rootdir: /home/palfrey/src/vcrpy
configfile: pyproject.toml
plugins: vcr-1.0.2, cov-4.1.0, httpbin-2.0.0
collected 1 item

test.py::test_main PASSED

================================================================ warnings summary =================================================================
test.py::test_main
  /home/palfrey/.virtualenvs/vcrpy/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <socket.socket fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>

  Traceback (most recent call last):
    File "/home/palfrey/.pyenv/versions/3.8.14/lib/python3.8/contextlib.py", line 510, in __exit__
      if cb(*exc_details):
  ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.25', 52936), raddr=('34.223.124.45', 80)>

    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

The pytest-vcr could be the issue, but given the connection.close() patch seems to fix this, I doubt it!

[admirito]

here is an even simpler one without pytest:

I've been trying to make this one fail locally to try and fix this (and also provide a new unit test for vcrpy) but can't seem to do so. Is there anything else that needs installing? It just seems to work with no failure for me.

Here are the steps to reproduce the problem in a container to make sure you have exactly the same environment:

$ docker run --rm -it python:3.11 bash

root@container:/# pip install vcrpy==5.1.0 requests==2.31.0
root@container:/# cat > test.py
# copy/paste the code from my previous comment (https://github.com/kevin1024/vcrpy/issues/710#issuecomment-1667240075) and press ctrl-d

root@container:/# python test.py
sys:1: ResourceWarning: unclosed <socket.socket fd=3, family=2, type=1, proto=6, laddr=('172.17.0.2', 50746), raddr=('34.223.124.45', 80)>
ResourceWarning: Enable tracemalloc to get the object allocation traceback

[graingert]

Looks like this was reproduced and there's a PR to fix it. I'm also having the same issue on my test suite

[graingert]

resolved in #811