Explicitly close UploadFile once request is complete for Python 3.11 #1967

phillipuniverse · 2022-11-29T00:36:01Z

phillipuniverse
Nov 29, 2022

I was upgrading a FastAPI project (0.88) to Python 3.11. I have warnings turned on to raise errors in pytest. When running my code in Python 3.11 I get the following exception:

E               pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function SpooledTemporaryFile.__del__ at 0x10639e340>
E               
E               Traceback (most recent call last):
E                 File "/Users/phillip/.pyenv/versions/3.11.0/lib/python3.11/tempfile.py", line 732, in __del__
E                   _warnings.warn(
E               ResourceWarning: Unclosed file <tempfile.SpooledTemporaryFile object at 0x10c88b790>

With an endpoint that looks like:

app = FastAPI()

@app.post("/upload")
async def upload(files: Sequence[UploadFile] = File([]))
    pass

And a test like:

from httpx import AsyncClient

async def test_uploads(AsyncClient client):
    file = Path(__file__).with_name("testfile.pdf").open("rb")
    response = await client.post(
        "/freight/invoices",
        files=[("files", file)]
    )
    file.close()

Debugging into this a bit, I see where the UploadFile is created from the multipart request

starlette/starlette/formparsers.py

Lines 232 to 236 in 88e9fc1

    
           file = UploadFile( 
        
               filename=filename, 
        
               content_type=content_type.decode("latin-1"), 
        
               headers=Headers(raw=item_headers), 
        
           )

but it's never explicitly closed that I can see. The new warning is only present in Python 3.11 as it was added to CPython at python/cpython@78e70be#diff-6553a99f3ae04c9fc9c2349ac27037bfe3b274d5ca54d5800d9c5b5f11e29d21R707-R715 and is only raised when the file is garbage collected.

The warning is annoying but seems semi-safe to just silence as the file ends up getting closed anyway in the implementation of __del__.

I could potentially work on a fix, but it seems like it would be tricky to keep track of what files had been uploaded on the request to keep track of what needs to be explicitly closed only after the request has been fully completed. Any guidance on a general area to start looking for a fix?