excessive disk space by temp files

[sevenlayercookie]

I am running latest version in Docker (from Docker repo). Vanilla, unmodified. Have been runnign off and on for about two months. My server crashed due to disk space being full, and upon looking into I see that /var/lib/docker/overlay2/xxxxxx/diff/tmp/ is full of chromium temp files, totaling to 86gb. They all seem to be related to southwest check in. I've also noticed fairly high CPU use of chromium in htop, dozens of chromium processes with 170 gb virtual memory size.

It's possible this is an issue with how Docker handles cache and temp files. But just wanted to report it in case others have similar issues.

[drippyer]

I believe I encountered this previously and it appears to be fixed in the develop branch, though I am now running into #201 after switching.

On main, running in Docker on Unraid, I had received an alert that tmp storage was more full than expected. Upon checking "Container Size" within the Unraid UI, I could see that this container was taking up more than 10GB of the 20GB allotted to the Docker image. As mentioned, the develop branch seemed to fix that error.

I also noticed that my container's restart configuration was set to unless-stopped rather than on-failure. And in "typical" debugging behavior, I changed both variables at once, so I'm not certain which exactly fixed the excessive logging.

[jdholtz]

That's very strange. As @drippyer pointed out, the fix may be moving to the develop branch. This script isn't responsible for handling Chromium directly (it uses SeleniumBase for that), so it's quite possible that this bug was fixed in a newer SeleniumBase version that the develop branch is using--although there isn't any mention of that in the changelog. Once #201 is fixed, I can run the script for a few days and see if I can reproduce this issue.

[sevenlayercookie]

That's very strange. As @drippyer pointed out, the fix may be moving to the develop branch. This script isn't responsible for handling Chromium directly (it uses SeleniumBase for that), so it's quite possible that this bug was fixed in a newer SeleniumBase version that the develop branch is using--although there isn't any mention of that in the changelog. Once #201 is fixed, I can run the script for a few days and see if I can reproduce this issue.

I'm going to uninstall and reinstall from scratch and see if it continues to be an issue

[jdholtz]

Hey @sevenlayercookie, are you still running into this issue? If so, I can dig deeper into this.

[sevenlayercookie]

Hey @sevenlayercookie, are you still running into this issue? If so, I can dig deeper into this.

You know it hadn't been an issue for months, and then I switched branches again about a month ago and ended up having the same issue again and had to uninstall the docker again.

[jdholtz]

I've had the script running in a Docker container for the past month and haven't seen more than 1 or 2 of these temp files be created. Is the script running normally for you or are there failures happening with the browser a lot (e.g. crashes, 429s, etc.)?