Headers not present on some requests

[pcarn]

Version

Auto-Southwest Check-In v7.3 (develop branch)

Browser Version

Version 122.0.6261.129 (Official Build) (arm64)

Description

Continuing thread from #237

Sometimes the URL to set headers is not called, and it gets stuck on Waiting for headers_set to be set

This commit was added to help, but I still had a flight fail to check-in because it was stuck waiting for headers_set to be set.

If I can get it to happen again, I'll add logs again of what URLs are getting called.

To Reproduce

Not sure, it's intermittent. I have a user and password in my config file. It usually works fine, but sometimes will get stuck. It got stuck on an actual flight check-in the last time I tried.

Expected Behavior

It works normally.

Relevant logs and program output

2024-03-23 20:18:48 DEBUG Process-1[reservation_monitor:130]: Sleeping for 86356 seconds
2024-03-23 22:24:54 DEBUG Process-2:2[checkin_handler:100]: Acquiring lock...
2024-03-23 22:24:54 DEBUG Process-2:2[checkin_handler:102]: Lock acquired
2024-03-23 22:24:54 DEBUG Process-2:2[checkin_scheduler:51]: Refreshing headers for current session
2024-03-23 22:24:54 DEBUG Process-2:2[webdriver:109]: Starting webdriver for current session
2024-03-23 22:24:55 DEBUG Process-2:2[webdriver:125]: Using browser version: 122.0.6261.129
2024-03-23 22:24:55 DEBUG Process-2:2[webdriver:129]: Loading Southwest Check-In page
2024-03-23 22:24:56 DEBUG Process-2:2[webdriver:67]: Waiting for valid headers
2024-03-23 22:24:56 DEBUG Process-2:2[webdriver:158]: Waiting for headers_set to be set

### Additional context

_No response_

[jdholtz]

Hey @pcarn, are you still running into this issue after updating to the latest commit on the develop branch? #274 was just merged which did change how this was handled (a side effect) so that may have helped fix this issue.

[pcarn]

I'll try and see if it happens again. It's intermittent so hard to tell sometimes.

Right now it's hard to test because I get a lot of Encountered a Too Many Requests error while logging in. Skipping reservation retrieval errors, even when I'm running it for the first time in several days.

[jdholtz]

Right now it's hard to test because I get a lot of Encountered a Too Many Requests error while logging in. Skipping reservation retrieval errors

See #277. The develop image should significantly help evading the Too Many Requests error.

[pcarn]

It's not looking any better for me, on the current develop branch.

Logs

2024-07-03 13:53:44 DEBUG MainProcess[log:24]: Initialized the application 2024-07-03 13:53:44 DEBUG MainProcess[main:113]: Auto-Southwest Check-In v7.5 2024-07-03 13:53:44 DEBUG MainProcess[main:71]: Called with 0 arguments 2024-07-03 13:53:44 DEBUG MainProcess[config:132]: Initializing configuration file 2024-07-03 13:53:44 DEBUG MainProcess[config:161]: Reading the configuration file 2024-07-03 13:53:44 DEBUG MainProcess[config:174]: Reading configuration from environment variables 2024-07-03 13:53:44 DEBUG MainProcess[config:144]: Creating configurations for 1 accounts 2024-07-03 13:53:44 DEBUG MainProcess[config:151]: Creating configurations for 0 reservations 2024-07-03 13:53:44 INFO MainProcess[main:97]: Monitoring 1 account and 0 reservations

2024-07-03 13:53:44 DEBUG Process-1[reservation_monitor:168]: Acquiring lock...
2024-07-03 13:53:44 DEBUG Process-1[reservation_monitor:170]: Lock acquired
2024-07-03 13:53:44 DEBUG Process-1[reservation_monitor:192]: Retrieving reservations for account
2024-07-03 13:53:44 DEBUG Process-1[webdriver:126]: Starting webdriver for current session
2024-07-03 13:53:47 DEBUG Process-1[webdriver:143]: Using browser version: 126.0.6478.127
2024-07-03 13:53:47 DEBUG Process-1[webdriver:147]: Loading Southwest home page (this may take a moment)
2024-07-03 13:54:00 DEBUG Process-1[webdriver:100]: Logging into account to get a list of reservations and valid headers
2024-07-03 13:54:04 DEBUG Process-1[webdriver:178]: Waiting for headers_set to be set
2024-07-03 13:54:04 DEBUG Process-1[webdriver:182]: headers_set set successfully
2024-07-03 13:54:04 DEBUG Process-1[webdriver:178]: Waiting for login_request_id to be set
2024-07-03 13:54:05 DEBUG Process-1[webdriver:170]: Login response has been received
2024-07-03 13:54:05 DEBUG Process-1[webdriver:182]: login_request_id set successfully
2024-07-03 13:54:05 DEBUG Process-1[webdriver:238]: Logging in failed for an unknown reason
2024-07-03 13:54:05 WARNING Process-1[reservation_monitor:201]: Encountered a Too Many Requests error while logging in. Skipping reservation retrieval
2024-07-03 13:54:05 DEBUG Process-1[reservation_monitor:181]: Lock released
2024-07-03 13:54:05 DEBUG Process-1[reservation_monitor:130]: Sleeping for 86378 seconds

[ne0ark]

I have also run into this issue. If you start and stop the Docker container a couple of times, the request will exceed the allowed limits. It seems that SW is rate-limiting requests. @jdholtz I reverted back to js_click because after changing to click, this error seemed to happen more often.

[dmytrokoren]

@pcarn and @ne0ark can you try this

Docker: docker pull dmytrokoren/auto-southwest-check-in:alpha-proxy
Local: dmytrokoren/auto-southwest-check-in : modified_webdriver_and_updated_requirements

[ne0ark]

@pcarn and @ne0ark can you try this

Docker: docker pull dmytrokoren/auto-southwest-check-in:alpha-proxy Local: dmytrokoren/auto-southwest-check-in : modified_webdriver_and_updated_requirements

Yes, I have been using modified_webdriver_and_updated_requirements on docker, and it works great!

[babehboi]

It's been a long time since posting, but I thought I'd add something:

Although it doesn't affect anything, but seems to be happening is the following error:
Notice: Webdriver time out during reservation retrieval for XXXXX. Skipping reservation retrieval until next interval.

What I've noticed, although I haven't been able to confirm without editing the code is that for the lines that I indicated check_fares false it would happen on the subsequent check? The initial run was good.

[dmytrokoren]

It's been a long time since posting, but I thought I'd add something:

Although it doesn't affect anything, but seems to be happening is the following error: Notice: Webdriver time out during reservation retrieval for XXXXX. Skipping reservation retrieval until next interval.

What I've noticed, although I haven't been able to confirm without editing the code is that for the lines that I indicated check_fares false it would happen on the subsequent check? The initial run was good.

@babehboi

Try this (this should resolve your issue, have been running this for two days with 4hrs interval, no errors)

Docker: docker pull dmytrokoren/auto-southwest-check-in:alpha-window-server
Local: dmytrokoren/auto-southwest-check-in : modified_webdriver_and_updated_requirements

[babehboi]

I'll give it a try. I'm running the alpha-proxy while on unraid.

Still got it once upon initial start-up with [alpha-window-server]

[dmytrokoren]

I'll give it a try. I'm running the alpha-proxy while on unraid.

Still got it once upon initial start-up with [alpha-window-server]

alpha-proxy is outdated.

alpha-window-server you will get either once on initial or second refresh... after that I did not get any errors.

[dmytrokoren]

@pcarn @ne0ark @babehboi

Give this image a try:

Docker: docker pull dmytrokoren/auto-southwest-check-in:alpha-legacy-windock

[babehboi]

@pcarn @ne0ark @babehboi

Give this image a try:

Docker: docker pull dmytrokoren/auto-southwest-check-in:alpha-legacy-windock

Thanks. I've been testing this one the past couple of days. It seems to be stable and haven't had that issue in a while

[dmytrokoren]

@pcarn @ne0ark @babehboi

Give this image a try:

Docker: docker pull dmytrokoren/auto-southwest-check-in:alpha-legacy-windock

Thanks. I've been testing this one the past couple of days. It seems to be stable and haven't had that issue in a while

Thanks I'll commit the code soon with these changes