You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi experts,
I am using celery worker to execute my tasks and I have Redis as the broker and result backend.
celery - 4.4.7
kombu - 4.6.11
redis - 6
While my worker executes a long running task (longer than Redis server side connection timeout value), my celery worker main process eats up to 100% CPU.
After my worker finishes current task and get the next task, Kombu hub will know the fd is closed. And will create a new connection.
Then my celery worker main process will release the CPU. But will eats up to 100% CPU again if I have a long-running task again.
I wont run into this CPU 100% issue if I have Redis connection timeout set to 0.
But this is not acceptable in my situation.
Any suggestion to resolve this CPU 100% issue?
I notice that kombu hub keeps get READ event from the BRPOP channel after Redis connection timed out. Which makes CPU 100%.
Not sure how to fix this issue.
Appreciate your comment on this.
Thanks.
The text was updated successfully, but these errors were encountered:
Hi experts,
I am using celery worker to execute my tasks and I have Redis as the broker and result backend.
celery - 4.4.7
kombu - 4.6.11
redis - 6
While my worker executes a long running task (longer than Redis server side connection timeout value), my celery worker main process eats up to 100% CPU.
After my worker finishes current task and get the next task, Kombu hub will know the fd is closed. And will create a new connection.
Then my celery worker main process will release the CPU. But will eats up to 100% CPU again if I have a long-running task again.
I wont run into this CPU 100% issue if I have Redis connection timeout set to 0.
But this is not acceptable in my situation.
Any suggestion to resolve this CPU 100% issue?
I notice that kombu hub keeps get READ event from the BRPOP channel after Redis connection timed out. Which makes CPU 100%.
Not sure how to fix this issue.
Appreciate your comment on this.
Thanks.
The text was updated successfully, but these errors were encountered: