You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implement parallel startup with startup intervals between multiple tasks in multirun mode
Motivation
During neural network training, it is necessary to load the model and dataset onto the GPU.
We select GPUs by sorting their available resources.
If parallel tasks are started simultaneously, because GPU loading takes time, there is a high probability that sorting functions running at the same time will choose the same GPU, which may result in memory overflow.
The text was updated successfully, but these errors were encountered:
Jinshijiming
changed the title
multirun模式下,实现多任务之间有启动间隔的并行启动
Implement parallel startup with startup intervals between multiple tasks in multirun mode
Jan 20, 2025
🚀 Feature Request
Implement parallel startup with startup intervals between multiple tasks in multirun mode
Motivation
During neural network training, it is necessary to load the model and dataset onto the GPU.
We select GPUs by sorting their available resources.
If parallel tasks are started simultaneously, because GPU loading takes time, there is a high probability that sorting functions running at the same time will choose the same GPU, which may result in memory overflow.
The text was updated successfully, but these errors were encountered: