Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OSMount] Add waits/retry during Ubuntu minion prep stage. #307

Merged
merged 1 commit into from
Apr 3, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions coriolis/osmorphing/osmount/ubuntu.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,24 @@ def check_os(self):

def setup(self):
super(UbuntuOSMountTools, self).setup()
self._exec_cmd("sudo -E apt-get update -y")
self._exec_cmd("sudo -E apt-get install lvm2 psmisc -y")

# NOTE(aznashwan): it's possible that if the Ubuntu minion has
# unattended automatic upgrades enabled, the package list
# locks (/var/lib/apt/lists/lock*) may be held because
# another package list refresh is happening.
# Apart from relying on possibly not-yet-installed tools like `fuser`,
# or checking every /proc/*/fd ourselves, we simply retry it:
retry_ssh_cmd = utils.retry_on_error(
max_attempts=10, sleep_seconds=30)(self._exec_cmd)
retry_ssh_cmd("sudo -E apt-get update -y")

# NOTE(aznashwan): in case an unattended upgrade is already happening
# and is at the package installation stage (in which case the
# /var/lib/dpkg/* locks will be held), we pass a 10-minute timeout:
self._exec_cmd(
"sudo -E apt-get -o DPkg::Lock::Timeout=600 "
"install lvm2 psmisc -y")

self._exec_cmd("sudo modprobe dm-mod")

def _allow_ssh_env_vars(self):
Expand Down
Loading