Merge pull request #65 from cdot65/17-enhance-upgrade-script-to-suppo…

…rt-majorminor-version-upgrades-for-pan-os-firewalls

Enhance Upgrade Script for Major/Minor Version Support in PAN-OS Firewalls

.vscode/settings.json

@@ -2,13 +2,16 @@
   "cSpell.words": [
     "adminpassword",
     "apikey",
+    "apipassword",
+    "apiuser",
     "defusedxml",
     "dynaconf",
     "Dynaconf",
     "hhotfix",
     "highavailability",
     "hostnames",
     "levelname",
+    "lifecycles",
     "malformatted",
     "nics",
     "NXDOMAIN",

docker/Dockerfile

@@ -14,8 +14,8 @@ WORKDIR /app
 ADD settings.yaml /app
 
 # Install any needed packages specified in requirements.txt
-# Note: The requirements.txt should contain pan-os-upgrade==0.4.2
-RUN pip install --no-cache-dir pan-os-upgrade==0.4.2
+# Note: The requirements.txt should contain pan-os-upgrade==0.4.3
+RUN pip install --no-cache-dir pan-os-upgrade==0.4.3
 
 # Set the locale to avoid issues with emoji rendering
 ENV LANG C.UTF-8

docs/about/release-notes.md

@@ -2,10 +2,21 @@
 
 Welcome to the release notes for the `pan-os-upgrade` tool. This document provides a detailed record of changes, enhancements, and fixes in each version of the tool.
 
+## Version 0.4.3
+
+**Release Date:** *<20240129>*
+
+### What's New
+
+- Created workflow that will download the base image if making a major/minor upgrade
+- Included new download settings to be overridden with `pan-os-upgrade` settings
+- Provide helpful message when a target version is not selected, providing suggestions of similar versions that are available.
+
 ## Version 0.4.2
 
 **Release Date:** *<20240127>*
 
+<!-- trunk-ignore(markdownlint/MD024) -->
 ### What's New
 
 - Created mechanism to override the default settings of `pan-os-upgrade`

docs/index.md

@@ -81,7 +81,7 @@ pan-os-upgrade batch
 Panorama hostname or IP: panorama.cdot.io
 Panorama username: cdot
 Panorama password:
-Firewall target version (ex: 10.1.2): 10.2.3
+Firewall target version (ex: 10.1.2): 10.2.7-h3
 Filter string (ex: hostname=Woodlands*) []: hostname=Woodlands*
 Dry Run? [y/N]:
 ===========================================================================
@@ -93,19 +93,68 @@ No settings.yaml file was found. Default values will be used.
 Create a settings.yaml file with 'pan-os-upgrade settings' command.
 ===========================================================================
 ✅ panorama.cdot.io: Connection to Panorama established. Firewall connections will be proxied!
-📝 Woodlands-fw2: 007954000123452 192.168.255.44
-📝 Woodlands-fw1: 007954000123451 192.168.255.43
+📝 Woodlands-fw2: 007954001234562 192.168.255.44
+🚀 Woodlands-fw2: Getting 007954001234562 deployment information...
+📝 Woodlands-fw1: 007954001234561 192.168.255.43
+🚀 Woodlands-fw1: Getting 007954001234561 deployment information...
+📝 Woodlands-fw2: Target device deployment: passive
 📝 Woodlands-fw2: HA mode: passive
+🚀 Woodlands-fw2: Getting 007954001234562 deployment information...
+📝 Woodlands-fw1: Target device deployment: active
 📝 Woodlands-fw1: HA mode: active
+🚀 Woodlands-fw1: Getting 007954001234561 deployment information...
+📝 Woodlands-fw2: Target device deployment: passive
+📝 Woodlands-fw1: Target device deployment: active
+📝 Woodlands-fw2: Local state: passive, Local version: 10.1.3, Peer version: 10.1.3
+📝 Woodlands-fw1: Local state: active, Local version: 10.1.3, Peer version: 10.1.3
+📝 Woodlands-fw2: Version comparison: equal
+📝 Woodlands-fw1: Version comparison: equal
+📝 Woodlands-fw2: Target device is passive
 🔍 Woodlands-fw1: Detected active target device in HA pair running the same version as its peer. Added target device to revisit list.
-📝 Woodlands-fw2: Current version: 10.2.2-h2
-📝 Woodlands-fw2: Target version: 10.2.3
-✅ Woodlands-fw2: Upgrade required from 10.2.2-h2 to 10.2.3
-✅ Woodlands-fw2: version 10.2.3 is available for download
-✅ Woodlands-fw2: Base image for 10.2.3 is already downloaded
-🚀 Woodlands-fw2: Performing test to see if 10.2.3 is already downloaded...
-✅ Woodlands-fw2: version 10.2.3 already on target device.
-✅ Woodlands-fw2: 10.2.3 has been downloaded and sync'd to HA peer.
+📝 Woodlands-fw2: Current version: 10.1.3
+📝 Woodlands-fw2: Target version: 10.2.7-h3
+✅ Woodlands-fw2: Upgrade required from 10.1.3 to 10.2.7-h3
+✅ Woodlands-fw2: version 10.2.7-h3 is available for download
+❌ Woodlands-fw2: Base image for 10.2.7-h3 is not downloaded. Attempting download...
+🔍 Woodlands-fw2: version 10.2.0 is not on the target device
+🚀 Woodlands-fw2: version 10.2.0 is beginning download
+Device 007954001234562 downloading version: 10.2.0
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 3 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 34 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 67 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 99 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 131 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 164 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 196 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 227 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 258 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 290 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 322 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 353 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.0 - HA will sync image - Elapsed time: 386 seconds
+✅ Woodlands-fw2: 10.2.0 downloaded in 418 seconds
+✅ Woodlands-fw2: Base image 10.2.0 downloaded successfully
+✅ Woodlands-fw2: Pausing for 60 seconds to let 10.2.0 image load into the software manager before downloading 10.2.7-h3
+📝 Woodlands-fw2: Current version: 10.1.3
+📝 Woodlands-fw2: Target version: 10.2.7-h3
+✅ Woodlands-fw2: Upgrade required from 10.1.3 to 10.2.7-h3
+✅ Woodlands-fw2: version 10.2.7-h3 is available for download
+✅ Woodlands-fw2: Base image for 10.2.7-h3 is already downloaded
+🚀 Woodlands-fw2: Performing test to see if 10.2.7-h3 is already downloaded...
+🔍 Woodlands-fw2: version 10.2.7-h3 is not on the target device
+🚀 Woodlands-fw2: version 10.2.7-h3 is beginning download
+Device 007954001234562 downloading version: 10.2.7-h3
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 3 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 36 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 67 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 99 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 132 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 163 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 195 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 227 seconds
+🔧 Woodlands-fw2: Downloading version 10.2.7-h3 - HA will sync image - Elapsed time: 260 seconds
+✅ Woodlands-fw2: 10.2.7-h3 downloaded in 291 seconds
+✅ Woodlands-fw2: 10.2.7-h3 has been downloaded and sync'd to HA peer.
 🚀 Woodlands-fw2: Performing snapshot of network state information...
 ✅ Woodlands-fw2: Network snapshot created successfully
 🚀 Woodlands-fw2: Performing readiness checks to determine if firewall is ready for upgrade...
@@ -120,66 +169,79 @@ Create a settings.yaml file with 'pan-os-upgrade settings' command.
 ✅ Woodlands-fw2: HA peer sync test has been completed.
 🚀 Woodlands-fw2: Performing backup of configuration to local filesystem...
 🚀 Woodlands-fw2: Not a dry run, continue with upgrade...
-🚀 Woodlands-fw2: Performing upgrade to version 10.2.3...
-🚀 Woodlands-fw2: Attempting upgrade to version 10.2.3 (Attempt 1 of 3)...
-Device 007954000123452 installing version: 10.2.3
+🚀 Woodlands-fw2: Performing upgrade to version 10.2.7-h3...
+🚀 Woodlands-fw2: Attempting upgrade to version 10.2.7-h3 (Attempt 1 of 3)...
+Device 007954001234562 installing version: 10.2.7-h3
 ✅ Woodlands-fw2: Upgrade completed successfully
-🚀 Woodlands-fw2: Rebooting the passive HA target device...
+🚀 Woodlands-fw2: Rebooting the target device...
 📝 Woodlands-fw2: Command succeeded with no output
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: Target device is rebooting...
-🔧 Woodlands-fw2: HA passive target device rebooted but not yet synchronized with its peer. Will try again in 60 seconds.
-🔧 Woodlands-fw2: HA passive target device rebooted but not yet synchronized with its peer. Will try again in 60 seconds.
-🔧 Woodlands-fw2: HA passive target device rebooted but not yet synchronized with its peer. Will try again in 60 seconds.
-🔧 Woodlands-fw2: HA passive target device rebooted but not yet synchronized with its peer. Will try again in 60 seconds.
-🟧 Woodlands-fw2: HA passive target device rebooted but did not complete a configuration sync with the active after 5 attempts.
+🟧 Woodlands-fw2: Retry attempt 1 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 2 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 3 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 4 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 5 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 6 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 7 due to error: 007954001234562 not connected
+🟧 Woodlands-fw2: Retry attempt 8 due to error: 007954001234562 not connected
+📝 Woodlands-fw2: Current device version: 10.2.7-h3
+✅ Woodlands-fw2: Device rebooted to the target version successfully.
 🚀 panorama.cdot.io: Revisiting firewalls that were active in an HA pair and had the same version as their peers.
-📝 Woodlands-fw1: 007954000123451 192.168.255.43
+📝 Woodlands-fw1: 007954001234561 192.168.255.43
+🚀 Woodlands-fw1: Getting 007954001234561 deployment information...
+📝 Woodlands-fw1: Target device deployment: active
 📝 Woodlands-fw1: HA mode: active
+🚀 Woodlands-fw1: Getting 007954001234561 deployment information...
+📝 Woodlands-fw1: Target device deployment: active
+📝 Woodlands-fw1: Local state: active, Local version: 10.1.3, Peer version: 10.1.3
+Waiting for HA synchronization to complete on Woodlands-fw1. Attempt 1/3
+🚀 Woodlands-fw1: Getting 007954001234561 deployment information...
+📝 Woodlands-fw1: Target device deployment: active
+HA synchronization still in progress on Woodlands-fw1. Rechecking after wait period.
+Waiting for HA synchronization to complete on Woodlands-fw1. Attempt 2/3
+🚀 Woodlands-fw1: Getting 007954001234561 deployment information...
+📝 Woodlands-fw1: Target device deployment: non-functional
+HA synchronization complete on Woodlands-fw1. Proceeding with upgrade.
+📝 Woodlands-fw1: Version comparison: older
+📝 Woodlands-fw1: Target device is on an older version
+📝 Woodlands-fw1: Suspending HA state of active
 ❌ Woodlands-fw1: Error suspending active target device HA state: argument of type 'NoneType' is not iterable
-📝 Woodlands-fw1: Current version: 10.2.2-h2
-📝 Woodlands-fw1: Target version: 10.2.3
-✅ Woodlands-fw1: Upgrade required from 10.2.2-h2 to 10.2.3
-✅ Woodlands-fw1: version 10.2.3 is available for download
-✅ Woodlands-fw1: Base image for 10.2.3 is already downloaded
-🚀 Woodlands-fw1: Performing test to see if 10.2.3 is already downloaded...
-✅ Woodlands-fw1: version 10.2.3 already on target device.
-✅ Woodlands-fw1: 10.2.3 has been downloaded and sync'd to HA peer.
+📝 Woodlands-fw1: Current version: 10.1.3
+📝 Woodlands-fw1: Target version: 10.2.7-h3
+✅ Woodlands-fw1: Upgrade required from 10.1.3 to 10.2.7-h3
+✅ Woodlands-fw1: version 10.2.7-h3 is available for download
+✅ Woodlands-fw1: Base image for 10.2.7-h3 is already downloaded
+🚀 Woodlands-fw1: Performing test to see if 10.2.7-h3 is already downloaded...
+✅ Woodlands-fw1: version 10.2.7-h3 already on target device.
+✅ Woodlands-fw1: 10.2.7-h3 has been downloaded and sync'd to HA peer.
 🚀 Woodlands-fw1: Performing snapshot of network state information...
 ✅ Woodlands-fw1: Network snapshot created successfully
 🚀 Woodlands-fw1: Performing readiness checks to determine if firewall is ready for upgrade...
 ✅ Woodlands-fw1: Passed Readiness Check: Check if there are pending changes on device
 ✅ Woodlands-fw1: Passed Readiness Check: No Expired Licenses
 ✅ Woodlands-fw1: Passed Readiness Check: Check if NTP is synchronized
+✅ Woodlands-fw1: Passed Readiness Check: Check if the clock is synchronized between dataplane and management plane
 ✅ Woodlands-fw1: Passed Readiness Check: Check connectivity with the Panorama appliance
 ✅ Woodlands-fw1: Readiness Checks completed
 🚀 Woodlands-fw1: Checking if HA peer is in sync...
-🟧 Woodlands-fw1: HA peer state is not in sync. This will be noted, but the script will continue.
+✅ Woodlands-fw1: HA peer sync test has been completed.
 🚀 Woodlands-fw1: Performing backup of configuration to local filesystem...
 🚀 Woodlands-fw1: Not a dry run, continue with upgrade...
-🚀 Woodlands-fw1: Performing upgrade to version 10.2.3...
-🚀 Woodlands-fw1: Attempting upgrade to version 10.2.3 (Attempt 1 of 3)...
-Device 007954000123451 installing version: 10.2.3
+🚀 Woodlands-fw1: Performing upgrade to version 10.2.7-h3...
+🚀 Woodlands-fw1: Attempting upgrade to version 10.2.7-h3 (Attempt 1 of 3)...
+Device 007954001234561 installing version: 10.2.7-h3
 ✅ Woodlands-fw1: Upgrade completed successfully
-🚀 Woodlands-fw1: Rebooting the passive HA target device...
+🚀 Woodlands-fw1: Rebooting the target device...
 📝 Woodlands-fw1: Command succeeded with no output
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-🔧 Woodlands-fw1: Target device is rebooting...
-✅ Woodlands-fw1: HA passive target device rebooted and synchronized with its peer in 631 seconds
+🟧 Woodlands-fw1: Retry attempt 1 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 2 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 3 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 4 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 5 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 6 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 7 due to error: 007954001234561 not connected
+🟧 Woodlands-fw1: Retry attempt 8 due to error: 007954001234561 not connected
+📝 Woodlands-fw1: Current device version: 10.2.7-h3
+✅ Woodlands-fw1: Device rebooted to the target version successfully.
 ✅ panorama.cdot.io: Completed revisiting firewalls
 ```
 

docs/user-guide/docker/execution.md

@@ -187,7 +187,6 @@ $ docker run \
 -v $(pwd)/logs:/app/logs \
 -it \
 ghcr.io/cdot65/pan-os-upgrade:latest batch
-
 Panorama hostname or IP: panorama.cdot.io
 Panorama username: cdot
 Panorama password: