Skip to content

Commit

Permalink
ci: Use supported ansible-lint action; run ansible-lint against the c…
Browse files Browse the repository at this point in the history
…ollection

The old ansible-community ansible-lint is deprecated.  There is a
new ansible-lint github action.

The latest Ansible repo gating tests run ansible-lint against
the collection format instead of against individual roles.
We have to convert the role to collection format before running
ansible-test.

This also requires tox-lsr 3.2.1 - bump other actions to use 3.2.1

Role developers can run this locally using
`tox -e collection,ansible-lint-collection`
See linux-system-roles/tox-lsr#125

Signed-off-by: Rich Megginson <rmeggins@redhat.com>
  • Loading branch information
richm committed Jan 8, 2024
1 parent f4fb0e6 commit 8e0e2b2
Show file tree
Hide file tree
Showing 8 changed files with 222 additions and 31 deletions.
28 changes: 17 additions & 11 deletions .github/workflows/ansible-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ on: # yamllint disable-line rule:truthy
branches:
- main
workflow_dispatch:
env:
LSR_ROLE2COLL_NAMESPACE: fedora
LSR_ROLE2COLL_NAME: linux_system_roles
permissions:
contents: read
jobs:
Expand All @@ -26,18 +29,21 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v4

- name: Fix up role meta/main.yml namespace and name
- name: Install tox, tox-lsr
run: |
set -euxo pipefail
mm=meta/main.yml
if [ -f "$mm" ]; then
if ! grep -q '^ *namespace:' "$mm"; then
sed "/galaxy_info:/a\ namespace: linux_system_roles" -i "$mm"
fi
if ! grep -q '^ *role_name:' "$mm"; then
sed "/galaxy_info:/a\ role_name: nbde_client" -i "$mm"
fi
fi
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.2.1"
- name: Convert role to collection format
run: |
set -euxo pipefail
TOXENV=collection lsr_ci_runtox
coll_dir=".tox/ansible_collections/$LSR_ROLE2COLL_NAMESPACE/$LSR_ROLE2COLL_NAME"
# ansible-lint action requires a .git directory???
# https://github.com/ansible/ansible-lint/blob/main/action.yml#L45
mkdir -p "$coll_dir/.git"
- name: Run ansible-lint
uses: ansible-community/ansible-lint-action@v6
uses: ansible/ansible-lint@v6
with:
working_directory: .tox/ansible_collections/${{ env.LSR_ROLE2COLL_NAMESPACE }}/${{ env.LSR_ROLE2COLL_NAME }}
2 changes: 1 addition & 1 deletion .github/workflows/ansible-managed-var-comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- name: Install tox, tox-lsr
run: |
set -euxo pipefail
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.1.1"
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.2.1"
- name: Run ansible-plugin-scan
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ansible-plugin-scan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- name: Install tox, tox-lsr
run: |
set -euxo pipefail
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.1.1"
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.2.1"
- name: Run ansible-plugin-scan
run: |
Expand Down
18 changes: 1 addition & 17 deletions .github/workflows/ansible-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,28 +33,12 @@ jobs:
- name: Install tox, tox-lsr
run: |
set -euxo pipefail
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.1.1"
pip3 install "git+https://github.com/linux-system-roles/tox-lsr@3.2.1"
- name: Convert role to collection format
run: |
set -euxo pipefail
# Remove to avoid running ansible-test on unrelated file
rm -f .pandoc_template.html5
TOXENV=collection lsr_ci_runtox
# copy the ignore files
coll_dir=".tox/ansible_collections/$LSR_ROLE2COLL_NAMESPACE/$LSR_ROLE2COLL_NAME"
# wokeignore:rule=sanity
ignore_dir="$coll_dir/tests/sanity"
if [ ! -d "$ignore_dir" ]; then
mkdir -p "$ignore_dir"
fi
# wokeignore:rule=sanity
for file in .sanity-ansible-ignore-*.txt; do
if [ -f "$file" ]; then
# wokeignore:rule=sanity
cp "$file" "$ignore_dir/${file//*.sanity-ansible-}"
fi
done
- name: Run ansible-test
uses: ansible-community/ansible-test-gh-action@release/v1
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-unit-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ jobs:
run: |
set -euxo pipefail
python -m pip install --upgrade pip
pip install "git+https://github.com/linux-system-roles/tox-lsr@3.1.1"
pip install "git+https://github.com/linux-system-roles/tox-lsr@3.2.1"
# If you have additional OS dependency packages e.g. libcairo2-dev
# then put them in .github/config/ubuntu-requirements.txt, one
# package per line.
Expand Down
3 changes: 3 additions & 0 deletions .ostree/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*NOTE*: The `*.txt` files are used by `get_ostree_data.sh` to create the lists
of packages, and to find other system roles used by this role. DO NOT use them
directly.
132 changes: 132 additions & 0 deletions .ostree/get_ostree_data.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
#!/usr/bin/env bash

set -euo pipefail

ostree_dir="${OSTREE_DIR:-"$(dirname "$(realpath "$0")")"}"

if [ -z "${4:-}" ] || [ "${1:-}" = help ] || [ "${1:-}" = -h ]; then
cat <<EOF
Usage: $0 packages [runtime|testing] DISTRO-MAJOR[.MINOR] [json|yaml|raw|toml]
The script will use the packages and roles files in $ostree_dir to
construct the list of packages needed to build the ostree image. The script
will output the list of packages in the given format
- json is a JSON list like ["pkg1","pkg2",....,"pkgN"]
- yaml is the YAML list format
- raw is the list of packages, one per line
- toml is a list of [[packages]] elements as in https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/composing_installing_and_managing_rhel_for_edge_images/index#creating-an-image-builder-blueprint-for-a-rhel-for-edge-image-using-the-command-line-interface_composing-a-rhel-for-edge-image-using-image-builder-command-line
The DISTRO-MAJOR.MINOR is the same format used by Ansible for distribution e.g. CentOS-8, RedHat-8.9, etc.
EOF
exit 1
fi
category="$1"
pkgtype="$2"
distro_ver="$3"
format="$4"
pkgtypes=("$pkgtype")
if [ "$pkgtype" = testing ]; then
pkgtypes+=(runtime)
fi

get_rolepath() {
local ostree_dir role rolesdir roles_parent_dir coll_path path
ostree_dir="$1"
role="$2"
roles_parent_dir="$(dirname "$(dirname "$ostree_dir")")"
rolesdir="$roles_parent_dir/$role/.ostree"
# assumes collection format
if [ -d "$rolesdir" ]; then
echo "$rolesdir"
return 0
fi
# assumes legacy role format like linux-system-roles.$role/
for rolesdir in "$roles_parent_dir"/*-system-roles."$role"/.ostree; do
if [ -d "$rolesdir" ]; then
echo "$rolesdir"
return 0
fi
done
# look elsewhere
coll_path="${ANSIBLE_COLLECTIONS_PATH:-}"
if [ -z "$coll_path" ]; then
coll_path="${ANSIBLE_COLLECTIONS_PATHS:-}"
fi
if [ -n "${coll_path}" ]; then
for path in ${coll_path//:/ }; do
for rolesdir in "$path"/ansible_collections/*/*_system_roles/roles/"$role"/.ostree; do
if [ -d "$rolesdir" ]; then
echo "$rolesdir"
return 0
fi
done
done
fi
1>&2 echo ERROR - could not find role "$role" - please use ANSIBLE_COLLECTIONS_PATH
exit 2
}

get_packages() {
local ostree_dir pkgtype pkgfile rolefile
ostree_dir="$1"
for pkgtype in "${pkgtypes[@]}"; do
for suff in "" "-$distro" "-${distro}-${major_ver}" "-${distro}-${ver}"; do
pkgfile="$ostree_dir/packages-${pkgtype}${suff}.txt"
if [ -f "$pkgfile" ]; then
cat "$pkgfile"
fi
done
rolefile="$ostree_dir/roles-${pkgtype}.txt"
if [ -f "$rolefile" ]; then
local roles role rolepath
roles="$(cat "$rolefile")"
for role in $roles; do
rolepath="$(get_rolepath "$ostree_dir" "$role")"
if [ -z "$rolepath" ]; then
1>&2 echo ERROR - could not find role "$role" - please use ANSIBLE_COLLECTIONS_PATH
exit 2
fi
get_packages "$rolepath"
done
fi
done | sort -u
}

format_packages_json() {
local comma pkgs pkg
comma=""
pkgs="["
while read -r pkg; do
pkgs="${pkgs}${comma}\"${pkg}\""
comma=,
done
pkgs="${pkgs}]"
echo "$pkgs"
}

format_packages_raw() {
cat
}

format_packages_yaml() {
while read -r pkg; do
echo "- $pkg"
done
}

format_packages_toml() {
while read -r pkg; do
echo "[[packages]]"
echo "name = \"$pkg\""
echo "version = \"*\""
done
}

distro="${distro_ver%%-*}"
ver="${distro_ver##*-}"
if [[ "$ver" =~ ^([0-9]*) ]]; then
major_ver="${BASH_REMATCH[1]}"
else
echo ERROR: cannot parse major version number from version "$ver"
exit 1
fi

"get_$category" "$ostree_dir" | "format_${category}_$format"
66 changes: 66 additions & 0 deletions README-ostree.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# rpm-ostree

The role supports running on [rpm-ostree](https://coreos.github.io/rpm-ostree/)
systems. The primary issue is that the `/usr` filesystem is read-only, and the
role cannot install packages. Instead, it will just verify that the necessary
packages and any other `/usr` files are pre-installed. The role will change the
package manager to one that is compatible with `rpm-ostree` systems.

## Building

To build an ostree image for a particular operating system distribution and
version, use the script `.ostree/get_ostree_data.sh` to get the list of
packages. If the role uses other system roles, then the script will include the
packages for the other roles in the list it outputs. The list of packages will
be sorted in alphanumeric order.

Usage:

```bash
.ostree/get_ostree_data.sh packages runtime DISTRO-VERSION FORMAT
```

`DISTRO-VERSION` is in the format that Ansible uses for `ansible_distribution`
and `ansible_distribution_version` - for example, `Fedora-38`, `CentOS-8`,
`RedHat-9.4`

`FORMAT` is one of `toml`, `json`, `yaml`, `raw`

* `toml` - each package in a TOML `[[packages]]` element

```toml
[[packages]]
name = "package-a"
version = "*"
[[packages]]
name = "package-b"
version = "*"
...
```

* `yaml` - a YAML list of packages

```yaml
- package-a
- package-b
...
```

* `json` - a JSON list of packages

```json
["package-a","package-b",...]
```

* `raw` - a plain text list of packages, one per line

```bash
package-a
package-b
...
```

What format you choose depends on which image builder you are using. For
example, if you are using something based on
[osbuild-composer](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/composing_installing_and_managing_rhel_for_edge_images/index#creating-an-image-builder-blueprint-for-a-rhel-for-edge-image-using-the-command-line-interface_composing-a-rhel-for-edge-image-using-image-builder-command-line),
you will probably want to use the `toml` output format.

0 comments on commit 8e0e2b2

Please sign in to comment.