Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from number5:master #161

Open
wants to merge 388 commits into
base: master
Choose a base branch
from
Open

Conversation

pull[bot]
Copy link

@pull pull bot commented May 19, 2020

See Commits and Changes for more details.


Created by pull[bot]. Want to support this open source service? Please star it : )

@pull pull bot added the ⤵️ pull label May 19, 2020
TheRealFalcon and others added 29 commits October 1, 2020 16:32
This means that the integration tests do not need to install
test-requirements.txt in order to successfully import `conftest.py`.
pycloudlib's default behaviour is to raise an exception if cloud-init
fails to run in an instance being launched.  For cloud-init testing, we
want our test assertions to flag up failures, so we disable this
behaviour for instances we launch.
…during enumeration of physical network devices (#591)
This moves logging into `report_diagnostic_event`, to clean up its callsites.
Consider valid product names as valid chassis asset tags when detecting
OpenStack platform before crawling for OpenStack metadata.

As `ds-identify` tool uses product name as valid chassis asset tags,
let's replicate the behaviour in the OpenStack platform detection too.

This change should be backwards compatible and a temporary fix for the
current limitations on the OpenStack platform detection.

LP: #1895976
Specifically:
* `apt_configure_sources_list`
* `ntp_servers`
* `set_password_list`
* `users_groups`

Although not currently run in Travis, `set_password_list_string` was
ported over alongside `set_password_list` (as `test_set_password`).
…ing (#590)

DataSourceAzure previously writes the preprovisioning
reported ready marker file before it goes through the
report ready workflow. On certain VM instances, the
marker file is successfully written but then reporting
ready fails.

Upon rare VM reboots by the platform, cloud-init sees
that the report ready marker file already exists.
The existence of this marker file tells cloud-init
not to report ready again (because it mistakenly
assumes that it already reported ready in
preprovisioning).

In this scenario, cloud-init instead erroneously
takes the reprovisioning workflow instead of
reporting ready again.
Translate the following tests from `cloud_tests` to the new integration test framework:

* test_runcmd.py
* seed_random_data.py
* set_hostname.py
* set_hostname_fqdn.py
* snap.py
* ssh_auth_key_fingerprints_disable.py
* ssh_auth_key_fingerprints_enable.py
* ssh_import_id.py
* ssh_keys_generate.py
* ssh_keys_provided.py
* timezone.py
* write_files.py
…(#576)

bddeb builds a .deb package using the template packaging files in
packages/debian/.

The new --packaging-branch flag allows to specify a git branch
where to pull the packaging (i.e. the debian/ directory) from.
This is useful to build a .deb package from master with the very
same packaging which is used for the uploads.
The following commit merged all ssh keys into a default user file
`~/.ssh/authorized_keys` in sshd_config had multiple files configured for
AuthorizedKeysFile:

commit f1094b1
Author: Eduardo Otubo <otubo@redhat.com>
Date:   Thu Dec 5 17:37:35 2019 +0100

    Multiple file fix for AuthorizedKeysFile config (#60)

This commit ignored the case when sshd_config would have a single file for
AuthorizedKeysFile, but a non default configuration, for example
`~/.ssh/authorized_keys_foobar`. In this case cloud-init would grab all keys
from this file and write a new one, the default `~/.ssh/authorized_keys`
causing the bug.

rhbz: #1862967

Signed-off-by: Eduardo Otubo <otubo@redhat.com>
This is implemented as a copy/paste of the `citest` integration testing
script: the two are not intended to co-exist long-term, so it isn't
worth further complicating an already complex part of our Travis
configuration for short-term code reuse.

The two changes from the `citest` definition: the test framework
executed on the last line of `script`, and it is given a `name` so we
can easily ignore failures.
`create_swap()` was previously catching and not re-raising the
ProcessExecutionError that indicated swap creation failure; this meant
that the fallback logic could never be triggered.

This commit adds the required re-raise (as well as removing a duplicated
log message).

LP: #1897099
update_resolve_conf_file is no longer used.  The last reference
to it was removed in c368047, which was itself a "remove dead
code" commit.
* docs: Add hot to use cloud-localds to boot qemu

There is a complete lack of documentation on using cloud-localds with
cloud-init to boot an image locally.

Drive by, added some more whitepapers, blogs, and videos

* fix line length

* * add where cloud-localds comes from
* add more specific example with metadata and network config

* Add link to cloud-utils package
ensure_file needed modification to support doing this, so this commit
also includes the following changes:

test_util: add tests for util.ensure_file
util: add preserve_mode parameter to ensure_file
util: add (partial) type annotations to ensure_file

LP: #1900837
* Separated IntegrationClient into separate cloud and instance abstractions.
  This makes it easier to control the lifetime of the pycloudlib's cloud
  and instance abstractions separately.
* Created new cloud-specific subclasses accordingly
* Moved platform parsing and initialization code into its own file
* Created new session-wide autorun fixture to automatically initialize
  and destroy the dynamic cloud
Gentoo's hostname file format instead of being just the host name
is hostname=thename". The old code works fine when the file has no comments
but if there is a comment the line

```
gentoo_hostname_config = 'hostname="%s"' % conf
```

can render an invalid hostname file that looks similar to

```
hostname="#This is the host namehello"
```

The fix inserts the hostname in a gentoo friendly way so that it gets
handled by HostnameConf as a whole and comments are handled and preserved
Also update MAC addresses used in testcases to remove quotes where not
required and add single quotes where quotes are required.
Mapped from lp-markusschade
Reliable Scalable Cluster Technology (RSCT) is a set of software
components that together provide a comprehensive clustering
environment(RAS features) for IBM PowerVM based virtual machines. RSCT
includes the Resource Monitoring and Control (RMC) subsystem. RMC is a
generalized framework used for managing, monitoring, and manipulating
resources. RMC runs as a daemon process on individual machines and needs
creation of unique node id and restarts during VM boot.

LP: #1895979

Co-authored-by: Scott Moser <smoser@brickies.net>
If an OVS bridge was used as the only/primary interface, the 'init'
stage failed with a "Not all expected physical devices present" error,
leaving the system with a broken SSH setup.

LP: #1898997
The static and static6 subnet types for network_data.json were
being ignored by the Openstack handler, this would cause the code to
break and not function properly.

As of today, if a static6 configuration is chosen, the interface will
still eventually be available to receive router advertisements or be set
from NetworkManager to wait for them and cycle the interface in negative
case.

It is safe to assume that if the interface is manually configured to use
static ipv6 address, there's no need to wait for router advertisements.
This patch will set automatically IPV6_AUTOCONF and IPV6_FORCE_ACCEPT_RA
both to "no" in this case.

This patch fixes the specific behavior only for RHEL flavor and
sysconfig renderer. It also introduces new unit tests for the specific
case as well as adjusts some existent tests to be compatible with the
new options. This patch also addresses this problem by assigning the
appropriate subnet type for each case on the openstack handler.

rhbz: #1889635
rhbz: #1889635

Signed-off-by: Eduardo Otubo otubo@redhat.com
anhvoms and others added 28 commits April 27, 2021 12:40
This reverts commit 74fa008.

During pre-release testing, we discovered two issues with this commit.
Firstly, there's a typo in the udevadm command that causes a TypeError
for _all_ growpart executions.  Secondly, the LVM resizing does not
appear to successfully resize everything up to the LV, though some
things do get resized.

We certainly want this change, so we'll be happy to review and land it
alongside an integration test which confirms that it is working as
expected.

LP: #1922742
Bump the version in cloudinit/version.py to 21.2 and update ChangeLog.

LP: #1927254
AlmaLinux OS is RHEL-compatible so all the changes needed are trivial.
UDEVADM_CMD is defined but not actually used in cc_disk_setup.py
so remove it.

Also modify the comment at top of read_parttbl function to remove the
reference to udevadm which implies it is used to scan the partition table.
In newer versions of python, when using urllib.parse, lines containing
newline or tab characters now get sanitized. This caused a unit test to
fail.

See https://bugs.python.org/issue43882
Control is currently limited to boot events, though this should
allow us to more easily incorporate HOTPLUG support. Disabling
'instance-first-boot' is not supported as we apply networking config
too early in boot to have processed userdata (along with the fact
that this would be a pretty big foot-gun).

The concept of update events on datasource has been split into
supported update events and default update events. Defaults will be
used if there is no user-defined update events, but user-defined
events won't be supplied if they aren't supported.
When applying the networking config, we now check to see if the event
is supported by the datasource as well as if it is enabled.

Configuration looks like:
updates:
  network:
    when: ['boot']
httpretty now logs all requests by default which gets mixed up with our
logging tests. Also we were incorrectly setting a logging level to
'None', which now also causes issues with the new httpretty version.
See gabrielfalcao/HTTPretty#419
In #856 we added the ability to use partprobe instead of blockdev for
reading partitions. Test that partprobe succeeds where blockdev fails.

Also add a mechanism to our integration tests to allow a callable to be
called between `lxc init` and `lxc start`
In the case of a static network, we now set the MTU according to the
meta-data.
…(#882)

Presently, mirror keys cannot be associated with primary/security
mirrors.  Unfortunately, this prevents use of Landscape-managed
package mirrors as the mirror key for the Landscape-hosted repository
cannot be provided.

This patch allows the same key-related fields usable on "sources"
entries to be used on the "primary" and "security" entries as well.

LP: #1925395
Ensure no Traceback when 'chef_license' is set
Rocky Linux is a RHEL-compatible distribution so all changes that have
been made should be trivial.
Ensure we've got a clean environment before we restart the network.

In some cases, the `sh /etc/netstart` is not enough to restart the
network. A previous default route remains in the route table and
as a result the network is broken.
Also `sh /netstart` does not kill `dhclient`.
The problen happens for instance with OVH OpenStack SBG3.
dhclient output that contains brackets for pxe variables will break 
the dhclient parsing regex line. This fix retains the current                   
functionality while fixing this particular issue.
Security scanners are often simple minded and complain on arbitrary
settings such as file permissions. For /var/log/* having world read is
one of these cases.
Reuse the FreeBSD logic to be able to switch between Python3 versions
easily.
instance-data.json redacts sensitive data for non-root users. Since user
data is consumed as root, we should be consuming the non-redacted data
instead.

LP: #1931392
- Mostly based on FreeBSD, the main exception is that
  `find_devs_with_on_freebsd` does not work.
- Since we cannot get the CDROM or the partition labels,
  `find_devs_with_on_dragonflybsd()` has a more naive approach and
  returns all the block devices.
- small document update for ReportEventStack explaining post_files
parameter
- small unit test for test_reporting demonstrating the close of an
event with optional post_files list
@number5 number5 deleted the branch stackriot-labs:master July 8, 2021 02:11
@number5 number5 deleted the master branch July 8, 2021 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.