Update Guide (#958)

* update tests and few other fixes

* Clean HEAD

* Clean HEAD

* Clean HEAD

* guide

CONVERSION_GUIDE.md

@@ -35,7 +35,7 @@ These are the variables that are used to connect the platform and are the same a
 
 ### Connection Variables
 
-These are the connection variables, These are used to connect to the platform.
+These are the connection variables, These are used to connect to the platform. These replace things like controller_hostname, ah_host, and other global connection variables.
 
 |Variable Name|Default Value|Required|Description|Example|
 |:---|:---:|:---:|:---|:---|
@@ -122,8 +122,103 @@ controller_configuration vars:
 
 eda_configuration vars:
 
-- eda_credentials
 - eda_controller_tokens
-- eda_projects
+- eda_credential_types
+- eda_credentials
 - eda_decision_environments
+- eda_event_streams
+- eda_projects
 - eda_rulebook_activations
+
+## Converted variables
+
+### Controller
+
+|Previous Name|New Name|
+|:---|:---|
+|`controller_hostname`|`aap_hostname`|
+|`controller_hostname`|`aap_username`|
+|`controller_password`|`aap_password`|
+|`controller_oauthtoken`|`aap_token`|
+|`controller_validate_certs`|`aap_validate_certs`|
+|`controller_request_timeout`|`aap_request_timeout`|
+|`controller_user_accounts`|`aap_user_accounts`|
+|`controller_teams`|`aap_teams`|
+
+### Hub
+
+|Previous Name|New Name
+|`ah_host`|`aap_hostname`|
+|`ah_username`|`aap_username`|
+|`ah_password`|`aap_password`|
+|`ah_validate_certs`|`aap_validate_certs`|
+|`ah_token`|`aap_token`|
+|`ah_users`|`aap_user_accounts`|
+|`ah_groups`|`aap_teams`|
+
+### Eda
+
+|Previous Name|New Name
+|`eda_host`|`aap_hostname`|
+|`eda_username`|`aap_username`|
+|`eda_password`|`aap_password`|
+|`eda_token`|`aap_token`|
+|`eda_validate_certs`|`aap_validate_certs`|
+|`eda_request_timeout`|`aap_request_timeout`|
+|`eda_users`|`aap_user_accounts`|
+
+### Role Vars
+
+The following role vars have been updated as well. there are too many as its one for each role, so will go into how the name is crafted.
+
+For example the previous var of
+controller_configuration_projects_loop_delay
+is still
+controller_configuration_projects_loop_delay
+however
+ah_configuration_collection_repository_async_delay
+is now
+ah_configuration_collection_repository_loop_delay
+
+These are the following global variables.
+
+`aap_configuration_enforce_defaults`
+`aap_configuration_secure_logging`
+`aap_configuration_async_retries`
+`aap_configuration_async_delay`
+`aap_configuration_loop_delay`
+`aap_configuration_async_dir`
+
+This is the format that is used for each using the appropriate prefix
+
+Prefixes
+
+- controller_configuration
+- ah_configuration
+- gateway
+- eda_configuration_
+
+Format:
+`prefix`+`role_name`+_enforce_defaults`
+`prefix`+`role_name`+_secure_logging`
+`prefix`+`role_name`+_async_retries`
+`prefix`+`role_name`+_async_delay`
+`prefix`+`role_name`+_loop_delay`
+`prefix`+`role_name`+_async_dir`
+
+use these to tweak how the role runs, this is particulary useful for projects syncing, long lists of job templates, and other tasks that can take a while to loop around. These all have defaults, and each global var is described above as to what it does.
+
+## Dispatch changes
+
+The dispatch role has changed to use the following order:
+
+- gateway roles
+- hub roles
+- controller roles
+- eda roles
+
+These loop through all the applicable roles to create objects in the AAP. They will skip the role if the variable used in that role is not defined. you can tweak each services set of roles, or only run a single services roles by using the `aap_configuration_dispatcher_roles` variables. Refer to the [Dispatch role readme](roles/eda_controller_tokens/README.md) for more information.
+
+## Roles moved
+
+Any role that was not creating and managing objects on the AAP was moved to the [extended collection](https://github.com/redhat-cop/aap_configuration_extended). This includes lookup plugins, filtree and other roles.

roles/dispatch/README.md

@@ -6,26 +6,230 @@ An Ansible Role to run all roles in the infra.aap_configuration collection.
 
 ## Variables
 
-Each role has its own variables, for information on those please see each role which this role will call. This role has one key variable `gateway_dispatch_roles` and its default value is shown below:
+This is a meta role, its purpose is to run the other roles in the collection, it does not run all of them, and can be used to call roles in a custom order, The control variable is the `aap_configuration_dispatcher_roles` which will pull roles from the different services and run them, If you wish to just run a subset of services from our default lists remove entries from this var.
 
 ```yaml
-gateway_dispatch_roles:
-  - {role: settings, var: settings_list, tags: settings}
-  - {role: users, var: aap_user_accounts, tags: users}
-  - {role: authenticators, var: authenticators_list, tags: authenticators}
-  - {role: authenticator_maps, var: authenticator_maps_list, tags: authenticator_maps}
-  - {role: http_ports, var: http_ports_list, tags: http_ports}
-  - {role: organizations, var: organizations_list, tags: organizations}
-  - {role: teams, var: platform_teams, tags: teams}
-  - {role: service_clusters, var: gateway_service_clusters, tags: service_clusters}
-  - {role: service_keys, var: service_keys_list, tags: service_keys}
-  - {role: service_nodes, var: gateway_service_nodes, tags: service_nodes}
-  - {role: services, var: gateway_services, tags: services}
-  - {role: routes, var: gateway_routes, tags: routes}
-  - {role: role_user_assignments, var: role_user_assignments_list, tags: role_user_assignments}
+aap_configuration_dispatcher_roles: >
+  {{ gateway_configuration_dispatcher_roles
+   + ah_configuration_dispatcher_roles
+   + controller_configuration_dispatcher_roles
+   + eda_configuration_dispatcher_roles
+  }}
 ```
 
-Note that each item has three elements:
+In addition each service has its own subset of roles, and each role has its own tag that can be used as well.
+
+### Gateway Roles
+
+```yaml
+gateway_configuration_dispatcher_roles:
+  - role: gateway_authenticators
+    var: gateway_authenticators
+    tags: authenticators
+  - role: gateway_authenticator_maps
+    var: gateway_authenticator_maps
+    tags: authenticator_maps
+  - role: gateway_settings
+    var: gateway_settings
+    tags: settings
+  - role: gateway_applications
+    var: aap_applications
+    tags: applications
+  - role: gateway_http_ports
+    var: http_ports_list
+    tags: http_ports
+  - role: gateway_organizations
+    var: aap_organizations
+    tags: organizations
+  - role: gateway_service_clusters
+    var: gateway_service_clusters
+    tags: service_clusters
+  - role: gateway_service_keys
+    var: gateway_service_keys
+    tags: service_keys
+  - role: gateway_service_nodes
+    var: gateway_service_nodes
+    tags: service_nodes
+  - role: gateway_services
+    var: gateway_services
+    tags: services
+  - role: gateway_role_user_assignments
+    var: gateway_role_user_assignments
+    tags: role_user_assignments
+  - role: gateway_routes
+    var: gateway_routes
+    tags: routes
+  - role: gateway_teams
+    var: aap_teams
+    tags: teams
+  - role: gateway_users
+    var: aap_user_accounts
+    tags: teams
+```
+
+#### Hub Roles
+
+```yaml
+ah_configuration_dispatcher_roles:
+  - role: hub_namespace
+    var: ah_namespaces
+    tags: namespaces
+  - role: hub_collection
+    var: ah_collections
+    tags: collections
+  - role: hub_ee_registry
+    var: ah_ee_registries
+    tags: registries
+  - role: hub_ee_repository
+    var: ah_ee_repositories
+    tags: repos
+  - role: hub_ee_repository_sync
+    var: ah_ee_repository_sync
+    tags: reposync
+  - role: hub_ee_image
+    var: ah_ee_images
+    tags: images
+  - role: hub_ee_registry
+    var: ah_ee_registries
+    tags: registry
+  - role: hub_ee_registry_index
+    var: ah_ee_registries
+    tags: ee_indices
+  - role: hub_ee_registry_sync
+    var: ah_ee_registries
+    tags: regsync
+  - role: hub_collection_remote
+    var: ah_collection_remotes
+    tags: collectionremote
+  - role: hub_collection_repository
+    var: ah_collection_repositories
+    tags: collectionsrep
+  - role: hub_collection_repository_sync
+    var: ah_collection_repositories
+    tags: collectionsrepsync
+```
+
+#### Controller Roles
+
+```yaml
+controller_configuration_dispatcher_roles:
+  - role: controller_settings
+    var: controller_settings
+    tags: settings
+  - role: controller_organizations
+    var: aap_organizations
+    tags: organizations
+    assign_galaxy_credentials_to_org: false
+    assign_default_ee_to_org: false
+    assign_notification_templates_to_org: false
+  - role: controller_instances
+    var: controller_instances
+    tags: instances
+  - role: controller_instance_groups
+    var: controller_instance_groups
+    tags: instance_groups
+  - role: controller_labels
+    var: controller_labels
+    tags: labels
+  - role: controller_credential_types
+    var: controller_credential_types
+    tags: credential_types
+  - role: controller_credentials
+    var: controller_credentials
+    tags: credentials
+  - role: controller_credential_input_sources
+    var: controller_credential_input_sources
+    tags: credential_input_sources
+  - role: controller_execution_environments
+    var: controller_execution_environments
+    tags: execution_environments
+  - role: controller_applications
+    var: aap_applications
+    tags: applications
+  - role: controller_notification_templates
+    var: controller_notifications
+    tags: notification_templates
+  - role: controller_organizations
+    var: aap_organizations
+    tags: organizations
+    assign_galaxy_credentials_to_org: true
+    assign_default_ee_to_org: true
+    assign_notification_templates_to_org: true
+  - role: controller_projects
+    var: controller_projects
+    tags:
+      - inventories
+      - projects
+  - role: controller_inventories
+    var: controller_inventories
+    tags: inventories
+  - role: controller_inventory_sources
+    var: controller_inventory_sources
+    tags:
+      - inventories
+      - inventory_sources
+  - role: controller_inventory_source_update
+    var: controller_inventory_sources
+    tags:
+      - inventories
+      - inventory_sources
+  - role: controller_hosts
+    var: controller_hosts
+    tags:
+      - inventories
+      - hosts
+  - role: controller_bulk_host_create
+    var: controller_bulk_hosts
+    tags:
+      - inventories
+      - bulk_hosts
+  - role: controller_host_groups
+    var: controller_groups
+    tags:
+      - inventories
+      - host_groups
+  - role: controller_job_templates
+    var: controller_templates
+    tags: job_templates
+  - role: controller_workflow_job_templates
+    var: controller_workflows
+    tags: workflow_job_templates
+  - role: controller_schedules
+    var: controller_schedules
+    tags: schedules
+  - role: controller_job_launch
+    var: controller_launch_jobs
+    tags: job_launch
+  - role: controller_workflow_launch
+    var: controller_workflow_launch_jobs
+    tags: workflow_launch
+
+```
+
+#### Eda Roles
+
+```yaml
+eda_configuration_dispatcher_roles:
+  - role: eda_credentials
+    var: eda_credentials
+    tags: credential
+  - role: eda_controller_tokens
+    var: eda_controller_tokens
+    tags: controller_token
+  - role: eda_projects
+    var: eda_projects
+    tags: project
+  - role: eda_decision_environments
+    var: eda_decision_environments
+    tags: decision_environment
+  - role: eda_rulebook_activations
+    var: eda_rulebook_activations
+    tags: rulebook_activation
+```
+
+#### Dispatch role var list keys
+
+Each role in each service has its own variables, for information on those please see each role which this role will call. Each role is called and each item has three elements:
 
 - `role` which is the name of the role within infra.aap_configuration
 - `var` which is the variable which is used in that role. We use this to prevent the role being called if the variable is not set

roles/dispatch/defaults/main.yml

@@ -80,6 +80,7 @@ ah_configuration_dispatcher_roles:
   - role: hub_collection_repository_sync
     var: ah_collection_repositories
     tags: collectionsrepsync
+
 controller_configuration_dispatcher_roles:
   - role: controller_settings
     var: controller_settings