filetree_create with flatten enabled outputs duplicate entries and mal-formed yaml files.

[Joshua-rose]

Summary

files output by the filetree_create have multiple document starts --- and document ends ... See inventory_sources.yaml excerpt below.

Issue Type

• Bug Report

Ansible, Collection, Controller details

ansible --version

ansible [core 2.15.3]
  config file = None
  configured module search path = ['/home/joshua/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/joshua/.local/lib/python3.10/site-packages/ansible
  ansible collection location = /home/joshua/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/joshua/.local/bin/ansible
  python version = 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

ansible-galaxy collection list

# /home/joshua/.ansible/collections/ansible_collections
Collection                     Version
------------------------------ -------
infra.controller_configuration 2.6.0

# /home/joshua/.local/lib/python3.10/site-packages/ansible_collections
Collection                     Version
------------------------------ -------
amazon.aws                     6.3.0
ansible.netcommon              5.1.2
ansible.posix                  1.5.4
ansible.utils                  2.10.3
ansible.windows                1.14.0
arista.eos                     6.0.1
awx.awx                        22.6.0
azure.azcollection             1.16.0
check_point.mgmt               5.1.1
chocolatey.chocolatey          1.5.1
cisco.aci                      2.7.0
cisco.asa                      4.0.1
cisco.dnac                     6.7.3
cisco.intersight               1.0.27
cisco.ios                      4.6.1
cisco.iosxr                    5.0.3
cisco.ise                      2.5.14
cisco.meraki                   2.15.3
cisco.mso                      2.5.0
cisco.nso                      1.0.3
cisco.nxos                     4.4.0
cisco.ucs                      1.10.0
cloud.common                   2.1.4
cloudscale_ch.cloud            2.3.1
community.aws                  6.2.0
community.azure                2.0.0
community.ciscosmb             1.0.6
community.crypto               2.15.0
community.digitalocean         1.24.0
community.dns                  2.6.0
community.docker               3.4.8
community.fortios              1.0.0
community.general              7.3.0
community.google               1.0.0
community.grafana              1.5.4
community.hashi_vault          5.0.0
community.hrobot               1.8.1
community.libvirt              1.2.0
community.mongodb              1.6.1
community.mysql                3.7.2
community.network              5.0.0
community.okd                  2.3.0
community.postgresql           2.4.3
community.proxysql             1.5.1
community.rabbitmq             1.2.3
community.routeros             2.9.0
community.sap                  1.0.0
community.sap_libs             1.4.1
community.skydive              1.0.0
community.sops                 1.6.4
community.vmware               3.9.0
community.windows              1.13.0
community.zabbix               2.1.0
containers.podman              1.10.2
cyberark.conjur                1.2.0
cyberark.pas                   1.0.19
dellemc.enterprise_sonic       2.2.0
dellemc.openmanage             7.6.1
dellemc.powerflex              1.7.0
dellemc.unity                  1.7.1
f5networks.f5_modules          1.25.1
fortinet.fortimanager          2.2.1
fortinet.fortios               2.3.1
frr.frr                        2.0.2
gluster.gluster                1.0.2
google.cloud                   1.2.0
grafana.grafana                2.1.5
hetzner.hcloud                 1.16.0
hpe.nimble                     1.1.4
ibm.qradar                     2.1.0
ibm.spectrum_virtualize        1.12.0
infinidat.infinibox            1.3.12
infoblox.nios_modules          1.5.0
inspur.ispim                   1.3.0
inspur.sm                      2.3.0
junipernetworks.junos          5.2.0
kubernetes.core                2.4.0
lowlydba.sqlserver             2.1.0
microsoft.ad                   1.3.0
netapp.aws                     21.7.0
netapp.azure                   21.10.0
netapp.cloudmanager            21.22.0
netapp.elementsw               21.7.0
netapp.ontap                   22.7.0
netapp.storagegrid             21.11.1
netapp.um_info                 21.8.0
netapp_eseries.santricity      1.4.0
netbox.netbox                  3.13.0
ngine_io.cloudstack            2.3.0
ngine_io.exoscale              1.0.0
ngine_io.vultr                 1.1.3
openstack.cloud                2.1.0
openvswitch.openvswitch        2.1.1
ovirt.ovirt                    3.1.2
purestorage.flasharray         1.20.0
purestorage.flashblade         1.12.1
purestorage.fusion             1.6.0
sensu.sensu_go                 1.14.0
servicenow.servicenow          1.0.6
splunk.es                      2.1.0
t_systems_mms.icinga_director  1.33.1
telekom_mms.icinga_director    1.34.1
theforeman.foreman             3.12.0
vmware.vmware_rest             2.3.1
vultr.cloud                    1.8.0
vyos.vyos                      4.1.0
wti.remote                     1.0.5

Controller version

 ________________________________________________ 
<  Ansible Automation Platform Controller 4.4.8  >
 ------------------------------------------------ 
          \
          \   ^__^
              (oo)\_______
              (__)      A )\
                  ||----w |
                  ||     ||
                    

• ansible installation method: one of source, pip, OS package, EE

OS / ENVIRONMENT

WSL on Windows 11

NAME STATE VERSION

• Ubuntu Running 1

Desired Behavior

correctly formatted yaml documents preferably deduped.

Actual Behavior

---
controller_inventory_sources:
---
controller_inventory_sources:
  - name: "InvAsCode Cac Generation"
    description: ""
    organization: "<redacted>"
    source: "scm"
    source_project: "<redacted>"
    source_path: "<redacted>"
    inventory: "Cac generation inventory"
    update_on_launch: "True"
    overwrite: "False"
...
  - name: "InvAsCode Cac Generation"
    description: ""
    organization: "<redacted>"
    source: "scm"
    source_project: "<redacted>"
    source_path: "<redacted>"
    inventory: "Cac generation inventory"
    update_on_launch: "True"
    overwrite: "False"
...

Please give some details of what is actually happening.
Include a [minimum complete verifiable example] with:

• playbook / task

---
- hosts: all
  connection: local
  gather_facts: false
  vars:
    controller_username: "{{ vault_controller_username | default(lookup('env', 'CONTROLLER_USERNAME')) }}"
    controller_password: "{{ vault_controller_password | default(lookup('env', 'CONTROLLER_PASSWORD')) }}"
    controller_hostname: "{{ vault_controller_hostname | default(lookup('env', 'CONTROLLER_HOST')) }}"
    controller_validate_certs: "{{ vault_controller_validate_certs | default(lookup('env', 'CONTROLLER_VERIFY_SSL')) }}"

  pre_tasks:
    - name: "Setup authentication (block)"
      block:
        - name: "Get the Authentication Token for the future requests"
          ansible.builtin.uri:
            url: "https://{{ controller_hostname }}/api/v2/tokens/"
            user: "{{ controller_username }}"
            password: "{{ controller_password }}"
            method: POST
            force_basic_auth: true
            validate_certs: "{{ controller_validate_certs }}"
            status_code: 201
          register: authtoken_res

        - name: "Set the oauth token to be used since now"
          ansible.builtin.set_fact:
            controller_oauthtoken: "{{ authtoken_res.json.token }}"
            controller_oauthtoken_url: "{{ authtoken_res.json.url }}"
      no_log: "{{ controller_configuration_filetree_create_secure_logging | default('false') }}"
      when: controller_oauthtoken is not defined
      tags:
        - always


  roles:
    - infra.controller_configuration.filetree_create

  post_tasks:
    - name: "Delete the Authentication Token used"
      ansible.builtin.uri:
        url: "https://{{ controller_hostname }}{{ controller_oauthtoken_url }}"
        user: "{{ controller_username }}"
        password: "{{ controller_password }}"
        method: DELETE
        force_basic_auth: true
        validate_certs: "{{ controller_validate_certs }}"
        status_code: 204
...

• configuration file / list

# content of keys.yml
---
controller_username: '<redacted>'
controller_password: '<redacted>'
controller_hostname: <redacted>'
controller_validate_certs: false
cyberark_api_key: '<redacted>'
env: <redacted>
flatten_output: true

• error
  No console error produced.

STEPS TO REPRODUCE

run the following from the command line

ansible-playbook playbooks/export_with_filetree.yml -i inventory.yml -e @keys.yml -vvvvv -e output_path=$PWD/Output/Path

[djdanielsson]

@ivarmu any ideas?

[ivarmu]

Yes... needed to take care of the output creation in the same way it's done in the inventories... I'm working on it!