Skip to content
This repository has been archived by the owner on Jun 17, 2024. It is now read-only.

Add lineage export / import #2

Open
cmgrote opened this issue Jun 15, 2018 · 3 comments
Open

Add lineage export / import #2

cmgrote opened this issue Jun 15, 2018 · 3 comments
Labels
enhancement New feature or request

Comments

@cmgrote
Copy link
Contributor

cmgrote commented Jun 15, 2018

In particular, for operational metadata (ie. OMD XML)

@cmgrote cmgrote added the enhancement New feature or request label Jun 15, 2018
@cmgrote
Copy link
Contributor Author

cmgrote commented Jun 15, 2018

Will need to:

  • translate hostnames in OMD XML from source engine to target engine (source we take from what's in the OMD XML, target we can infer from Ansible vars?)
  • enable lineage in target project (so operational lineage shows up after import)
  • import the OMD XML (using istool)
  • setup connection mappings

Ideally should also include some optional logic that prevents loading the same OMD that already exists in the target side:

  • query OMD (is this possible via REST?) to determine whether there are any changes to load from an incoming OMD XML

Connection mappings can be setup via REST API to data_connection_mapping object, and its bound_to_database property. Automatically setting up connection mappings will rely on the common metadata to first be present to actually do the binding... Will need to check whether the target for bound_to_database is actually present before trying to do the mapping.

Should optionally limit the OMD's we import to only those where there is a unique set of runtime parameters (only unique parameters show in lineage anyway, so would be needless duplicate processing to load multiple runtimes of the same set of parameters -- it won't have any impact on lineage). There does not appear to be any way to query this information -- or even export it -- only to import it and purge it. So we would likely need to keep some kind of cache of the parameter combinations ourselves...

@cmgrote
Copy link
Contributor Author

cmgrote commented Jul 6, 2018

Would be nice to only lookup and attempt to enable lineage on the unique projects (rather than every project returned by the replacement step, in which there will likely be many duplicates).

@cmgrote
Copy link
Contributor Author

cmgrote commented Mar 10, 2020

Also: currently all of the uri calls will result in the opening of a new session against Information Server, and they won't close it at the end. Should fix this to both re-use any session that has initially been opened as well as closing the sessions at the end of their use (so they don't just hang around until they eventually timeout).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant