Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples and templates around 'metadata' properties is not fully updated. #46

Open
afoerster opened this issue Aug 9, 2018 · 1 comment

Comments

@afoerster
Copy link
Contributor

afoerster commented Aug 9, 2018

Before, specific metadata tags such as META_SOURCE were added to a table. A later addition added a metadata tag so arbitrary fields could be treated as metadata and added to hive table properties

table:
  metadata:
     source: some relational db
     contactInfo: person@company.com

Some templates still use the old metadata format: https://github.com/Cargill/pipewrench/blob/d0b06fbe1876481ace14f48f8adee45563193b2d/templates/sqoop-parquet-hdfs-hive-merge/create-report-table.sql

This example will probably run with the new format but still has the old tags: https://github.com/Cargill/pipewrench/blob/422f396baa726c01f78a7dcb3dc1aa98182083a1/examples/sqoop-parquet-full-load/tables.yml

All the 'old' style metadata should be removed.

@sada3390
Copy link

sada3390 commented Oct 26, 2018

I am working on it Tony. @afoerster

This was referenced Oct 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants