Skip to content

Commit

Permalink
add parquet, lab exploration
Browse files Browse the repository at this point in the history
created single parquet files for each table in CCRS. Cleaned the lab results table by renaming testnames to canonicals, removing obvious data entry errors, and finally adding units, testname, and testing group columns. Some initial exploration of the lab results data included.
  • Loading branch information
ripnyt-ripnyt authored Feb 22, 2023
1 parent e83f5fb commit 36cb2a9
Show file tree
Hide file tree
Showing 12 changed files with 313,692 additions and 0 deletions.
291,164 changes: 291,164 additions & 0 deletions explore/LabResultTHC_Moisture_explore.ipynb

Large diffs are not rendered by default.

15,956 changes: 15,956 additions & 0 deletions inflation_errors/df_bad_rows.csv

Large diffs are not rendered by default.

76 changes: 76 additions & 0 deletions inflation_errors/df_results.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
file,columns_removed
Areas_0.tsv,145
Inventory_2.tsv,0
SalesDetail_37.tsv,0
SaleHeader_29.tsv,0
SaleHeader_51.tsv,0
SaleHeader_13.tsv,0
SaleHeader_12.tsv,0
Plant_2.tsv,0
SaleHeader_56.tsv,0
SalesDetail_16.tsv,0
SaleHeader_41.tsv,0
Contacts_0.tsv,0
SalesDetail_62.tsv,0
SalesDetail_27.tsv,0
Product_5.tsv,2332
Inventory_5.tsv,0
SalesDetail_49.tsv,0
SaleHeader_27.tsv,0
SaleHeader_46.tsv,0
Plant_0.tsv,0
SalesDetail_63.tsv,0
SalesDetail_50.tsv,0
SaleHeader_42.tsv,0
Product_1.tsv,1287
InventoryAdjustment_1.tsv,96
SalesDetail_24.tsv,0
LabResult_1.tsv,0
SalesDetail_8.tsv,0
SaleHeader_54.tsv,0
Inventory_7.tsv,0
SalesDetail_39.tsv,0
SalesDetail_18.tsv,0
SaleHeader_7.tsv,0
SalesDetail_53.tsv,0
SaleHeader_15.tsv,0
SaleHeader_21.tsv,0
SalesDetail_54.tsv,0
SaleHeader_59.tsv,0
SaleHeader_39.tsv,0
SalesDetail_48.tsv,0
Inventory_0.tsv,3
SalesDetail_19.tsv,0
SalesDetail_6.tsv,3
SaleHeader_28.tsv,0
SalesDetail_30.tsv,0
SalesDetail_5.tsv,12
Inventory_15.tsv,0
SalesDetail_60.tsv,0
Inventory_12.tsv,2
SaleHeader_50.tsv,0
SaleHeader_35.tsv,0
SaleHeader_52.tsv,0
SalesDetail_15.tsv,0
SaleHeader_10.tsv,0
SalesDetail_25.tsv,0
Product_7.tsv,12072
Plant_8.tsv,0
SaleHeader_48.tsv,0
SalesDetail_56.tsv,0
SaleHeader_61.tsv,0
SaleHeader_40.tsv,0
SalesDetail_51.tsv,0
Inventory_1.tsv,0
SalesDetail_41.tsv,0
SalesDetail_2.tsv,0
SalesDetail_14.tsv,0
SalesDetail_34.tsv,0
SalesDetail_59.tsv,0
SalesDetail_47.tsv,0
SaleHeader_25.tsv,0
InventoryPlantTransfer_0.tsv,3
SalesDetail_11.tsv,0
SalesDetail_4.tsv,0
SaleHeader_11.tsv,0
SaleHeader_16.tsv,0
Loading

0 comments on commit 36cb2a9

Please sign in to comment.