Big Data Case Study: Complete ETL

Implementing: Hadoop, Horton Works, Apache Ambari, Hive, Oozie, Data Warehosuing, Java, JDBC, MySQL Server/Workbench.

Complete setup and execution instructions found in README.docx.

Summarized Explanation of Modules Developed:

1 & 2. Develop a reporting system in Java to display results of specfic queries based on transactional and customer data of a credit card system database.

3. Transform the data to required specifications.

4. Load the transformed data into HDFS.

5. Automate the proccess of warehousing using Oozie.

6. Test reporting accuracy through visulization (client presentation).

Project Requirments Documents Contents:

case_study Flow.pdf - The proposed steps.

Credit Card Management System_SRD.pdf - Software requirments and complete ETL.

deliverable.xlsx - Description of functional requirments deliverables.

Functional Requirments.pdf - Features to be developed for the entire project.

Mapping Document.xlsx - Description of data transformation from MySQL Server into HDFS.

Source File Structure.xlsx - Table and data structure in MySQL Server.