- Scrape Data from that Website
- Transform the data and
- Create a Dashboard using Tableau.
- Find a story using the dataset.
Data consist of every university rank, name , location, points from 2018 to 2022. Not only that, inside of each university, their latest information is also collected.
You can run this code and collect datasets. Thus you can build your own tableau Dashboard. To do this you have to follow these instructions.
-
Download Google Chrome Driver from based on your device:
unzip the downloaded file and collect the path where The Driver is saved. -
Install Python on your device from Here
-
Download this Folder file, unzip the downloaded file, And Rename the file name.
-
(Optional) Create an environment for this project inside of the unzipped file. Click right button of mouse and select Open in Terminal and write these code one by one
Set-ExecutionPolicy Unrestricted
pip install virtualenv
or
pip3 install virtualenv
virtualenv env
env\Scripts\Activate.ps1
- Then in the terminal write
pip install -r requirements.txt
or
pip3 install -r requirements.txt
- After Completing install packages, write
cd scripts
Hit Enter
- Then Write
python scrapper_transformer.py
- Hit Enter and Wait for 5 hours, Please Do not Touch Anything.
After Completing all of works You'll get 5 Outputs in csv Format
Unprocessed Data:
-
data1.csv
-
data2.csv
-
data3.csv
Processed or Transformed Data:
-
QS_World_Universities_Data_Life_Science_Medicine.csv
-
Qs_World_Ranked_University_Name_Point_2018_to_2022_Life_Science_Medicine.csv
If you did everything according to the above instructions then Data Transformation has already been done already.
In the data transformation a few things were completed..
-
Dropping unnecessary columns.
-
Clearing data from unwanted punctuation.
-
Creating a new Row based on a particular column(s).
-
Converting data type of columns.
-
Splitting one column into two.
-
Save Clear wanted dataset.
-
Using 'QS_World_Universities_Data_Life_Science_Medicine.csv' dataset I've created This Dashboard with That Story
- USA, UK, Australia are still open for Bachelor admissions.
- Sydney and Monash University have the largest number of international students. Due to better living opportunities, job benefits, scholarships and deductible expenses.
- More faculty or more students does not improve a university's score. Improved by education.
- The USA offers the best General programs in the world. Always open for International students.
- Most top universities have an average of 55-70 H-Index citations.
- There are very few opportunities for students who want to do Masters in Europe.
-
Based on 'Qs_World_Ranked_University_Name_Point_2018_to_2022_Life_Science_Medicine.csv' dataset another Dashboard has been created with Story
- We can see that even on average Canada has the Most good ranked Universities.
- In 2018 Harverd was the 1st ranked university among all Universities in the world.
- China has numerous best universities in Asia.
- The better the point, the better the rank.
- Harvard, Oxford universities retain their ranks as in the past, some universities like Duke University have their ranks changing over the years.
-
Around 4-5 hours to complete the entire process from start to finish depending on the user's device.
-
Do not touch, minimize or close any pop up Google Chrome.
-
Data might be changed over time.
This entire project is for Data Analysis
. It's suitable for beginners. This project can be executed by anyone. Due to the webdriver's extensive website trip, data collection takes a lengthy time.