README

Table of Contents

• Introduction
• Prerequisites
• Software Prerequisites
  • Required Python Modules
  • Installing Jekyll
    • Running Jekyll Locally
• Getting Started
• Uploading changes to Github
• Making changes on _config.yml and additional_config.yml
• Configuration
• Structure of the Project
• Preprocessing
  • Preprocessing for Adding Images
    • File type for images
  • Preprocessing Required for Linking Local Images
  • Preprocessing Required for Images on Google Drive
  • Adding categories
  • Editing the Record Pages HTML Template
• Displaying Map
• Customization
  • Map Functionality for Records
    • Markers for Record Maps
    • Title of the Markers for Record Maps
  • Adding Additional Metadata Inside the HTML head Tag
  • Adding Additional Stylesheet or Javascript
  • Search Functionality
  • Excluding Files from Search
  • Adding Images for a Record Page from Local Source
  • Adding Images for a Record Page from Google Drive
  • Photo Gallery for Records
• Important Pages
  • Collections Page
  • Record Page Front Matter Template
    • Components of the Record Page HTML Template
• Generating Record Pages
• Record Page
• Terminology

Introduction

This is a template that uses Jekyll and is based off feeling-responsive theme by Phlow.

The purpose of this project is to generate a data-driven website from a CSV. Please also refer to the Terminology section.

To get started, please follow the steps in this README.

Prerequisites

This project uses Jekyll as its static site generator, therefore users are highly recommended to have a basic understanding of Jekyll, please refer to Jekyll official website for more information. The layout used in this project make use of front matter variables.

Users will need to have basic understanding of Python3, especially topics such as variables, if statements and function, in order to make modifications to the scripts provided in the _import folder.

Also users will need to make modification to the record.html file to display their required front matter variables, therefore users will need to have a basic understanding of HTML, especially tables and some basic understanding of liquid templating to modify the layout for record.html.

It is recommended to also have basic understanding of CSS and Javascript, however it is not required if user wish to just generate a data driven website without any further customisation.

Software Prerequisites

To run the Python scripts provided in the _import folder, the user will need to have Python version 3.5.2 or above installed.

To test the website locally, the user will need to have Jekyll installed. It is also assumed that you are using Ubuntu operating system if you wish to test your website locally before uploading to Github Pages.

NOTE: If users do not have required Python Modules and attempt to run the Python scripts provided, they will receive errors. If errors occur, please install the required Python modules that are missing. Please refer to the Required Python Module section in regards to which modules are needed for the Python scripts provided.

Please refer to Python3 documentation if you need more information on Python modules and how to install Python modules.

Back to TOC

Required Python Modules

The required modules to run the Python scripts in _import folder are in the requirements.txt file. Please download the modules listed in that file. Navigate to the _import folder and run pip install -r requirements.txt

Back to TOC

Installing Jekyll

Please refer to Jekyll official documentation for instructions on how to install Jekyll on your system.

If you are using a Windows system, please refer to the page regarding installing Jekyll on Windows.

Running Jekyll Locally

Assuming Jekyll is installed, open up the terminal at the root of the folder of the project to see your website locally. Please enter the following command in the terminal:

bundle exec jekyll serve --config _config.yml,_config_dev.yml

Another option is to run the shell script local_jekyll_build.sh located at the root path of this project.

Back to TOC

Getting Started

This section will briefly discuss the process required in this project where the outcome is a website created. Please ensure that you have read the Prerequisites section and the Software Prerequisites section before you continue with the following steps.

1. Customise the project configuration options such as the title of the project, the logo, navigation bar that appears at the top of the page. Refer to Configuration section for more information.
2. Using the record.html located inside the _layouts folder as a template, work through the file and change it according to your requirements. Refer to the Editing the Record Pages HTML Template section for more information.
3. Auto generate record pages using the script provided. Refer to the Generating Record Pages section for more information.

Back to TOC

Uploading changes to Github

This section is only relevant for users hosting their website on GitHub. Because this is a static website, every time users wish to push their changes (assuming the website is already on Github):

1. Delete the _posts folder.
2. Recompile the record pages by running the makeSite.py Python script and additional scripts if needed.
3. git add then git commit, and finally git push to the remote repo.

Making changes on _config.yml and additional_config.yml

If users are running Jekyll locally to see their changes, please stop the server, then re-run to see the changes made in the _config.yml or additional_config.yml file.

Back to TOC

Configuration

The _config.yml file contains configuration options for the website. For basic configuration, do the following steps.

1. Open the _config.yml file and work through it.

   • The main front matter variables for basic customisation is:
     • title
     • slogan
     • description
     • credits
     • author
     • url
     • baseurl
     • More details can be found on GitHub article regarding Jekyll configuration and Jekyll offical documentation configuration section.
   • Rather than adding custom data directly into the _config.yml file that can be accessed throughout the site via Liquid templating, users are recommended to add it into the additional_config.yml file which is located in the _data folder.
   • For example, the variable google-map-marker can be accessed via site.data.additional_config.google-map-marker.
   • Refer to Jekyll datafile page for more information.

2. Add your logo.png (if you have one) to the /assets/img/ folder.

3. Open _data/socialmedia.yml and add your own social media links.

4. Open _data/navigation.yml and customise your navigation.

   • A navigation link consist of 3 components, title, url and side.

     • title is the name of the link.
     • url is the relative link to the page.
     • side is whether the link would be located on the left-hand or right-hand side of the navigation bar. It is recommended to put the navigation links on the left hand side to keep the Search bar at its current location.

5. Open _data/language.yml and translate the theme if necessary.

6. Open _data/services.yml and customise links in the footer.

7. Open _data/authors.yml and edit author information and set default author in config.yml.

Back to TOC

Structure of the Project

This section will explain the structure of the project and describe the top-level folders for the project:

• assets folder contain the CSS, Javascript used in the project and also images that are used for the site such as the logo and default images for records.
• _data contains the configuration YAML files for the project.
• _images folder contains the images used in the site.
• _import contains scripts to generate record pages from a CSV and link images to the associated record page.
• _includes contains content that can be included into files.
• _pages contains pages of the website.
• _posts contains the YAML files for the record pages.
• _sass contains SASS partials of the project.
• _site contains the generated site that Jekyll outputs. This folder is added into .gitignore file.

Back to TOC

Preprocessing

Before generating record pages, ensure your data is well-formed for minimal hassle. A record should have a column that contains the unique identifier for that record.

Back to TOC

Preprocessing for Adding Images

While it is possible to manually link the images to a record page, there are two other ways to link images to a record page.

1. Link local images to record pages using the Python script local_images.py in the _import folder. Please see the preprocessing required for linking local images section for more information.
2. Read a CSV that contains Google Drive Links to images for records and write it to the associated record page using the google_drive.py Python script in the _import folder.

Back to TOC

File type for images

Please ensure your images are in one of the following types so that the script is able to recognise the file as images:

1. .png
2. .jpg
3. .jpeg

Back to TOC

Preprocessing Required for Linking Local Images

First, please ensure your files are of the types discussed in the file type section, please otherwise convert your image to one of the file type mentioned in that section.

In the root folder of the project, create a folder called images if it doesn't exist yet. Inside the images folder, group the images by the unique identifier of the record. For each record, create a new folder inside the images folder with the name matching the unique identifier for that record.

Example:

For example, given a record with the unique id 1000 and the following images associated with that record:

• 1000_Detail_Profile_of_RT.JPG
• 1000_Detail_RT2.JPG
• 1000_Large_RT.JPG
• 1000_Large_RT_Scale.JPG
• 1000_Overview_S.JPG
• 1000_Overview_year2009.JPG
• 1000_RT.JPG

The user will then need to create a new folder with the name 1000 and put all those images inside that folder. This results in the following structure:

images/
├── 1000
│   ├── 1000_Detail_Profile_of_RT.JPG
│   ├── 1000_Detail_RT2.JPG
│   ├── 1000_Large_RT.JPG
│   ├── 1000_Large_RT_Scale.JPG
│   ├── 1000_Overview_S.JPG
│   ├── 1000_Overview_year2009.JPG
│   ├── 1000_RT.JPG

Back to TOC

Preprocessing Required for Images on Google Drive

First, please ensure your files are of the types mentioned in the file type section, otherwise convert your image to one of the file type discussed in that discussion.

To use the provided google_drive.py Python script, the URL of the Google Drive images will need to be extracted into a CSV. The script assumes that the length of the unique identifier is uniform, and that the unique identifier is appended to the start of the file name.

Example:

Assume a record with the unique identifier of 1000 has 3 images associated with it:

• image-one.jpg
• image-two.jpg
• image-three.jpg

The user will then append 1000 to the filename of those images:

• 1000_image-one.jpg
• 1000_image-two.jpg
• 1000_image-three.jpg

Back to TOC

Adding categories

In the Collections Page, record pages are grouped by categories.

In the following screenshot, there are 5 categories:

• 1000-1999
• 2000-2999
• 3000-3999
• 4000-4999
• 5000-5999

Each category corresponds to a tab. For the purpose of this section, you can ignore the map tab, because it is not a category.

There are several pages listed under the 1000-1999 category, such as "TRAP Mound - 1000", "TRAP Mound - 1001". This is because of how the records are organized for this project, however your project could be different.

The category front matter variable determines on which category appears in the collection page and which category or categories the record page will be in.

Example:

---
...
title: TRAP Mound - 1000
record_id: '1000'
categories:
- 1000-1999
header: no
...

---

Notice that the categories front matter variable is a list that contains one value, 1000-1999, this indicates that the record page will appear under the 1000-1999 category. It is possible to have a record page under multiple categories such as this:

categories:
- 1000-1999
- 2000-2999

This example above will make that record page appear in two categories, namely, 1000-1999 and 2000-2999.

Please refer to Collections Page section for more details regarding the collection page.

Back to TOC

Editing the Record Pages HTML Template

The record pages use _layout/record.html as its layout. To modify the layout for record pages, users will need to understand the components of the file. Have a look at the components of the record page HTML, please see the Components of the Record Page Template section.

Back to TOC

Displaying Map

This project uses Google Map Javascript API. If the user wishes to use Google Map API for this project, please go to Google Map website and get a API key, then go to additional_config.yml and set the value of the variable javascript_google_map to true.

Example:

javascript_google_map: true

Customization

Before auto generating record pages, the user can customise the site. This section will discuss some customizations available to the user.

Back to TOC

Map Functionality for Records

The map functionality for records is split into several components:

1. The div element with the id record_map is located in the record.html file inside the _layouts folder. The element has 3 attributes which store the title, latitude and longitude of that record.
2. The CSS required for the styling of the div element which is in the sass file _google_maps.scss.
3. The Javascript function that uses the Maps Javascript API. This is located in the _custom_js/record_pages_map.js and importing that script is located in the _includes/additional_helper/additional_footer.html file.

The default setting for the map functionality is that it uses the latitude and longitude front matter variables of the record page. So if users wish to have a map, the column name in the CSV must also correspond to that name.

NOTE: It is fine to have the column names "Latitude" and "Longitude" in the CSV, because the script that generates a page for each row then converts the column names to lowercase when it passes it into the front matter for the record page.

• IMPORTANT: The format of the coordinates should also be in decimal degrees which is what Google Map API uses.

Back to TOC

Markers for Record Maps

The google-map-marker variable in the additional_config.yml file is used to determine whether to add a marker that points to the location of the record. If the user does not wish to have a Google Map marker then the user can edit the google-map-marker variable and change the value from true to false, and if they wish to have the marker back then they would reverse it, changing the value from false to true.

Back to TOC

Title of the Markers for Record Maps

The title front matter variable in the record page is used as the title of the Google Map marker.

Back to TOC

Adding Additional Metadata Inside the HTML head Tag

In the _includes/helper folder, there is a HTML file called head.html. This contains all the default information about the page inside the <head> tags.

If the user wants to add additional information enclosed inside the <head> tag, it is recommended to add it into the additional_head.html file to distinguish between the default metadata and the new metadata defined by the user. The additional metadata will be added after the default metadata because we are using Jekyll include tag to include the content from additional_head.html into the head.html file. However, the user is free to modify the head.html file.

Back to TOC

Adding Additional Stylesheet or Javascript

For users that wish to customise the styling and behavior of their website by adding additional stylesheet or Javascript, please refer to the additional additional javascript or stylesheet document for more information.

Back to TOC

Search Functionality

The current search function allows the user to search records by title and record_id variables. If users wish to customise their search, please read this tutorial which is used as the starting point to implement searching in this project.

• pages/search.md is the file which contains the search data.
• assets/js/search.js is the file which contains the Javascript logic to perform the search.

Back to TOC

Excluding Files from Search

To explicitly exclude a page from search result, please add exclude_from_lunr variable to the page front matter and set the value to true with no surrounding quotes, that means, "true" or 'true' is not correct.

Example: Correct value

exclude_from_lunr: true

Example: Incorrect values

exclude_from_lunr: "true"

exclude_from_lunr: 'true'

Back to TOC

Adding Images for a Record Page from Local Source

Please ensure to read and follow the instructions provided in the preprocessing required for linking local images section before proceeding with this section.

Please follow the steps listed below:

1. If you have run the make_site.py Python script to generate the record pages, then you can skip this step, please otherwise read the generating record pages section and follow the instructions in that section.
2. Run the local_images.py Python script. Once that is done, you have finished linking images from the images/ folder to its associated record page.

Once you have finished all the steps, run the website to test if the images were successfully associated with the record page.

Back to TOC

Adding Images for a Record Page from Google Drive

Please ensure to read the Generating Google Drive Link to CSV page to generate the CSV that contains the Google Drive CSV.

Once you have the generated CSV, please follow the following steps:

1. Ensure that the CSV file is in the _import folder.
2. Ensure the value for the key google_drive_csv_file in customizable-variables.yaml is the same as the file name of the CSV with the Google Drive Links.
3. If you have run the make_site.py Python script to generate the record pages, you can skip this step and move onto step 4, otherwise, please read the Generating Record Pages section and follow the instructions in that section.
4. Run the google_drive.py Python script. Once that is done, you have finish adding images from Google Drive.

Once you have finish all the steps, run the website to test if the images were successfully associated with the record page.

Back to TOC

Photo Gallery for Records

The photo gallery is implemented using slick. There are two photo galleries in a record page, one is used to display photos for that record and the second is used as a navigation. The implementation is split into three parts:

1. The HTML structure is defined in the template HTML page for the records, which by default is in _layouts/record.html.
2. The CSS that is used to style the photo gallery such as the the navigation buttons for the photo gallery is in assets/slick/slick-theme.css.
3. The Javascript is used to define the logic of the photo gallery is in assets/custom_js/slick-settings.js.

Back to TOC

Important Pages

Collections Page

The _pages/pages-root-folder/collections.md is the web page that is used to display the links to all the record pages. The organization of the pages is grouped by categories where one tab corresponds to a category. A record page can appear in one or more category. To add a record page under a category, add the name of the category in the YAML list for that record page under the categories key.

If the user wishes to change the content of the page, they can edit the collections.md file in the _pages/pages-root-folder/ folder.

Back to TOC

Record Page Front Matter Template

The YAML template for record pages is the _import/template.yaml file. The script that will generate record pages will use that as a template for the record pages. The layout file that _import/template.yaml uses is the _layout/record.html file.

Back to TOC

Components of the Record Page HTML Template

1. Subheadline
2. Title
3. Table
4. Photo gallery
5. Content tabs

Back to TOC

Generating Record Pages

1. Edit the customizable-variables.yaml files to suit your requirements.
2. customise the category_generation.py to your requirements. Refer here for more information regarding the script. Refer to Adding Categories section for more information about categories.
3. Open your terminal and install the required dependencies listed in the requirements.txt file.
4. Run the Python3 script makeSite.py.

The rest of this section will discuss how to generate record pages in detail.

In the _import folder there is a Python script named makeSite.py. The purpose of the script is to generate a separate YAML file for each row in the CSV. Each column corresponds to a front matter variable in the YAML file. The variable will be the name of the column with all space characters replaced with the underscore character _ and the characters are converted to all lowercase (EXCEPT the unique identifier) .

NOTE: The unique identifier will be stored into the front matter variable record_id. You would need to provide the column name (case-sensitive) so that the script can extract the id for the record.

Example:

There is a CSV file with 3 columns, 'TRAP ID', 'Max diameter', 'Surrounding Land Use', where the values are respectively 1000, 34, "Annual Agriculture" for a row. Then in the corresponding record page for that row, the front matter variable will be in the following format. If we choose 'TRAP ID' as the record_id, the following code snippet will be the result.

record_id: '1000'
max_diameter: '34'
surrounding_land_use: Annual Agriculture

Back to TOC

Record Page

A record page is a page in the _post folder. It contains front matter variables which store the attributes of that record. The front matter variables are enclosed between triple dashes.

Here is a simple example:

---
layout: record
title: TRAP Mound - 1000

record_id: '1000'
---

In the example above, there are three front matter variable, namely layout, title, and record_id. The front matter variables are used by the template to display data specific to that record.

NOTE: The front matter variables must be the first thing in the file.

Back to TOC

Terminology

• record: a row in the CSV.
• record page: a markdown page that contains front matter variables, where a front matter variable corresponds to a column and its value for that record.
  • A post in Jekyll is in our case a record page.
• attribute: a column in the CSV.

Back to TOC