Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorial for 3d explorer + replace former names with 3d explorer ever… #49

Merged
merged 7 commits into from
Mar 22, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/Scanner/docker/docker_compose.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ To use your own local database, we provide a docker compose recipe that:


### Use pre-built docker image
You can use the pre-built images `romidb` & `plantviewer`, accessible from the ROMI dockerhub, to eaasily test & use the plantviewer with your own database [^3].
You can use the pre-built images `romidb` & `plantviewer`, accessible from the ROMI dockerhub, to easily test & use the `plant 3d explorer` with your own database [^3].

[^3]: https://docs.docker.com/compose/compose-file/#image

Expand Down
5 changes: 2 additions & 3 deletions docs/Scanner/docker/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,16 @@ We hereafter list the docker containers, their availability and provides link to
- [ ] **plantimager** is not available yet and explanations [there](plantimager_docker.md)
- [ ] **plantinterpreter** is not available yet and explanations [there](plantinterpreter_docker.md)
- [ ] **virtualplantscanner** is not available yet and explanations [there](virtualplantimager_docker.md)
- [x] **plantviewer** is available [here](https://hub.docker.com/repository/docker/roboticsmicrofarms/plantviewer) and explanations [there](plantviewer_docker.md)
- [x] **plantviewer** is available [here](https://hub.docker.com/repository/docker/roboticsmicrofarms/plantviewer) and explanations [there](plant3dexplorer_docker.md)


## Use cases with docker-compose
In this section we reference the "real-life" use cases of our software.

### Use the plantviewer on a local database directory
### Use the plant 3d explorer on a local database directory
To easiest way to use the plantviewer on a local database directory without installing the ROMI libraries (and their dependencies) is to use the pre-built docker image and add a docker-compose YAML recipe.
See [here](docker_compose.md#database--plantviewer) for more details.


## DockerHub

### roboticsmicrofarms
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Docker container for ROMI plantviewer
Docker container for ROMI `plant 3d explorer`
=====================================

The plant visualizer is a webapp that dialog with the database to display images & some quantitative traits.
Expand All @@ -24,11 +24,11 @@ To create a local ROMI database:

* python package install, look [here](../install/romidb_setup.md).
* `romidb` docker image, look [here](romidb_docker.md).
* docker-compose YAML recipe (start both a `romidb` & a `plantviewer` docker image connected to the db), look [here](docker_compose.md).
* docker-compose YAML recipe (start both a `romidb` & a `plant 3d explorer` docker image connected to the db), look [here](docker_compose.md).


## Use pre-built docker image
You can easily download and start the pre-built `plantviewer` docker image with:
You can easily download and start the pre-built `plant 3d explorer` docker image with:
```bash
docker run -p 3000:3000 roboticsmicrofarms/plantviewer:$VTAG
```
Expand All @@ -43,7 +43,7 @@ docker run --env REACT_APP_API_URL='0.0.0.0' -p 3000:3000 roboticsmicrofarms/pla
[^1]: https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file

## Build docker image
To build the image, from the `3d-plantviewer` root directory, run:
To build the image, from the `plant 3d explorer` root directory, run:
```bash
export VTAG="latest"
docker build -t roboticsmicrofarms/plantviewer:$VTAG .
Expand Down
2 changes: 1 addition & 1 deletion docs/Scanner/install/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ In the following subsections we will details how to install ROMI software for a
0. Plant scans acquisition using the ROMI plant scanner to a database [here](plant_imager_setup.md).
0. Plant reconstruction pipelines from existing plant scans in a database [here](plant_reconstruction_setup.md).
0. Virtual plant creation (3D modelling of plant architecture with LPY), virtual scan (mimic plant scanner with blender) & reconstruction (same as 2.) [here](virtual_plant_setup.md).
0. Create a web server hosting the plantviewer GUI [here](visualizer_setup.md).
1. Create a web server hosting the plant 3d explorer GUI [here](./plant3dexplorer_setup.md).

!!!note
You can find **docker images** for use cases #1, #3 & #5 in the dockerhub repository of the ROMI project [here](https://hub.docker.com/orgs/roboticsmicrofarms/repositories).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
Install ROMI software for virtual plants acquisition & reconstruction
Install the ROMI `Plant 3d explorer`
=====================================================================

To follow this guide you should have a `conda` environment, see [here](create_env.md).
For the sake of clarity it will be called `plant_visualizer`.
For the sake of clarity it will be called `Plant 3d explorer`.

!!! note
If you do not want the hassle of having to create environment & install python libraries, there is a pre-built docker image, with usage instructions [here](../docker/plantviewer_docker.md).
If you do not want the hassle of having to create environment & install python libraries, there is a pre-built docker image, with usage instructions [here](../docker/plant3dexplorer_docker.md).


## Pre-requisite

The plantviewer relies on:
The `Plant 3d explorer` relies on:

- `node`
- `npm`
Expand Down Expand Up @@ -40,17 +40,17 @@ npm install
```


## Use the plantviewer
## Use the `Plant 3d explorer`

### With the official ROMI database
You can use the ROMI database to test the installation of the plantviewer:
You can use the ROMI database to test the installation of the `Plant 3d explorer`:
```bash
export REACT_APP_API_URL='https://db.romi-project.eu'
npm start
```

### With a runnning local database
If you have followed the install instructions of the ROMI database ([here](romidb_setup.md)), you can use it with the viewer:
If you have followed the install instructions of the ROMI database ([here](romidb_setup.md)), you can use it with the `Plant 3d explorer`:
```bash
export REACT_APP_API_URL='0.0.0.0'
npm start
Expand Down
1 change: 0 additions & 1 deletion docs/Scanner/install/romidb_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ python3.7 -m pip install flask flask-restful flask-cors && \
python3.7 -m pip install .
```


## Initialize a ROMI database

The `FSDB` class from the `romidata` module is used to manage a local file system for data storage.
Expand Down
186 changes: 186 additions & 0 deletions docs/Scanner/tutorials/plant-3d-explorer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
How to see directly the results of your plant phenotyping with the `plant-3d-explorer` ?
================================

## Objective
Throughout the whole process of plant phenotyping, viewing data is often needed.
This tutorial explains how to use the romi `plant-3d-explorer`, a web-server tool, to explore, display and interact with most of the diverse data generated during a typical plant phenotyping experiment from 2D images (2D images, 3D objects like meshes or point cloud, quality evaluations, trait measurements).
After this tutorial, you should be able to:
* connect the plant-3d-explorer to a database containing the phenotyping data of one to several plants ;
* explore the database content with the `plant-3d-explorer` menu page ;
* For each plant, display, overlay and inspect in 3d every data generated during analysis

## Prerequisite
* install romi `plant-3d-explorer` (from [source](https://github.com/romi/plant-3d-explorer) or using a [docker image](../docker/plant3dexplorer_docker.md) ) & read [install procedure](../install/plant3dexplorer_setup.md)
* install romi `plantdb` (from [source](https://github.com/romi/plantdb) or using a [docker image](../docker/romidb_docker.md)) & read [install procedure](../install/romidb_setup.md)
* install romi `plant-3d-vision` (from [source](https://github.com/romi/plant-3d-vision) or using a [docker image](../docker/plantinterpreter_docker.md)) & read [install procedure](../install/plant_reconstruction_setup.md)
* Create and activate isolated python environment (see the procedure [here](/docs/Scanner/install/create_env.md) )

!!! note
using **docker**
<img src="/assets/images/ext/docker_logo2.png" alt="docker_logo" width="50">

You can avoid installs by using **docker** only. Read first the docker procedures ( ['docker for plant-3d-vision'](../docker/plantinterpreter_docker.md) and
['docker-compose to run both database and 3d explorer with docker containers'](../docker/docker_compose.md) ). In the following tutorial (steps 1, 2 and 3), follow the docker logo to adapt the procedure.

## Linked documentation
* [Manual of the romi plant-3d-explorer](/docs/plant_3d_explorer/index.md)

## Step-by-step tutorial
Principle: the `plant-3d-explorer` is a web client that displays in your favorite web browser data exposed by a server (here, romi `plantdb`) on a particular url. The process consists in pointing the server to your folder of interest, starting the server and starting the client that points to the served url.

!!!note
the `plant-3d-explorer` has only been developed and tested on Chrome.


### 1. Preparing your database for display by the `plant-3d-explorer`
**Starting point:** your *database* is made of one or several *datasets*, which all correspond to a single plant phenotyping experiment: each dataset contains at least 2D images (raw acquisitions) and metadata, and possibly several other data generated by subsequent 3D reconstruction, segmentation and analysis.

!!! Note

Your database must follow the rules of romi databases: please make sure that you comply to [requirements](../install/romidb_setup/#initialize-a-romi-database).

You can also download an example database [here](https://db.romi-project.eu/models/test_db.tar.gz).

*example*: let's consider a database called `my_experiment` containing 3 datasets (named plant1, plant2, plant3) generated by phenotyping three plants.

* open a terminal and go to your local database directory
* if romi commands (like `romi_run_task`) are not accessible from your terminal, activate the appropriate python environment (e.g. using venv or conda) required for romi commands (or [read this procedure](../install/create_env.md))
* Process all datasets for display by the `plant-3d-explorer` by running the following code

```bash
dataset_list=('plant1','plant2','plant3')

for ds in "${dataset_list[@]}"
do
romi_run_task Visualization path_to/my_experiment/"$ds"/ --config ~/config/ml_pipe_real.toml
done
```
!!! Note
For more information about using romi_run_task command, the Visualization task and the config file, please read XXXXX.

!!! Note
<img src="/assets/images/ext/docker_logo2.png" alt="docker_logo" width="50"> *for docker users*

- Start a docker container by mounting your database as a volume ([details](../docker/plantinterpreter_docker.md/#manually))

- In the container, run the same Visualization Task has above

**check result**: a new folder called `Visualization` should have been created in each dataset of your database

!!! Note
<img src="/assets/images/ext/docker_logo2.png" alt="docker_logo" width="50"> *for docker users*

Skip step 2 & 3 and follow instead instructions given by['docker-compose to run both database and 3d explorer with docker containers'](../docker/docker_compose.md)

### 2. Connect your database to a local server
- Continue in the same shell terminal (if you open a new terminal, do not forget to activate appropriate python environment)
- set the DB location using the `DB_LOCATION` environment variable
- Type the following commands to launch the server:
```bash
export DB_LOCATION=/path/to/your/db
romi_scanner_rest_api #command that starts the server
```
**check result**: the terminal prints various information given by the server (e.g. number of datasets in the database). Do not stop this terminal as this will shut down the server

### 3. Connect the `plant-3d-explorer` to the server
- Open a new terminal
- go to your local cloned directory of `plant-3d-explorer/`
- start the frontend visualization server by entering:
```bash
npm start
```
You should now be able to access the `plant-3d-explorer` on [http://localhost:3000](http://localhost:3000). Depending on you system preferences, your default web browser may automatically open a window displaying the server content. if not, open your web browser and enter [http://localhost:3000](http://localhost:3000) in the url bar.

!!! note
You need to add a file `.env.local` at project's root to set the API URL: ```REACT_APP_API_URL='{`API URL}'```.
Without this, the app will use: http://localhost:5000 which is the default for `romi_scanner_rest_api`.

### 4. Explore your database content via the menu page

!!! note
More description about the function of the menu page: read [here](/docs/plant-3d-explorer/index.md)

The starting page of the `plant-3d-explorer` lists the datasets of the connected database as a table and looks like this:

<kbd>
<img src="/assets/images/plant-3d-explorer/plant3dexplorer_menupage.png" alt="plant3dexplorer_menupage" width="1000">
</kbd>

* The top search bar allows you to find particular datasets based on keywords.
* Data filters: in the header row, click on an icon to activate the filter (datasets that do *not* contain the data will be filtered out)

* <img src="/assets/images/plant-3d-explorer/plant3dexplorer_availabledata.png" alt="availdata" width="100"> 3d objects generated from the plant 2D images (icons respectively stand for: mesh, point cloud, segmented point cloud, skeleton and organs)

* <img src="/assets/images/plant-3d-explorer/plant3dexplorer_phyllotaxisfilter.png" alt="phyllofilter" width="100"> or phyllotaxis data (manual or computed phyllotaxis measurements)

* Open a dataset with the green 'Open' button at the far right of a row <img src="/assets/images/plant-3d-explorer/plant3dexplorer_openbutton.png" alt="openbutton" width="100">


### 5. View a single dataset and all related data

!!! note
This is only a brief description to allow a quick start. More description [here](/docs/plant-3d-explorer/index.md)

By default, the `plant-3d-explorer` displays in the main panel the skeleton and the organs (if available) and phyllotaxis data (as graphs) in the right panel.

<kbd>
<img src="/assets/images/plant-3d-explorer/plant3dexplorer_dataset_initpage.png" alt="plant3dexplorer_dataset_initpage" width="1000">
</kbd>

Mouse-over most elements provides a brief description.

#### Select the 3D layers to display
In the top left corner of the main panel, icons allows you to quickly (un)select 3D layers (if available):

<img src="/assets/images/plant-3d-explorer/plant3dexplorer_3dlayers_icons.png" alt="plant3dexplorer_dataset_initpage" width="250" style="display:block; margin-left: auto; margin-right: auto;">

* White icons are active, dark grey are available but not active, light grey are not available for this dataset
* From left to right, icons represents respectively the mesh, the pointcloud, the segmented point cloud, the skeleton and the organs.


In the center of the middle panel are icons for general viewing options:
<img src="/assets/images/plant-3d-explorer/plant3dexplorer_view-icons.png" alt="plant3dexplorer_view-icons.png" width="100" style="display:block; margin-left: auto; margin-right: auto;">

* Activate the camera icon displays the camera poses (only works if overlay with 2 images is deactivated)
* Click the round arrow to reset the view

#### Moving the view in the "free" 3D (without 2D overlay)
Easy movements are accessible with a mouse:

* scroll to zoom in/out
* left click rotate
* right click translate

#### Activate overlay with 2D images
click on any image of the bottom carousel to activate the display of 2 images in the main panel. On Mouse-over, a single picture is enlarged and proposes to open it in the main panel.

<img src="/assets/images/plant-3d-explorer/plant3dexplorer_carousel-open.png" alt="plant3dexplorer_view-icons.png" width="800">

Overlay with active 3D layers is automatic. In the carousel, the box around the active displayed 2D image is now permanent.

To close the 2D overlay, just click the close button of the boxed picture in the carousel.

<img src="/assets/images/plant-3d-explorer/plant3dexplorer_carousel-close.png" alt="plant3dexplorer_view-icons.png" width="800">

#### Moving the view with 2D overlay
Note that movement control with the mouse slightly changes compared to the "free" view without 2D overlay. Notably, the free rotation mode is not possible anymore, since it is constrained by the real movements made by the camera when it took the pictures.

* Slide right/left the active box picture in the carousel to reproduce the camera movement
* scroll to zoom in/out
* left-click to translate

#### The phyllotaxis measure plots
Plots represent the successive measures of divergence angles (left, in degrees) and internode length (right, in mm) between consecutive pairs of organs (here fruits) along the stem, from the base to the inflorescence tip. Both plots can be closed by clicking the cross at the far right the plot's title. Closed plot panels can be re-opened by clicking a green "+" sign appearing at the to right-hand corner when at least one plot is closed.

Mouse-over any of the two plots highlights an interval that correspond to a measure between two consecutive organs segmented by the analysis. The interval appears synchronously on both plots if opened. This interval and the organs are numbered by their order from the base of the stem, these numbers appear on the X-axis of the plots.

When the 'organ' 3d-layer is activated in the main panel,
automatically select the organ pair in the main panel that correspond to the current interval pointed by the mouse in the graph (the 3d layer of other organs disappear). The two graphs (divergence angles and internodes) are synchronized.

clicking an interval

#### Go back to main page
In the top left corner of the page, click "all scans"

<img src="/assets/images/plant-3d-explorer/plant3dexplorer_allscans_backmenu.png" alt="backtomenu" width="100">

51 changes: 0 additions & 51 deletions docs/Scanner/tutorials/visualizer.md

This file was deleted.

Binary file added docs/assets/images/ext/docker_logo2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
## The scan list page

When first opening the app, you will be greeted with a home
page displaying every available scans.
A scan is defined as a folder containing acquisition of plant data (e.g. 2D RGB images, manual measures) and a set of 3D reconstructions and analysis (e.g. point cloud, mesh, automated measures,...)
Expand Down
File renamed without changes.
Loading