Project | Arxiv | Dataset | Dataset_2
Keras implementation for learning a mapping from SEM gray images to colorful images, for example:
Nanoscale Microscopy Images Colourization Using Neural Networks
Note: Please check out our Tensorflow implementation for End2End ColorNet(End2End.ipynb) and CNN-NST(cnn_nst.ipynb).
- Linux or OSX
- NVIDIA GPU + CUDA CuDNN (CPU mode and CUDA without CuDNN may work with minimal modification, but untested)
- Keras with tensorflow backend
- Install keras with tensorflow backend and dependencies from https://keras.io/#installation
- Install python packages
jupyter-notebook
,scikit-image
andopencv
pip install scikit-image
pip install jupyter
pip install opencv-python
-
Install livelossplot(optional) - a live monitor during training.
-
Clone this repo:
git clone https://github.com/isrugeek/semcolour
cd semcolour
check the code in End2End.ipynb
.
Setting the right gpu configure. ex:
os.environ["CUDA_VISIBLE_DEVICES"]="0,1,2"
Variable iteration
means epochs, change these settings for your own conditions.
# global varibles
trainset_path = "datasets/trainset"
devset_path = "datasets/devset"
testset_path = "datasets/testset"
batch_size = 16
# epochs
iteration = 100
target_size = (256, 256)
Run the last cell in the End2End.ipynb
file to test the datasets which in devset_path
and save these to results
directory.
flow_from_directory_regress.py
is rewrite the class ImageDataGenerator
, adding register_batch_process
method to make
our preprocess process more convenient, make sure u have this py file in your work directory.
We collected 800 colorful sem-images as SEMCOLORFUL1.0
in the datasets
directory split into devset
and trainset
.
The pre-trained models pretrained_end2end_model.h5
can be loaded by running follow line.
model.load_weights('pretrained_end2end_model.h5')
after load the pretrained weights, you can test the model directly instead of training.
Check the code in cnn_nst.ipynb
.
reference_img
indicates the reference images, dir_sem
is the directory which contains SEM gray images, sem_input
indicates specific the input gray sem image.
It doesn't like traditional neural style transfer process, It just learn how to map reference image from gray to color by CNN, then using the weights of the CNN to map the SEM input image to colorful image.
same_grey_same_colorize
function can provide a constraint to put same color on the parts which are same gray level,it can correct some CNN's mistakes.
If you use this code for your research, please cite our paper Nanoscale Microscopy Images Colourization Using Neural Networks :
@article{goytom2019nanoscale,
title={Nanoscale Microscopy Images Colourisation Using Neural Networks},
author={Goytom, Israel and Wang, Qin and Yu, Tianxiang and Sankaran, Kris and Lin, Dongdong},
journal={arXiv preprint arXiv:1912.07964},
year={2019}
}