PPCU IFOM YeastTracker

The aim of this library is to provide ML-based cell tracking solutions for yeast cells, using a novel time-symmetric tracking approach. It is based on a collaborative research project between PPCU and IFOM.

The codebase in this repository is frozen at the time of publication for transparency. For the dynamically updated code versions, please visit: https://github.com/SzaboGergely0419/Symmetry-Tracker

Paper publication is still in progress, for the preprint version please check out: https://arxiv.org/abs/2308.03887

While publication is in progress, please cite our work as:

@article{szabo2023enhancing,
  title={Enhancing Cell Tracking with a Time-Symmetric Deep Learning Approach},
  author={Szab{\'o}, Gergely and Bonaiuti, Paolo and Ciliberto, Andrea and Horv{\'a}th, Andr{\'a}s},
  journal={arXiv preprint arXiv:2308.03887},
  year={2023}
}

Installation Requirements

pip install numpy
pip install opencv-python
pip install matplotlib
pip install scikit-image
pip install scipy
pip install torch
pip install "git+https://github.com/facebookresearch/detectron2.git"
pip install segmentation-models-pytorch
pip install mrc==0.1.5

Resources

Models available at: https://users.itk.ppke.hu/~szage11/IFOM%20tracking/TrainedModels/

Sample data available at: https://users.itk.ppke.hu/~szage11/IFOM%20tracking/SampleData/

Getting Started

Example full run:

Example Prediction Videos

For example prediction videos of yeast tracking and various synthetic object tracking scenarios, check the sample_results folder.

The videos will be not displayed on github, they need to be downloaded.

Main functionalities

All fuctionalities which are intended to be directly used are listed here. The other functions and classes in the library have supportive roles and are only intended to be utilized by the functions listed here.

Segmentation

SingleVideoSegmentation

• At: ppcu_ifom_yeasttracker/segmentation/segmentator.py
• Performs segmentation on fixed shape (512,512) video input

DisplaySegmentation

• At: ppcu_ifom_yeasttracker/segmentation/segmentation_io.py
• Displays segmentation results generated by SingleVideoSegmentation() as images

WriteSegmentation

• At: ppcu_ifom_yeasttracker/segmentation/segmentation_io.py
• Saves segmentation results in standard txt format, which is later usable for tracking

Tracking

SingleVideoCellTracking

• At: ppcu_ifom_yeasttracker/tracking/tracker.py
• Performs cell tracking on a video which was already segmented using both the video and the segmentation (annotation)

InterpolateMissingCellPoints

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Interpolates the segmentations which should be part of contiuous tracks but were missed. This is not based on heuristics and will be part of the publication. Interpolated cell instances are displayed with different colors at DisplayCellPaths_IDs() and SaveCellPathsVideo().

PrintAllCellPaths

• At: ppcu_ifom_yeasttracker/tracking/tracking_io.py
• Displays the tracking results as ID chains

DisplayCellPaths_IDs

• At: ppcu_ifom_yeasttracker/tracking/tracking_io.py
• Displays the tracking results generated by SingleVideoCellTracking() as images

SaveOptimalAnnotation

• At: ppcu_ifom_yeasttracker/tracking/tracking_io.py
• Saves tracking results to an LZMA compressed pickle file readable by LoadOptimalAnnotation()

LoadOptimalAnnotation

• At: ppcu_ifom_yeasttracker/tracking/tracking_io.py
• Loads tracking results from an LZMA compressed pickle file generated by SaveOptimalAnnotation()

WriteOptimalAnnotation

• At: ppcu_ifom_yeasttracker/tracking/tracking_io.py
• Saves tracking results in standard txt format

SaveCellPathsVideo

• At: ppcu_ifom_yeasttracker/tracking/tracking_io.py
• Saves tracking results as an .mp4 video

Heuristical post-processing

These functionalities can improve / change tracking results significantly, but are based on additional heuristics and manual setup. They will not be included in the publication, but with proper setup, they may help improving the tracking results.

HeuristicalEquivalence

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Connects up broken up cell tracks which have a significant IOU at the breaking point with a given maximal time difference

ManualEquivalence

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Connects up broken up cell tracks based on a manually defined equivalence

RemoveShortPaths

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Removes the too short paths from a tracking

RemoveBorderPaths

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Removes the paths from a tracking which got too close to the edge of the recording

RemoveLargeNewbornCells

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Removes the paths from a tracking which start too large to be realistic for newborn cells

RemoveUnfinishedTracks

• At: ppcu_ifom_yeasttracker/tracking/post_processing.py
• Removes all the tracks which do not appear at the last frame of the tracking