Fine-Tuning SAM (Segment anything) for River Water Segmentation

Overview

This repository presents the Python code for fine-tuning the Segment Anything Model (SAM) to perform river water segmentation from close-range remote sensing imagery. This work is based on our paper published in IEEE Access:

A. Moghimi, M. Welzel, T. Celik, and T. Schlurmann, "A Comparative Performance Analysis of Popular Deep Learning Models and Segment Anything Model (SAM) for River Water Segmentation in Close-Range Remote Sensing Imagery," IEEE Access, 2024. IEEE Access

The easy-to-use and adaptable code for river water and other segmentation tasks and use for other remote sensing datasets:

Try it in Colab:

The LuFI-RiverSNAP.v1 (river water segmentation) Dataset in Google Drive:

Some examples of river water segmentation results on the LuFI-RiverSnap.v1. (a) Images and segmentation results generated by (b) U-Net_(ResNet50), (c) PSPNet_(ResNet50), (d) DeeplabV3+_(ResNet50), (e) PAN_(ResNet50), (f) LinkNet_(ResNet50), and (g) SAM were used as DL models for river water segmentation. Green: False Positives (FP) detection, Pink: False Negatives (FN) detection, Blue: correct detection of river water.

Try it in Colab:

Please also follow and read the reference codes we created for our fine-tuning SAM based on.

Original SAM Code: GitHub - Segment Anything

Fine-Tuning Tutorial: Encord Blog - Fine-Tune SAM

{Some examples of river water segmentation results on the LuFI-RiverSnap.\textit{v}1. (a) Images and segmentation results generated by (b) MobileSAM (TinyViT), (c) SAM (ViT-B), (d) and SAM (ViT-L)}

Dataset

The LuFI-RiverSNAP.v1 dataset for river water segmentation is available on multiple platforms:

• Kaggle: LuFI-RiverSNAP
• ISPRS ICWG III/IVa "Disaster Management" Datasets
• IEEE DataPort: LuFI-RiverSNAP

Try it in Colab:

Cite

Please cite the following papers if they help your research. You can use the following BibTeX entry:

@article{moghimi2024comparative,
  title={A Comparative Performance Analysis of Popular Deep Learning Models and Segment Anything Model (SAM) for River Water Segmentation in Close-Range Remote Sensing Imagery},
  author={Moghimi, Armin and Welzel, Mario and Celik, Turgay and Schlurmann, Torsten},
  journal={IEEE Access},
  year={2024},
  doi={https://doi.org/10.48550/arXiv.2304.02643},
  publisher={IEEE}
}

A. Moghimi, M. Welzel, T. Celik, and T. Schlurmann, "A Comparative Performance Analysis of Popular Deep Learning Models and Segment Anything Model (SAM) for River Water Segmentation in Close-Range Remote Sensing Imagery," in IEEE Access, doi: 10.1109/ACCESS.2024.3385425. https://ieeexplore.ieee.org/document/10493013

The original paper of SAM.

@inproceedings{kirillov2023segment,
  title={Segment anything},
  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C and Lo, Wan-Yen and others},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={4015--4026},
  doi={https://doi.org/10.48550/arXiv.2304.02643},
  year={2023}
}

Please cite the original SAM paper if you use our code and paper anyway, as our idea came from the original paper of SAM which changed the segmentation process and we appreciate it as helping us a lot.

Contact

For any queries or contributions, feel free to contact us.