Create instrument swath outlines as shapefiles #93

adybbroe · 2024-11-19T15:14:44Z

This PR adds a script to create Satellite instrument swath outlines in shapefile format, e.g. for use in an WMS/WFS service.

Also some re-factoring has been done in this PR.

Closes Should be possible to store swath outlines as shapefiles #90
Tests added
Tests passed
Passes git diff origin/main **/*py | flake8 --diff
Fully documented

…shapefile support Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

…ine-shapefile # Conflicts: # trollsched/tests/test_satpass.py

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

adybbroe · 2024-11-20T14:26:03Z

Here an example of how the script can be run:
create_shapefiles_from_schedule -s NOAA-21 NOAA-20 -t ./tle-202409060015.txt -x ./2024-09-06-00-45-08-acquisition-schedule-confirmation-nrk.xml -o /data/pass_outlines

TAlonglong · 2024-11-20T14:36:32Z

Could you add something like that example you gave to the docs?

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

codecov · 2024-11-20T19:49:31Z

Codecov Report

Attention: Patch coverage is 81.38528% with 43 lines in your changes missing coverage. Please review.

Project coverage is 60.35%. Comparing base (fccd0a5) to head (6d8344f).
Report is 53 commits behind head on main.

Files with missing lines	Patch %	Lines
trollsched/combine.py	11.11%	16 Missing ⚠️
trollsched/drawing.py	0.00%	13 Missing ⚠️
trollsched/shapefiles_from_schedule.py	89.39%	7 Missing ⚠️
trollsched/logger.py	77.77%	4 Missing ⚠️
trollsched/utils.py	81.81%	2 Missing ⚠️
trollsched/helper_functions.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #93      +/-   ##
==========================================
+ Coverage   55.71%   60.35%   +4.64%     
==========================================
  Files          14       18       +4     
  Lines        2107     2278     +171     
==========================================
+ Hits         1174     1375     +201     
+ Misses        933      903      -30

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

adybbroe · 2024-11-22T12:51:56Z

Could you add something like that example you gave to the docs?

Good point, absolutely. I have added a bare minimum at least.

adybbroe · 2024-11-22T13:02:14Z

Here just showing an example of a QGIS display of one shapefile produced by the script above:

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

…schedule into swathoutline-shapefile

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

trollsched/pass_scheduling_utils.py

trollsched/shapefiles_from_schedule.py

Co-authored-by: Panu Lahtinen <pnuu+git@iki.fi>

adybbroe · 2024-12-03T10:20:25Z

I think I should have actually removed the main block in shapefiles_from_schedule.py do you agree @pnuu ?

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

mraspaud

Wow, thanks a lot for the huge work in cleaning this package up! Looks good in general, but I have a few comments and suggestions/requests inline

.github/workflows/ci.yaml

.stickler.yml

docs/usage.rst

.bumpversion.cfg

pyproject.toml

trollsched/graph.py

trollsched/logger.py

trollsched/schedule.py

trollsched/shapefiles_from_schedule.py

trollsched/tests/test_pass_outlines.py

…errors Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Co-authored-by: Martin Raspaud <martin.raspaud@smhi.se>

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

…schedule into swathoutline-shapefile

…nd line args Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

docs/environment.yaml

adybbroe · 2025-01-06T12:20:20Z

I think the issues you raised have been attended to @mraspaud
But it is some time ago now, what do you say, can we accept it as is?
Any other comments from @TAlonglong or others?

trollsched/__init__.py

TAlonglong · 2025-01-07T15:48:47Z

trollsched/__init__.py

+                              "FENGYUN 3A": "FENGYUN-3A",
+                              "FENGYUN 3B": "FENGYUN-3B",
+                              "FENGYUN 3C": "FENGYUN-3C",
+                              "SUOMI NPP": "NPP"}


Do we need NOAA-20 and NOAA-21 or are they not needed to translate?

I am not sure. Maybe not, but you would know better, as you added this @TAlonglong . I suppose we will find out, and then fix it accordingly when/if needed in another PR.

TAlonglong

Wow, this was a lot of small fixes and updates.

Nice functionallity, I could also think of output WKT POLYGON string to be used wtih eg a postgis But This is not for this PR!

trollsched/graph.py

mraspaud

LGTM!

Co-authored-by: Martin Raspaud <martin.raspaud@smhi.se>

Adam.Dybbroe added 8 commits October 3, 2024 17:16

Refactor and use pyproject.toml for intallation - first draft add of …

34ac1d2

…shapefile support Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Version file is now generated upon installation

3609a71

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Merge branch 'main' into swathoutline-shapefile

23c0888

Merge branch 'bugfix-slstr-not-yet-properly-supported' into swathoutl…

03af390

…ine-shapefile # Conflicts: # trollsched/tests/test_satpass.py

Merge branch 'main' into swathoutline-shapefile

692eb25

Add a first script to create the shapefiles from the XML schedule file

5d14d72

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Add a basic logger module

29e7576

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Bugfix

9682861

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

adybbroe added the enhancement label Nov 19, 2024

adybbroe self-assigned this Nov 19, 2024

Adam.Dybbroe added 5 commits November 20, 2024 13:38

Attempt to fix the ruff/linter stuf in the pyproject toml file

d1533c2

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Try fix pyproject.toml

a78d7f4

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Trt fix the CI and github actions

8ed05a8

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Bugfix - we do not try to run CI tests with Basemap support

c364f6e

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Fix so the Code-factor bot is happy

e675cd2

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

adybbroe requested review from mraspaud, pnuu and TAlonglong November 20, 2024 14:24

Adam.Dybbroe added 3 commits November 20, 2024 16:25

Update pre-commit config -copying from Satpy

e2268f8

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Test add back rye

7a37ec3

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Fix cut'n'paste error

8150fdd

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Adam.Dybbroe added 2 commits November 22, 2024 13:46

Add a minimum amount of documentation for the new script

972f4a2

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Bugfix

8e82f37

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Adam.Dybbroe added 2 commits November 22, 2024 16:10

Add tests and fix style issues

472c1e0

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Fix codefactor complaints

354ff71

Adam.Dybbroe added 4 commits December 2, 2024 20:48

Fix more meaningful doc strings

22f689e

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Merge branch 'swathoutline-shapefile' of github.com:adybbroe/pytroll-…

ba59c38

…schedule into swathoutline-shapefile

Add more test coverage

9df160b

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Fix style issues

ba5f81e

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

adybbroe requested a review from pnuu December 3, 2024 08:40

pnuu reviewed Dec 3, 2024

View reviewed changes

trollsched/pass_scheduling_utils.py Outdated Show resolved Hide resolved

trollsched/shapefiles_from_schedule.py Outdated Show resolved Hide resolved

Update trollsched/pass_scheduling_utils.py

e545647

Co-authored-by: Panu Lahtinen <pnuu+git@iki.fi>

Remove __main__ from module. It uses the run part for the script

c5d3e40

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

mraspaud requested changes Dec 3, 2024

View reviewed changes

Adam.Dybbroe and others added 9 commits December 4, 2024 10:44

Clean up, removing unneccessary/unused stuff, and fixing cut'n'paste …

26c8f3b

…errors Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Update trollsched/schedule.py

754c607

Co-authored-by: Martin Raspaud <martin.raspaud@smhi.se>

Update trollsched/graph.py

bdc004c

Co-authored-by: Martin Raspaud <martin.raspaud@smhi.se>

Add more debug logging

38330e2

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Restructure imports

2b4ef25

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Merge branch 'swathoutline-shapefile' of github.com:adybbroe/pytroll-…

c949c58

…schedule into swathoutline-shapefile

Fix test to include the testing of running the main script with comma…

5243dbe

…nd line args Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Re-introduce some of the debug printouts using the logging module

a5a2e77

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

Update the example config with current satellites

086d579

Signed-off-by: Adam.Dybbroe <a000680@c22526.ad.smhi.se>

adybbroe requested review from pnuu and mraspaud December 4, 2024 21:50

mraspaud reviewed Dec 5, 2024

View reviewed changes

docs/environment.yaml Show resolved Hide resolved

TAlonglong reviewed Jan 7, 2025

View reviewed changes

trollsched/__init__.py Show resolved Hide resolved

TAlonglong reviewed Jan 7, 2025

View reviewed changes

TAlonglong approved these changes Jan 7, 2025

View reviewed changes

mraspaud reviewed Jan 8, 2025

View reviewed changes

trollsched/graph.py Outdated Show resolved Hide resolved

mraspaud approved these changes Jan 8, 2025

View reviewed changes

Use sensible file name for the schedule graph in pdf format

6d8344f

Co-authored-by: Martin Raspaud <martin.raspaud@smhi.se>

adybbroe merged commit 83a32e2 into pytroll:main Jan 8, 2025
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create instrument swath outlines as shapefiles #93

Create instrument swath outlines as shapefiles #93

adybbroe commented Nov 19, 2024 •

edited

Loading

adybbroe commented Nov 20, 2024

TAlonglong commented Nov 20, 2024

codecov bot commented Nov 20, 2024 •

edited

Loading

adybbroe commented Nov 22, 2024

adybbroe commented Nov 22, 2024

adybbroe commented Dec 3, 2024

mraspaud left a comment

adybbroe commented Jan 6, 2025

TAlonglong Jan 7, 2025

adybbroe Jan 8, 2025

TAlonglong left a comment

mraspaud left a comment

Create instrument swath outlines as shapefiles #93

Create instrument swath outlines as shapefiles #93

Conversation

adybbroe commented Nov 19, 2024 • edited Loading

adybbroe commented Nov 20, 2024

TAlonglong commented Nov 20, 2024

codecov bot commented Nov 20, 2024 • edited Loading

Codecov Report

adybbroe commented Nov 22, 2024

adybbroe commented Nov 22, 2024

adybbroe commented Dec 3, 2024

mraspaud left a comment

Choose a reason for hiding this comment

adybbroe commented Jan 6, 2025

TAlonglong Jan 7, 2025

Choose a reason for hiding this comment

adybbroe Jan 8, 2025

Choose a reason for hiding this comment

TAlonglong left a comment

Choose a reason for hiding this comment

mraspaud left a comment

Choose a reason for hiding this comment

adybbroe commented Nov 19, 2024 •

edited

Loading

codecov bot commented Nov 20, 2024 •

edited

Loading