Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ecmwf/covjsonkit
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 0.0.24
Choose a base ref
...
head repository: ecmwf/covjsonkit
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref

Commits on Oct 23, 2024

  1. Copy the full SHA
    5438061 View commit details
  2. Merge pull request #47 from ecmwf/feature/remove_unused_requirements

    Remove polytope-python from requirements
    awarde96 authored Oct 23, 2024
    Copy the full SHA
    c024fda View commit details
  3. Merge pull request #48 from ecmwf/develop

    Develop
    awarde96 authored Oct 23, 2024
    Copy the full SHA
    cf3e669 View commit details
  4. Bump version

    awarde96 committed Oct 23, 2024
    Copy the full SHA
    6f0e7be View commit details

Commits on Oct 29, 2024

  1. Fix path encoder

    awarde96 committed Oct 29, 2024
    Copy the full SHA
    8a32320 View commit details
  2. Fix formatting

    awarde96 committed Oct 29, 2024
    Copy the full SHA
    5d5465f View commit details
  3. Fix path for 2d

    awarde96 committed Oct 29, 2024
    Copy the full SHA
    a29d489 View commit details

Commits on Oct 30, 2024

  1. Merge pull request #49 from ecmwf/feature/fix_trajectory

    Fix path encoder
    awarde96 authored Oct 30, 2024
    Copy the full SHA
    8cb216e View commit details
  2. Bump version

    awarde96 committed Oct 30, 2024
    Copy the full SHA
    b513b19 View commit details
  3. Copy the full SHA
    901d9a9 View commit details
  4. Merge pull request #50 from ecmwf/develop

    Develop
    awarde96 authored Oct 30, 2024
    Copy the full SHA
    1c55ba6 View commit details

Commits on Nov 4, 2024

  1. Copy the full SHA
    bbdadd1 View commit details
  2. Remove numpy as requirement

    awarde96 committed Nov 4, 2024
    Copy the full SHA
    1ea0507 View commit details

Commits on Nov 5, 2024

  1. Update version

    awarde96 committed Nov 5, 2024
    Copy the full SHA
    178abb0 View commit details
  2. Merge pull request #51 from ecmwf/clean_up

    Clean up
    awarde96 authored Nov 5, 2024
    Copy the full SHA
    a4e6cfa View commit details
  3. Merge pull request #52 from ecmwf/develop

    Develop
    awarde96 authored Nov 5, 2024
    Copy the full SHA
    c17e5a6 View commit details

Commits on Nov 12, 2024

  1. Copy the full SHA
    29ad50f View commit details
  2. Copy the full SHA
    f002501 View commit details
  3. Copy the full SHA
    9a75c06 View commit details

Commits on Nov 13, 2024

  1. Copy the full SHA
    76a639d View commit details
  2. Remove unneeded comments

    awarde96 committed Nov 13, 2024
    Copy the full SHA
    c31bb41 View commit details
  3. Bump version

    awarde96 committed Nov 13, 2024
    Copy the full SHA
    17c556b View commit details
  4. Merge pull request #53 from ecmwf/feature/update_range_path_vp

    Feature/update range path vp
    awarde96 authored Nov 13, 2024
    Copy the full SHA
    a82622c View commit details
  5. Merge pull request #54 from ecmwf/develop

    Develop
    awarde96 authored Nov 13, 2024
    Copy the full SHA
    cd6e0ee View commit details

Commits on Nov 21, 2024

  1. Copy the full SHA
    4602daf View commit details
  2. Fix 4d path

    awarde96 committed Nov 21, 2024
    Copy the full SHA
    938081f View commit details
  3. Bump version

    awarde96 committed Nov 21, 2024
    Copy the full SHA
    dd57fcd View commit details

Commits on Nov 25, 2024

  1. Copy the full SHA
    46270c2 View commit details

Commits on Dec 12, 2024

  1. Merge pull request #55 from ecmwf/bug/fix_traj_coords

    Path now allows inflation on levelist
    awarde96 authored Dec 12, 2024
    Copy the full SHA
    b6a2d6f View commit details

Commits on Dec 18, 2024

  1. Copy the full SHA
    a94279b View commit details

Commits on Dec 19, 2024

  1. Merge pull request #56 from juntyr/patch-1

    Relax pandas version pin to support 2.2.x
    awarde96 authored Dec 19, 2024
    Copy the full SHA
    5130a0c View commit details
  2. Bump version

    awarde96 committed Dec 19, 2024
    Copy the full SHA
    5cb9d1b View commit details

Commits on Jan 7, 2025

  1. Update requiremnts

    awarde96 committed Jan 7, 2025
    Copy the full SHA
    ef8754e View commit details

Commits on Jan 15, 2025

  1. Copy the full SHA
    e7afe22 View commit details
  2. Merge pull request #57 from ecmwf/bug/fix_traj_coords

    Bug/fix traj coords
    awarde96 authored Jan 15, 2025
    Copy the full SHA
    73af20c View commit details
  3. Copy the full SHA
    6b7d79a View commit details

Commits on Jan 16, 2025

  1. Copy the full SHA
    2713e94 View commit details
  2. Black

    awarde96 committed Jan 16, 2025
    Copy the full SHA
    8b4c017 View commit details
  3. Bump version

    awarde96 committed Jan 16, 2025
    Copy the full SHA
    4fe1ddf View commit details
  4. Merge pull request #58 from ecmwf/feature/climatedt

    Feature/climatedt
    awarde96 authored Jan 16, 2025
    Copy the full SHA
    e97e13d View commit details
  5. Fix fro climate-dt

    awarde96 committed Jan 16, 2025
    Copy the full SHA
    b40360d View commit details
  6. Merge pull request #59 from ecmwf/develop

    Develop
    awarde96 authored Jan 16, 2025
    Copy the full SHA
    7b2dd87 View commit details
6 changes: 3 additions & 3 deletions covjsonkit/decoder/BoundingBox.py
Original file line number Diff line number Diff line change
@@ -84,9 +84,9 @@ def to_xarray(self):
coverage["mars:metadata"]["step"]
] = coverage["ranges"][parameter]["values"]

datetimes = list(set(datetimes))
numbers = list(set(numbers))
steps = list(set(steps))
datetimes = sorted(list(set(datetimes)))
numbers = sorted(list(set(numbers)))
steps = sorted(list(set(steps)))

new_values = {}
for parameter in values.keys():
6 changes: 3 additions & 3 deletions covjsonkit/decoder/Path.py
Original file line number Diff line number Diff line change
@@ -42,10 +42,12 @@ def to_xarray(self):
# Get coordinates
x = []
y = []
z = []
t = []
for coord in self.get_coordinates()["composite"]["values"]:
x.append(float(coord[1]))
y.append(float(coord[2]))
z.append(float(coord[3]))
t.append(coord[0])

values = {}
@@ -108,13 +110,11 @@ def to_xarray(self):
points=(["points"], list(range(0, len(x)))),
x=(["points"], x),
y=(["points"], y),
z=(["points"], z),
t=(["points"], t),
),
)
for mars_metadata in self.mars_metadata[0]:
ds.attrs[mars_metadata] = self.mars_metadata[0][mars_metadata]

# Add date attribute
# ds.attrs["date"] = self.get_coordinates()["t"]["values"][0]

return ds
97 changes: 69 additions & 28 deletions covjsonkit/decoder/VerticalProfile.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import pandas as pd
import xarray as xr

from .decoder import Decoder
@@ -53,39 +54,79 @@ def to_geopandas(self):
pass

def to_xarray(self):
dims = ["x", "y", "t", "number", "z"]
dims = [
"x",
"y",
"number",
"datetime",
"t",
"z",
]
dataarraydict = {}

# Get coordinates
coords = self.get_domains()
x = coords[0]["axes"]["x"]["values"]
y = coords[0]["axes"]["y"]["values"]
z = coords[0]["axes"]["z"]["values"]
steps = coords[0]["axes"]["t"]["values"]
steps = [step.replace("Z", "") for step in steps]
steps = pd.to_datetime(steps)
# steps = list(range(len(steps)))

num = []
datetime = []
steps = []
for coverage in self.covjson["coverages"]:
num.append(coverage["mars:metadata"]["number"])
datetime.append(coverage["mars:metadata"]["Forecast date"])
steps.append(coverage["mars:metadata"]["step"])

nums = list(set(num))
datetime = list(set(datetime))
steps = list(set(steps))

param_values = {}

for parameter in self.parameters:
param_values = [[[self.get_values()[parameter]]]]
for ind, value in enumerate(self.get_values()[parameter]):
coords = self.get_coordinates()[parameter]
x = [coords[ind][0][0]]
y = [coords[ind][0][1]]
t = [coords[ind][0][4]]
num = [coord[0][3] for coord in coords]
coords_z = coords[ind]
z = [int(coord[2]) for coord in coords_z]
param_coords = {
"x": x,
"y": y,
"t": t,
"number": num,
"z": z,
}
dataarray = xr.DataArray(
param_values,
dims=dims,
coords=param_coords,
name=parameter,
)
dataarray.attrs["type"] = self.get_parameter_metadata(parameter)["type"]
dataarray.attrs["units"] = self.get_parameter_metadata(parameter)["unit"]["symbol"]
dataarray.attrs["long_name"] = self.get_parameter_metadata(parameter)["observedProperty"]["id"]
dataarraydict[dataarray.attrs["long_name"]] = dataarray
param_values[parameter] = []
for i, num in enumerate(nums):
param_values[parameter].append([])
for j, date in enumerate(datetime):
param_values[parameter][i].append([])
for k, step in enumerate(steps):
param_values[parameter][i][j].append([])
for coverage in self.covjson["coverages"]:
if (
coverage["mars:metadata"]["number"] == num
and coverage["mars:metadata"]["Forecast date"] == date
and coverage["mars:metadata"]["step"] == step
):
param_values[parameter][i][j][k] = coverage["ranges"][parameter]["values"]

ds = xr.Dataset(dataarraydict)
for parameter in self.parameters:
param_coords = {
"x": x,
"y": y,
"number": nums,
"datetime": datetime,
"t": steps,
"z": z,
}

dataarray = xr.DataArray(
[[param_values[parameter]]],
dims=dims,
coords=param_coords,
name=parameter,
)

dataarray.attrs["type"] = self.get_parameter_metadata(parameter)["type"]
dataarray.attrs["units"] = self.get_parameter_metadata(parameter)["unit"]["symbol"]
dataarray.attrs["long_name"] = self.get_parameter_metadata(parameter)["observedProperty"]["id"]
dataarraydict[dataarray.attrs["long_name"]] = dataarray

ds = xr.Dataset(dataarraydict)
for mars_metadata in self.mars_metadata[0]:
if mars_metadata != "date" and mars_metadata != "step":
ds.attrs[mars_metadata] = self.mars_metadata[0][mars_metadata]
2 changes: 1 addition & 1 deletion covjsonkit/encoder/BoundingBox.py
Original file line number Diff line number Diff line change
@@ -89,7 +89,7 @@ def from_polytope(self, result):
fields["lat"] = 0
fields["param"] = 0
fields["number"] = [0]
fields["step"] = 0
fields["step"] = [0]
fields["dates"] = []
fields["levels"] = [0]

72 changes: 41 additions & 31 deletions covjsonkit/encoder/Path.py
Original file line number Diff line number Diff line change
@@ -84,14 +84,20 @@ def from_polytope(self, result):
fields["lat"] = 0
fields["param"] = 0
fields["number"] = [0]
fields["step"] = 0
fields["step"] = [0]
fields["dates"] = []
fields["levels"] = [0]
fields["s"] = []
fields["l"] = []

self.walk_tree(result, fields, coords, mars_metadata, range_dict)

if len(fields["l"]) == 0:
fields["l"] = [0]

logging.debug("The values returned from walking tree: %s", range_dict) # noqa: E501
logging.debug("The coordinates returned from walking tree: %s", coords) # noqa: E501
logging.debug("The fields: %s", fields)

self.add_reference(
{
@@ -103,65 +109,69 @@ def from_polytope(self, result):
}
)

for date in coords.keys():
coord = coords[date]["composite"]
coords[date]["composite"] = []
start = 0
for level in set(fields["l"]):
for s in set(fields["s"]):
if (date, level, fields["number"][0], fields["param"][0], s) in range_dict:
cor_len = len(range_dict[(date, level, fields["number"][0], fields["param"][0], s)])
end = start + cor_len
for cor in coord[int(start) : int(end)]:
coords[date]["composite"].append([s, cor[0], cor[1], level])
start = end
logging.debug("The coordinates returned from walking tree: %s", coords) # noqa: E501

combined_dict = {}

for date in fields["dates"]:
if date not in combined_dict:
combined_dict[date] = {}
for level in fields["levels"]:
for level in fields["l"]:
for num in fields["number"]:
if num not in combined_dict[date]:
combined_dict[date][num] = {}
for para in fields["param"]:
if para not in combined_dict[date][num]:
combined_dict[date][num][para] = {}
# for s, value in range_dict[date][level][num][para].items():
for s in fields["step"]:
for s in set(fields["s"]):
key = (date, level, num, para, s)
# for k, v in range_dict.items():
# if k == key:
if s not in combined_dict[date][num][para]:
combined_dict[date][num][para][s] = range_dict[key]
if key in range_dict:
combined_dict[date][num][para][s] = range_dict[key]
# combined_dict[date][num][para][s] = range_dict[key]
else:
# Cocatenate arrays
combined_dict[date][num][para][s] += range_dict[key]
if key in range_dict:
combined_dict[date][num][para][s] += range_dict[key]
# for s in fields["s"]:

logging.debug("The values returned from combined dicts: %s", combined_dict) # noqa: E501

levels = fields["levels"]
if fields["param"] == 0:
raise ValueError("No parameters were returned, date requested may be out of range")
for para in fields["param"]:
self.add_parameter(para)

logging.debug("The parameters added were: %s", self.parameters) # noqa: E501

for date in coords.keys():
coord = coords[date]["composite"]
coords[date]["composite"] = []
for level in levels:
for cor in coord:
coords[date]["composite"].append([cor[0], cor[1], level])

for date in combined_dict.keys():
for num in combined_dict[date].keys():
val_dict = {}
for step in combined_dict[date][num][self.parameters[0]].keys():
val_dict[step] = {}
for para in combined_dict[date][num].keys():
if para not in val_dict:
val_dict[para] = []
for step in combined_dict[date][num][para].keys():
val_dict[step][para] = combined_dict[date][num][para][step]
for step in val_dict.keys():
mm = mars_metadata.copy()
mm["number"] = num
mm["step"] = step
temp = []
for coord in coords[date]["composite"]:
temp.append([step] + coord)
coords[date]["composite"] = temp
mm["Forecast date"] = date
self.add_coverage(mm, coords[date], val_dict[step])

# self.add_coverage(mars_metadata, coords, range_dict)
# return self.covjson
# with open('data.json', 'w') as f:
# json.dump(self.covjson, f)
val_dict[para].extend(combined_dict[date][num][para][step])
mm = mars_metadata.copy()
mm["number"] = num
mm["Forecast date"] = date
if "levelist" in mm:
del mm["levelist"]
self.add_coverage(mm, coords[date], val_dict)

return self.covjson
Loading