Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Account for python's optimize mode to enable faster model building #633

Open
irm-codebase opened this issue Jul 4, 2024 · 2 comments
Open
Labels
enhancement v0.7 (upcoming) version 0.7
Milestone

Comments

@irm-codebase
Copy link
Contributor

irm-codebase commented Jul 4, 2024

What can be improved?

We have the possibility of a speed upgrade by changing "if then raise" statements to assert cases in the backend.
In cases were we know that a model is safe to run, we could call the -O option for python to skip these cases automatically.

We have these in several loops in the backend, so it could net a performance bump.

Version

v0.7

@irm-codebase irm-codebase added enhancement v0.7 (upcoming) version 0.7 labels Jul 4, 2024
@irm-codebase
Copy link
Contributor Author

Here is an example for the expression_parser.py file. The for is entirely for checking.
Although in this case, maybe if __debug__ is better than assert.

    def as_array(self) -> xr.DataArray:  # noqa: D102, override
        lhs, rhs = self._eval("array")
        where = self.eval_attrs["where_array"]
        for side, arr in {"left": lhs, "right": rhs}.items():
            extra_dims = set(arr.dims).difference(set(where.dims))
            if extra_dims:
                raise BackendError(
                    f"({self.eval_attrs['equation_name']}, {self.instring}) | "
                    f"The {side}-hand side of the equation is indexed over dimensions not present in `foreach`: {extra_dims}"
                )
        lhs_where = lhs.broadcast_like(where)
        rhs_where = rhs.broadcast_like(where)

        match self.op:
            case "==":
                op = np.equal
            case "<=":
                op = np.less_equal
            case ">=":
                op = np.greater_equal
        constraint = op(lhs_where, rhs_where, where=where.values, dtype=np.object_)
        return xr.DataArray(constraint)

@irm-codebase
Copy link
Contributor Author

irm-codebase commented Jul 4, 2024

Some hits of viable cases I found. Number on the left indicates "hits" (also national example).
Couldn't find a way to make the debugger add them all together, sadly.

image

@sjpfenninger sjpfenninger added this to the 0.7.x milestone Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement v0.7 (upcoming) version 0.7
Projects
None yet
Development

No branches or pull requests

2 participants