Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test fix #1159

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Test fix #1159

wants to merge 12 commits into from

Conversation

TorkelE
Copy link
Member

@TorkelE TorkelE commented Jan 8, 2025

Turns out that MTK now gives over/under determined system warnings for certain cases when SDEProblems are generated. What is slightly odd is that this is not the case for the corresponding ODEs (does this make sense @AayushSabharwal ?).

We had some tests which depended on checking warnings for when conserved quantities are eliminated. These checks no longer worked when MTK threw additional warnings. I have removed the MTK warnings. This makes tests pass, but we might want to check why these warnings are actually generated.

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Another failure has appeared, apparently affecting remake for SDEs (ODEs work fine). This is the example:

using Catalyst, StochasticDiffEq
model = @reaction_network begin
    @observables XY ~ X + Y
    (kp,kd), 0 <--> X
    (k1,k2), X <--> Y
end
@unpack XY, X, Y, kp, kd, k1, k2 = model

# Creates problem.
u0_vals = [X => 4, Y => 5]
tspan = (0.0, 10.0)
p_vals = [kp => 1.0, kd => 0.1, k1 => 0.25, k2 => 0.5]
sprob = SDEProblem(model,u0_vals, tspan, p_vals)

# tests remake.
rp = remake(sprob; u0 = [Y => 7])
rp[[X, Y]] == [4, 7] # false (should be true).
rp[X] # -7.0 (!, should be 4.0, X is never changed)
rp[Y] # 7.0 (correct)

oprob = ODEProblem(model,u0_vals, tspan, p_vals)
rp = remake(oprob; u0 = [Y => 7])
rp[X] # 4.0, correct.

I will raise with MTK

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

A different case, where we cannot even call remake

using Catalyst, StochasticDiffEq
model = @reaction_network begin
    (p1,2.0), 0 <--> X
    (p2,2.0), 0 <--> Y
end
@unpack X, Y, p1, p2 = model

# Creates problem.
u0_vals = [X => 4, Y => 5]
tspan = (0.0, 10.0)
p_vals = [p1 => 1.0, p2 => 0.1]
sprob = SDEProblem(model,u0_vals, tspan, p_vals)

# tests remake.
rp = remake(sprob; u0 = [Y => 7] # ERROR: Initialization incomplete. Not all of the state variables of the DAE system can be determined by the initialization. 

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

MTK issue: SciML/ModelingToolkit.jl#3295

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Another one! Turns out that parameter defaults are broken for DiscreteProblems (but not other problem types:

using Catalyst
model = @reaction_network begin
    @parameters Y0
    @species Y(t) = Y0
    (k1,k2), X <--> Y
end
@unpack X, Y, Y0, k1, k2 = model

u0 = [X => 4]
tspan = (0.0, 10.0)
p = [k1 => 0.25, k2 => 10, Y0 => 2]

oprob = ODEProblem(model, u0, tspan, p) # Works.
sprob = SDEProblem(model, u0, tspan, p) # Works.
dprob = DiscreteProblem(model, u0, tspan, p) # Errors.

@ChrisRackauckas
Copy link
Member

What error does that spit out?

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

ERROR: MethodError: no method matching unhack_observed(::Vector{Equation}, ::RecursiveArrayTools.ArrayPartition{Any, Tuple{…}})
The function `unhack_observed` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  unhack_observed(::Vector{Equation}, ::Vector{Equation})
   @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/nonlinear/initializesystem.jl:352

Stacktrace:
  [1] generate_initializesystem(sys::JumpSystem{…}; u0map::Dict{…}, pmap::Dict{…}, initialization_eqs::Vector{…}, guesses::Dict{…}, default_dd_guess::Float64, algebraic_only::Bool, check_units::Bool, check_defguess::Bool, name::Symbol, extra_metadata::@NamedTuple{…}, kwargs::@Kwargs{})
    @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/nonlinear/initializesystem.jl:17
  [2] generate_initializesystem
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/nonlinear/initializesystem.jl:6 [inlined]
  [3] ModelingToolkit.InitializationProblem{…}(sys::JumpSystem{…}, t::Float64, u0map::Dict{…}, parammap::Dict{…}; guesses::Dict{…}, check_length::Bool, warn_initialize_determined::Bool, initialization_eqs::Vector{…}, fully_determined::Nothing, check_units::Bool, use_scc::Bool, kwargs::@Kwargs{…})
    @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/diffeqs/abstractodesystem.jl:1307
  [4] InitializationProblem
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/diffeqs/abstractodesystem.jl:1285 [inlined]
  [5] #_#1075
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/diffeqs/abstractodesystem.jl:1263 [inlined]
  [6] InitializationProblem
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/diffeqs/abstractodesystem.jl:1262 [inlined]
  [7] #InitializationProblem#1073
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/diffeqs/abstractodesystem.jl:1251 [inlined]
  [8] InitializationProblem
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/diffeqs/abstractodesystem.jl:1250 [inlined]
  [9] maybe_build_initialization_problem(sys::JumpSystem{…}, op::Dict{…}, u0map::Dict{…}, pmap::Dict{…}, t::Float64, defs::Dict{…}, guesses::Dict{…}, missing_unknowns::Set{…}; implicit_dae::Bool, kwargs::@Kwargs{…})
    @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/problem_utils.jl:546
 [10] maybe_build_initialization_problem
    @ ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/problem_utils.jl:526 [inlined]
 [11] process_SciMLProblem(constructor::Type, sys::JumpSystem{…}, u0map::Vector{…}, pmap::Vector{…}; build_initializeprob::Bool, implicit_dae::Bool, t::Float64, guesses::Dict{…}, warn_initialize_determined::Bool, initialization_eqs::Vector{…}, eval_expression::Bool, eval_module::Module, fully_determined::Nothing, check_initialization_units::Bool, tofloat::Bool, use_union::Bool, u0_constructor::typeof(identity), du0map::Nothing, check_length::Bool, symbolic_u0::Bool, warn_cyclic_dependency::Bool, circular_dependency_max_cycle_length::Int64, circular_dependency_max_cycles::Int64, substitution_limit::Int64, use_scc::Bool, kwargs::@Kwargs{})
    @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/problem_utils.jl:669
 [12] DiscreteProblem(sys::JumpSystem{…}, u0map::Vector{…}, tspan::Tuple{…}, parammap::Vector{…}; use_union::Bool, eval_expression::Bool, eval_module::Module, kwargs::@Kwargs{})
    @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/jumps/jumpsystem.jl:428
 [13] DiscreteProblem(sys::JumpSystem{…}, u0map::Vector{…}, tspan::Tuple{…}, parammap::Vector{…})
    @ ModelingToolkit ~/.julia/packages/ModelingToolkit/iQ7So/src/systems/jumps/jumpsystem.jl:414
 [14] DiscreteProblem(::ReactionSystem{…}, ::Vector{…}, ::Tuple{…}, ::Vector{…}; name::Symbol, combinatoric_ratelaws::Bool, checks::Bool, kwargs::@Kwargs{})
    @ Catalyst ~/.julia/dev/Catalyst/src/reactionsystem_conversions.jl:855
 [15] DiscreteProblem(::ReactionSystem{…}, ::Vector{…}, ::Tuple{…}, ::Vector{…})
    @ Catalyst ~/.julia/dev/Catalyst/src/reactionsystem_conversions.jl:846
 [16] top-level scope
    @ ~/Desktop/Julia Playground/Environment - Catalyst only/catalyst_playground.jl:57
Some type information was truncated. Use `show(err)` to see complete types.

@isaacsas
Copy link
Member

isaacsas commented Jan 8, 2025

@TorkelE probably good to add these examples and their errors messages to the tracking MTK issue?

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

New error, but this one seems codecov related

Run codecov/codecov-action@v5
Run CC_ACTION_VERSION=$(cat ${GITHUB_ACTION_PATH}/src/version)
==> Running Action version 5.1.2
Run git config --global --add safe.directory /home/runner/work/Catalyst.jl/Catalyst.jl
Run CC_FORK="false"
Run if [ "false" == 'true' ] && [ "$CC_FORK" != 'true' ];
==> Token set from input
Run if [ -z "$CC_BRANCH" ] && [ -z "$CC_TOKEN" ] && [ "$CC_FORK" == 'true' ]
Run if [ -z "$CC_SHA" ];
Run ${GITHUB_ACTION_PATH}/dist/codecov.sh
     _____          _
    / ____|        | |
   | |     ___   __| | ___  ___ _____   __
   | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
   | |___| (_) | (_| |  __/ (_| (_) \ V /
    \_____\___/ \__,_|\___|\___\___/ \_/
                            Wrapper-0.0.31
                                  
==> Detected linux
 -> Downloading https://cli.codecov.io/latest/linux/codecov
==> Finishing downloading linux:latest
      Version: v9.1.1
 
gpg: directory '/home/runner/.gnupg' created
gpg: keybox '/home/runner/.gnupg/pubring.kbx' created
gpg: /home/runner/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" imported
gpg: Total number processed: 1
gpg:               imported: 1
==> Verifying GPG signature integrity
 -> Downloading https://cli.codecov.io/latest/linux/codecov.SHA256SUM
 -> Downloading https://cli.codecov.io/latest/linux/codecov.SHA256SUM.sig
 
gpg: Signature made Sat Dec  7 16:07:53 [20](https://github.com/SciML/Catalyst.jl/actions/runs/12675329977/job/35326113772#step:8:21)24 UTC
gpg:                using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpg: Good signature from "Codecov Uploader (Codecov Uploader Verification Key) <security@codecov.io>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: [27](https://github.com/SciML/Catalyst.jl/actions/runs/12675329977/job/35326113772#step:8:29)03 4E7F DB85 0E0B BC2C  62FF 806B B28A ED77 9869
codecov: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
codecov: FAILED
==> Could not verify SHASUM. Please contact Codecov if problem continues
    Exiting...
Error: Process completed with exit code 1.

(I am trying what happens if I just rerun it)

@isaacsas
Copy link
Member

isaacsas commented Jan 8, 2025

What's the new HC doc build failure?

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Not just HC (didn't see that one, but some in nonlinear solve and symbolic stochiometry tests), but we suddenly got a lots of errors, including stuff that passed an hour ago. I re-ran everything again to see what happens.

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Yeah, the symbolic stoichiometry tests are broken now. Which is funny as these are before the MTK tests that were failing (and fixes), so we shouldn't even have gotten there if those weren't working. I tried to reproduce locally, but worked fine. Pkg marked everything as up to date, but maybe there has been a very recent release which CI pulls but my local Pkg is unaware of.

@ChrisRackauckas
Copy link
Member

You're not looking at the generated code right? The only release of today is a fix to change ^ to NaNMath.pow, but that shouldn't break anything except tests that expect ^ specifically

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Not sure really. I have a bunch of other stuff to sort out, and there is already another bunch of tests failing. Will have a look at this again tomorrow, if I cannot reproduce locally then I will have to look into it deeper.

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Cannot see any HC problems, but what is causing issues is stuff like

MethodError: no method matching factorial(::Float64)

and

 MethodError: no method matching binomial(::Float64, ::Float64)

I think we had some problem like this long ago, but should have been sorted by now.

@isaacsas
Copy link
Member

isaacsas commented Jan 8, 2025

I'll make another PR later today to cap MTK at the previous release to see if that fixes stuff (and hopefully allow us to work on other PRs).

@isaacsas
Copy link
Member

isaacsas commented Jan 8, 2025

Sounds like jump initial conditions or integer parameters are now getting promoted to floating point?

@TorkelE
Copy link
Member Author

TorkelE commented Jan 8, 2025

Sounds like a good guess, hopefully we'll know for sure soon.

@ChrisRackauckas
Copy link
Member

Are you forcing split=false?

@isaacsas
Copy link
Member

isaacsas commented Jan 8, 2025

We don't do anything special like that in Catalyst.

@AayushSabharwal
Copy link
Member

AayushSabharwal commented Jan 9, 2025

For the SDE remake bug:

using Catalyst, StochasticDiffEq
model = @reaction_network begin
    @observables XY ~ X + Y
    (kp,kd), 0 <--> X
    (k1,k2), X <--> Y
end
@unpack XY, X, Y, kp, kd, k1, k2 = model

# Creates problem.
u0_vals = [X => 4, Y => 5]
tspan = (0.0, 10.0)
p_vals = [kp => 1.0, kd => 0.1, k1 => 0.25, k2 => 0.5]
sprob = SDEProblem(model,u0_vals, tspan, p_vals)

# tests remake.
rp = remake(sprob; u0 = [Y => 7])
rp[[X, Y]] == [4, 7] # false (should be true).
rp[X] # -7.0 (!, should be 4.0, X is never changed)
rp[Y] # 7.0 (correct)

I don't yet know how, but this is the equations in the initialization system before structural_simplify:

 Y(t) ~ 7
 XY(t) ~ Y(t) + X(t)

And after:

 Y(t) ~ 7.0
 XY(t) ~ -0.0
 XY(t) ~ Y(t) + X(t)

Somehow it's adding XY ~ 0

EDIT: the bug is in alias_elimination! somewhere

@ChrisRackauckas
Copy link
Member

Can we turn the alias elimination issue into an MTK issue to track?

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

Can we turn the alias elimination issue into an MTK issue to track?

SciML/ModelingToolkit.jl#3302

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

Ok, there's the issue ou mentioned again Sam. Still cannot reproduce locally even when I think I am on the same environment, so might take a bit to figure out.

Copy link
Member

@isaacsas isaacsas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the point of this PR to have a place to test against the latest versions of everything?

Also, what are you referring to by the problem that is reappearing? Tests were passing?

Project.toml Outdated Show resolved Hide resolved
Project.toml Outdated Show resolved Hide resolved
docs/Project.toml Outdated Show resolved Hide resolved
@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

I tried to mark the latest version (which we are testing against). Figured ModelingToolkit = "9.60" didn't make sense, as we aren't sure that versions beyond 9.60 actually work. Since 9.60 is the highest MTK version we have tested against, we shouldn't make u as compatible with later versions until that is actually tested. Wasn't this what we wanted to do with the capping?

@isaacsas
Copy link
Member

isaacsas commented Jan 9, 2025

No, the goal is to not have master be capped. Otherwise we won't see when things break. The point is to cap releases so they can't end up getting versions of MTK/SciMLBase that don't work with them.

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

Sounds good

@isaacsas
Copy link
Member

isaacsas commented Jan 9, 2025

(I only capped master yesterday so we could get all these PRs that are building up merged until the fixes you've raised issues about here are merged.)

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

This was the problem you were talking about: #1170 (comment), but seems to dealt with it (for now)already)

TorkelE and others added 3 commits January 9, 2025 14:05
Co-authored-by: Sam Isaacson <isaacsas@users.noreply.github.com>
Co-authored-by: Sam Isaacson <isaacsas@users.noreply.github.com>
Co-authored-by: Sam Isaacson <isaacsas@users.noreply.github.com>
@isaacsas
Copy link
Member

isaacsas commented Jan 9, 2025

I'm not seeing that it is fixed? I'm getting errors about it as of my most recent PR today.

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

No, it is not fixed. My commented was that the problem is still there, but I could not reproduce locally so might not be able to make an immediate fix. Then I referenced #1170 meaning that you were already taking care of it (as it commenting it out for now, although I referenced the issue, not the PR).

@isaacsas
Copy link
Member

isaacsas commented Jan 9, 2025

Got it.

@isaacsas
Copy link
Member

isaacsas commented Jan 9, 2025

Could it be you are using different package versions than the doc builds?

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

Something is definitely off. I have some other stuff I really need to sort out asap, but will sit down to have a closer look at it.

@TorkelE
Copy link
Member Author

TorkelE commented Jan 9, 2025

Ok, this one is ready now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants