Tutorial for using Optimization and DiffEqParamEstim for parameter fitting

[TorkelE]

Adds a single tutorial page. Currently put under catalyst_applications. Once #701 passes I will separately add this to the inverse problem section.

The base tutorial is done (but needs to be reread a bit), but some auxiliary sections need to be added.

[TorkelE]

Currently ready for Vaibhav to have a look at.

There is one problem, this bit (when fitting two data both for S and P) fails quite bad:

optprob_S_P = OptimizationProblem(loss_function_S_P, [1.0,1.0, 1.0])
optsol_S_P = solve(optprob_S_P, CMAEvolutionStrategyOpt())
oprob_fitted_S_P = remake(oprob; p=optsol_S_P.u)
fitted_sol_S_P = solve(oprob_fitted_S_P, Tsit5())
plot!(fitted_sol_S_P; idxs=[:S, :P], label="Fitted solution", linestyle=:dash, lw=6, color=[:lightblue :pink])

Is there some easy modification to options I can provide to make this work?

[isaacsas]

How does this fit in with the current Optimization based tutorial?

Also, why use CMAEvolutionStrategyOpt?

[TorkelE]

How does this fit in with the current Optimization based tutorial?

That will be moved to an separate example (and rewritten a bit and be a focus on Fitting parameters of oscillatory processes).

Also, why use CMAEvolutionStrategyOpt?

I just picked one (don't really know much on sensible selection of optimisation methods), happy to change.

[Vaibhavdixit02]

I second Sam's point above, CMAES doesn't seem like a good choice here. I would recommend using a first order method (BFGS) instead, you can use it from NLopt so that'll cover multiple solver packages as well which might be useful for users to see.

[TorkelE]

Yeah, I just wanted to select a optimiser from another package, I can change to that one.

[TorkelE]

Updated to use BFGS instead.

[TorkelE]

This is ready for merging now.

[TorkelE]

Will go through and check the merge and latest updates this evening. But in summary, this is now rebased on master (where the inverse problem section exists). I have also moved the previous parameter fitting tutorial to a sub folder for examples, and rewritten it a bit to make it an example of how to fit an oscillation (while this new tutorial simply shows the SciML/Optimization workflow).

[isaacsas]

I need to go through this carefully still, but I think I’m going to have the same comment about not dynamically generating graphs and such.

[TorkelE]

See the reply there. Yes, I agree. I will see what can be done. We could always merge and put in an issue that this should be dealt with at some point of time (although we are starting to get so many issues that it is easy for stuff to get lost there, which makes me slightly reluctant)

[TorkelE]

This one is ready for merging

[isaacsas]

Ok, I’ll let you know once I’ve read it carefully.

[TorkelE]

replaced by #740