This project is mirrored from https://*****@github.com/SciML/DiffEqFlux.jl.git. Pull mirroring updated .
  1. 23 Nov, 2020 4 commits
  2. 22 Nov, 2020 3 commits
  3. 21 Nov, 2020 1 commit
  4. 08 Nov, 2020 1 commit
  5. 07 Nov, 2020 2 commits
  6. 03 Nov, 2020 2 commits
  7. 31 Oct, 2020 1 commit
    • vaibhav dixit's avatar
      Add warning to sciml_train, add a dispatch of sciml_train to GalacticOptim and... · a8fa3208
      vaibhav dixit authored
      Add warning to sciml_train, add a dispatch of sciml_train to GalacticOptim and rename layer_sciml test file to galacticoptim  (#424)
      
      * CompatHelper: bump compat for "GalacticOptim" to "0.4"
      
      * Deprecate sciml_train and update all tests to use GalacticOptim
      
      * Use zygote as the default AD and use @deprecate
      
      * Add sciml_train dispatch that wraps GalacticOptim and give deprecation warning
      
      * remove GalacticOptim dependency
      
      Don't know how this slipped by...
      
      * Rename layers_sciml to galacticoptim and use sciml_train dispatch to GalacticOptim in tests
      
      * Add back GalaticOptim dependency
      
      * Concretise ADType in GalacticOptim dispatch
      
      * Remove kwargs splatting in OptimizationFunction
      
      * Remove old dispatches and dispatch to GalacticOptim
      
      * Fix spline_layer test typo
      
      * Use instantiate_function
      
      * pass nothing for p in instantiate_function
      
      * Wrap before instantiating
      
      * Update newton neural ode tests
      
      * Remove sciml_train from tests
      Co-authored-by: default avatargithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
      Co-authored-by: Christopher Rackauckas's avatarChris Rackauckas <[email protected]>
      a8fa3208
  8. 26 Oct, 2020 1 commit
  9. 12 Oct, 2020 1 commit
    • abstractingagent's avatar
      Another strategy to avoid local minima (#431) · cdd00857
      abstractingagent authored
      * Another strategy to avoid local minima 
      
      I discovered another strategy in order for one to be able to train a neural ODE and avoid local minima. I used flux because I didn't know how to do it with sciml, but it's pretty straightforward. Simply train both the initial conditions and the parameters first, and then reset the IC back to it's original values and train only the parameters. Follow it up with a longer ADAM run or BFGS for fine tuning. Sometimes in the first training round it doesn't fully converge (for the longer timespan like 0,10), but just run both iterations until their end or add more iterations and it always converges. Going to post the code here and then update the PR with the documentation added once the idea is verified.
      
      ```julia
      using DiffEqFlux, OrdinaryDiffEq, Flux, Optim, Plots, DifferentialEquations
      
      
      #Starting with an example with tspan (0, 5)
      u0 = Float32[2.0; 0.0]
      datasize = 30
      tspan = (0.0f0, 5.0f0)
      tsteps = range(tspan[1], tspan[2], length = datasize)
      
      function trueODEfunc(du, u, p, t)
          true_A = [-0.1 2.0; -2.0 -0.1]
          du .= ((u.^3)'true_A)'
      end
      
      prob_trueode = ODEProblem(trueODEfunc, u0, tspan)
      ode_data = Array(solve(prob_trueode, Tsit5(), saveat = tsteps))
      
      dudt2 = Chain(Dense(2,16, tanh),
                   Dense(16,2))
      
      
      p,re = Flux.destructure(dudt2) # use this p as the initial condition!
      dudt(u,p,t) = re(p)(u) # need to restrcture for backprop!
      prob = ODEProblem(dudt,u0,tspan)
      predict_n_ode()
      loss_n_ode()
      cb(;doplot=true)
      
      data = Iterators.repeated((), 1000)
      
      Flux.train!(loss_n_ode, Flux.params(u0, p), data,
                          Flux.Optimise.ADAM(0.05), cb = cb)
      
      u0 = Float32[2.0; 0.0]
      Flux.train!(loss_n_ode, Flux.params(p), data,
                  Flux.Optimise.ADAM(0.05), cb = cb)
      
      #Now use the same technique for longer tspan (0, 10)
      datasize = 30
      tspan = (0.0f0, 10.0f0)
      tsteps = range(tspan[1], tspan[2], length = datasize)
      
      prob_trueode = ODEProblem(trueODEfunc, u0, tspan)
      ode_data = Array(solve(prob_trueode, Tsit5(), saveat = tsteps))
      
      dudt2 = Chain(Dense(2,16, tanh),
                   Dense(16,2))
      
      
      p,re = Flux.destructure(dudt2) # use this p as the initial condition!
      dudt(u,p,t) = re(p)(u) # need to restrcture for backprop!
      prob = ODEProblem(dudt,u0,tspan)
      predict_n_ode()
      loss_n_ode()
      cb(;doplot=true)
      
      data = Iterators.repeated((), 1500)
      
      Flux.train!(loss_n_ode, Flux.params(u0, p), data,
                          Flux.Optimise.ADAM(0.05), cb = cb)
      
      u0 = Float32[2.0; 0.0]
      Flux.train!(loss_n_ode, Flux.params(p), data,
                  Flux.Optimise.ADAM(0.05), cb = cb)
      ```
      
      * Update local_minima.md
      
      Update documentation with example and text corresponding text
      
      * Update local_minima.md
      
      Update documentation with example and text corresponding text
      cdd00857
  10. 07 Oct, 2020 2 commits
  11. 04 Oct, 2020 1 commit
  12. 27 Sep, 2020 3 commits
  13. 25 Sep, 2020 6 commits
  14. 21 Sep, 2020 2 commits
  15. 19 Sep, 2020 1 commit
  16. 15 Sep, 2020 2 commits
  17. 11 Sep, 2020 1 commit
  18. 09 Sep, 2020 1 commit
  19. 08 Sep, 2020 1 commit
  20. 07 Sep, 2020 4 commits