Updating gradients in-place

[willtebbutt]

Add support for providing pre-allocated elements of the gradient tape. Currently viable as a hack of the following form:

using Nabla, NNlib

t = Tape()
W = Leaf(t, rand(5,5))
b = Leaf(t, rand(5))
x = rand(5)

y = σ.(W*x .+ b)

# Previously accumulated gradients.
∇W, ∇b = randn(5, 5), randn(5)

# Create empty tape.
rvs = Nabla.reverse_tape(y, randn(5))

# Point entries of tape towards previously allocated arrays.
rvs.tape[W.pos], rvs.tape[b.pos] = ∇W, ∇b

# Run the reverse pass.
Nabla.propagate(y.tape, rvs)

but this is not at all robust and will silently compute gradients incorrectly if one isn't careful.

[tmigot]

I am wondering if there was any update on this topic?

[willtebbutt]

I'm afraid not, and there is not going to be any progress any time soon. (I've just updated the readme to reflect the fact that no one is maintaining this repo anymore)

[tmigot]

Oh, sorry to hear that. Thanks for your answer.