Do you have a keras version?

[xuzhang5788]

No description provided.

[izmailovpavel]

Hi, currently we don't have any keras implementation or examples. We will try to provide an example some time soon, probably after may 18. If you come up with an example, please post it in this issue.

[xuzhang5788]

Thank you so much. I am looking forward to seeing your new implementation.

[nile649]

Hi, so where you guys able to make keras or tensorflow implementation? I did try your pytorch version code with cyclic learning rate. looking forward for keras implementation, if possible just provide the function.

Thanks

[happysmile123]

Hi, I use keras+tensorflow build a swa model ,the dataset is mnist ,i tried with cycle lr ,but i found the result not so stable ,sometimes the swa is better than single model ,however sometimes not .And I saw you trained 200-300epoches, i just trained several epoch like 2 or 3,but the accuracy is 0.98 or even better ,so when i averaged the weight , there seems to be little improvement .so I don not know whether swa is worth to use . After all ,if you could provide a keras or tensorflow version ,it could be better

[xuzhang5788]

Here is a SWA keras version (https://github.com/kristpapadopoulos/keras-stochastic-weight-averaging). But it only can do a constant learning rate for a user defined amount of epochs.

His swa.py is very similar to another swa.py showed in the article "Adding a cutting-edge deep learning training technique to the fast.ai library" at Medium.

@izmailovpavel Do you think we can use this keras implementation in our keras model training? Many thanks.

[samjia533332]

Hi thank you so much for your reply! I actually found another version that has both cyclic and constant. *More green and save our planet!* -- Sam Zhenyu Jia Data Mining Engineer NetEase, Inc. University of Michigan Master in Applied Statistics Class of 2015

…

On Fri, Nov 2, 2018 at 2:35 AM xuzhang5788 ***@***.***> wrote: Here is a SWA keras version ( https://github.com/kristpapadopoulos/keras-stochastic-weight-averaging). But it only can do a constant learning rate for a user defined amount of epochs. His swa.py is very similar to another swa.py showed in the article "Adding a cutting-edge deep learning training technique to the fast.ai library" at Medium. @izmailovpavel <https://github.com/izmailovpavel> Do you think we can use this keras implementation in our keras model training? Many thanks. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH5uTWrpZzia0ddNUUTXYXq4m-Z7BoKQks5uqz8DgaJpZM4TwNC-> .

[xuzhang5788]

@samjia533332
Would you like to give us a link about your found version? Many thanks.

[Venka97]

@samjia533332 @xuzhang5788 Did you find the version with cyclic lr for keras?

[simon-larsson]

I have implemented the constant and cyclic lr schedules with the batch norm fix in Keras here: https://github.com/simon-larsson/keras-swa

I´m not 100% on the cyclic schedule since there is no implementation of it here that I could use as reference.

[izmailovpavel]

@simon-larsson that's great :) Do the results you are getting look similar to what we get in PyTorch?

[simon-larsson]

@izmailovpavel the constant learning rate schedule gave similar results on cifar10 and 2-3% acc on some private datasets. I want to replicate more when I get access to hardware again.

Is it ok if I email you some questions about the cyclic learning schedule?

[izmailovpavel]

@simon-larsson great! Sure, feel free to email me at pi390@nyu.edu.