fix(optimizer/hybrid_zero_optim.py): fix layer norm grad allreduce when sp is True#53

Merged

huangting4201 merged 1 commit intoInternLM:developfrom huangting4201:fix/norm-weight-allreduce

Feb 22, 2024

+10-2

Commits on Feb 22, 2024

fix(optimizer/hybrid_zero_optim.py): fix layer norm grad allreduce when sp
huangting4201
committed