Only fuse pointwise with layernorm on the first argument #21957

Job	Run time
Check if image exists in registry	5s
cancel	2s
licensing	6m 8s
pyflakes	3m 5s
linux-fpga (ubuntu-20.04, debug)	4h 10m 37s
linux (ubuntu-20.04, debug)	4h 7m 5s
misspell	21s
linux (ubuntu-20.04, release)	3h 23m 5s
linux (ubuntu-20.04, codecov)	3h 17m 10s
Build SLES image	0s
Build image	0s
format	4m 53s
tidy	1h 29m 51s
sles	38m 34s
cppcheck	59m 47s
	18h 20m 43s

Provide feedback