Skip to content

Fix wrong token latency when batch size is greater than 1 #4708

Fix wrong token latency when batch size is greater than 1

Fix wrong token latency when batch size is greater than 1 #4708