[HELP] UCX rdma traffic only on one ib device.

[YaoHaoLau]

I have multiple mlx5 devices in my container and also created multiple endpoint for different RMA buffers' put. But all of rdma traffic distributes on single mlx5 device(the first) even bandwidth reaches its limit with UCX_NET_DEVICES specifies all.

However, It's possible to make these traffic on another mlx5 device with UCX_NET_DEVICES changes.

Is it possible to make ib traffic distributed on all of mlx5 devices with v1.16.0 ? Enable multi-rail doesn't help.

Thanks for your reply.

[YaoHaoLau]

After a quick scanning through the source code, enable MULTI_RAIL helps, but not elegant. Device/TL select looks smart, but not that flexible in real online situation.