TL/MLX5: one-sided mcast reliability init

[MamziB]

TL/MLX5: one-sided mcast reliability init that will be used for mcast-based allgather

[artemry-nv]

bot:retest

[janjust]

bot:retest

[samnordmann]

Can you please provide more context and explanation about the PR? It is a big PR so it is challenging to review without context and walkthrough.

I see different new options in ucc_tl_mlx5_mcast_coll_comm_init_spec_t, like enable_truly_zero_copy_pipelined_allgather, one_sided_reliability_enabled. Can you explain what is the role of them and why they cannot be treated in different PRs?

Also, can you add tests for those new features?

[MamziB]

@nsarka @janjust @samnordmann Thanks everyone for the constructive comments. I have pushed all the requested changes. Please take a look and feel free to hit the resolve button if they look good.

[MamziB]

/* below data structures are used in async design only */

@samnordmann These options are for algo selection and check if one-sided is enabled. Let me remove the options that are not necessary for this PR.

[MamziB]

@samnordmann Thanks for the comments. I added a new commit (a separate commit) with all the new requested changes. It will be easier this way to track what has changed, at the end I will squash all the commits into a single one.

[samnordmann]

thanks for addressing the comments!

[janjust]

Thank for addressing my comments

[MamziB]

@Sergei-Lebedev Thanks Sergey for the comments. I have resolved all of them.

[Sergei-Lebedev]

@MamziB pls rebase

[MamziB]

@Sergei-Lebedev Thanks for the comments, I have rebased it on top of latest master.