Skip to content

Commit

Permalink
show_gids: improvements for large scale
Browse files Browse the repository at this point in the history
On some hosts, script could run for many minutes.
Reason : On host with many NICs, each interface is scanned unordered
with all the GIDs.
Add:
1. Flag --slim - order the GID table, count number of empty GIDs,
break from loop once number of empty GIDs is larger than predefined MAX (2).
Explanation: GIDs are assigned in order, if GID is empty, it means all the following
GIDs will be empty as well.
Exception: if some GID index was freed and new GID was assigned, there might be holes.

2. Flag --dev - show_gids used to get device as an optional only argument.
As there is additional argument now, need dedicated option to allow several arguments.

3. Add help

Result:
By default - script will run as before - will scan all GIDs
--slim should be used for large scale to allow reasonable timed run.
  • Loading branch information
marishkinv committed Jul 11, 2023
1 parent 54a51a7 commit 4768504
Showing 1 changed file with 33 additions and 2 deletions.
35 changes: 33 additions & 2 deletions sbin/show_gids
Original file line number Diff line number Diff line change
Expand Up @@ -74,15 +74,46 @@ function print_gids()

echo -e "DEV\tPORT\tINDEX\tGID\t\t\t\t\tIPv4 \t\tVER\tDEV"
echo -e "---\t----\t-----\t---\t\t\t\t\t------------ \t---\t---"
DEVS=$1
#Break after predefined number of 0 GIDS found
#Assuming that the rest will be zero as well
#Needed on hosst with large number of NICs, to avoid script slow run
MAX_NUM_OF_ZERO_GIDS=2
IS_SLIM=0

while [ $# -gt 0 ]; do
case "$1" in
--dev=*)
DEVS="${1#*=}"
;;
--slim)
IS_SLIM=1
;;
*)
echo "show_gids will print gids table for all RDMA devices"
echo "-d|--dev=<rdma device> Can choose specifci mlx devices"
echo "-s|--slim will break gid table scan after predefine max num-2 of zero GIDs"
exit 1
esac
shift
done

if [ -z "$DEVS" ] ; then
DEVS=$(ls /sys/class/infiniband/)
fi

for d in $DEVS ; do
for p in $(ls /sys/class/infiniband/$d/ports/) ; do
for g in $(ls /sys/class/infiniband/$d/ports/$p/gids/) ; do
declare -i ZERO_GIDS_CNT
ZERO_GIDS_CNT=0
for g in $(ls -v /sys/class/infiniband/$d/ports/$p/gids/) ; do
gid=$(cat /sys/class/infiniband/$d/ports/$p/gids/$g);
if [ $gid = 0000:0000:0000:0000:0000:0000:0000:0000 ] ; then
if [ $IS_SLIM = 1 ] ; then
ZERO_GIDS_CNT+=1
if [ $ZERO_GIDS_CNT = $MAX_NUM_OF_ZERO_GIDS ] ; then
break
fi
fi
continue
fi
if [ $gid = fe80:0000:0000:0000:0000:0000:0000:0000 ] ; then
Expand Down

0 comments on commit 4768504

Please sign in to comment.