How specify target cuda device?

[tszhang97]

I'm using onnx gpu backend and want to set cuda:2 device. It seems that I could not select cuda device.

[Tau-J]

Hi @tszhang97 , thanks for using rtmlib. The easiest way is to set the visible device, e.g. CUDA_VISIBLE_DEVICES=gpu_ids python3 xxx.py

[tszhang97]

I know this operation, however I'm using rtmlib as a submodule of a large project. Maybe we can add provider_options like this:
provider_options = [{"device_id": device_id}]
self.session = ort.InferenceSession(path_or_bytes=onnx_model,
providers=[providers],
provider_options=provider_options)

[tszhang97]

Additional operation to get device_id and add provider_options works for me.

[Tau-J]

Additional operation to get device_id and add provider_options works for me.

Hi @tszhang97 , would you like to raise a PR to this repo? Any contribution is much appreciated.

[edwardnguyen1705]

Dear @Tau-J ,

Would it be possible to support specifying GPU ID like this cap-ntu/ML-Model-CI#37?

If multiple processes use the same GPU ID, it will out of CUDA mem.

[edwardnguyen1705]

Hi @Tau-J ,
Please check this PR: #44