multi-gpu inference
Currently model inference only runs on single GPU.
Inference using multiple GPUs could be done by splitting the list of images in dataset_to_infer into subsets, distributing subsets of dataset_to_infer to multiple GPUs.
(In an image window based inference, for each image the tasks of inference on window can also be distributed to multiple GPUs, but it can be difficult to implement/maintain.)