Does it support multi-GPU inference?
#12
by
micczzz
- opened
When I use the model.generate inference method in huggingface, it prompts：RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! (when checking argument for argument index in method wrapper_CUDA__index_select)