I would like to know if pytorch is using my GPU. It’s possible to detect with nvidia-smi if there is any activity from the GPU during the process, but I want something written in a python script.
Is there a way to accomplish this?
Asked by vinzee
This should work:
import torch torch.cuda.is_available() >>> True torch.cuda.current_device() >>> 0 torch.cuda.device(0) >>> <torch.cuda.device at 0x7efce0b03be0> torch.cuda.device_count() >>> 1 torch.cuda.get_device_name(0) >>> 'GeForce GTX 950M'
This tells me that CUDA is accessible and that one of your machines can utilize it (GPUs). PyTorch is now using Device 0, which is the GPU GeForce GTX 950M.
Answered by vinzee
I’m adding a method using torch.device because it hasn’t been proposed yet, and it’s pretty useful, especially when initializing tensors on the proper device.
# setting device on GPU if available, else CPU device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') print('Using device:', device) print() #Additional Info when using cuda if device.type == 'cuda': print(torch.cuda.get_device_name(0)) print('Memory Usage:') print('Allocated:', round(torch.cuda.memory_allocated(0)/1024**3,1), 'GB') print('Cached: ', round(torch.cuda.memory_reserved(0)/1024**3,1), 'GB')
versions from the past
Using device: cuda Tesla K80 Memory Usage: Allocated: 0.3 GB Cached: 0.6 GB
As previously stated, the device can be used to:
This allows for easy switching between CPU and GPU without having to change the code.
I’m providing some additional information on cached and allocated memory because there have been some questions and confusion about it:
You can either pass over a device as described further above in the post, or you can leave it blank and the current device will be used ().
Additional information: Old graphics cards with a Cuda compute capacity of 3.0 or lower may be seen, but Pytorch cannot use them! Hekimgil was the one who pointed this out! – “GPU0 is a GeForce GT 750M with cuda 3.0 functionality. This GPU is no longer supported by PyTorch since it is too old. We support a minimum cuda capability of 3.5.”
Answered by MBT
If you want to manually check whether your application is utilizing GPU resources and to what extent after you start the training loop, simply type watch in the terminal as in:
$ watch -n 2 nvidia-smi
Until you press ctrl+c, the usage statistics will be updated every 2 seconds.
You can use the more complex version of nvidia-smi with —query-gpu=…. if you require more control over more GPU information. Here’s a simple illustration of what I’m talking about:
$ watch -n 3 nvidia-smi --query-gpu=index,gpu_name,memory.total,memory.used,memory.free,temperature.gpu,pstate,utilization.gpu,utilization.memory --format=csv
It would produce the following statistics:
—query-gpu= query names …. Those settings will be ignored if not specified, and no statistics will be returned.
You may also verify if PyTorch correctly finds your CUDA installation by running the following command:
In : import torch In : torch.cuda.is_available() Out: True
True status means that PyTorch is configured correctly and is using the GPU although you have to move/place the tensors with necessary statements in your code.
Look into this module if you want to perform this in Python code:
https://github.com/jonsafari/nvidia-ml-py alternatively download it from pypi: https://pypi.python.org/pypi/nvidia-ml-py/
Answered by kmario23
Just one little detour from a practical standpoint:
import torch dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
This developer now knows whether to use cuda or CPU.
When going to cuda, there is a distinction between dealing with models and tensors. At first, it seems weird.
import torch import torch.nn as nn dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu") t1 = torch.randn(1,2) t2 = torch.randn(1,2).to(dev) print(t1) # tensor([[-0.2678, 1.9252]]) print(t2) # tensor([[ 0.5117, -3.6247]], device='cuda:0') t1.to(dev) print(t1) # tensor([[-0.2678, 1.9252]]) print(t1.is_cuda) # False t1 = t1.to(dev) print(t1) # tensor([[-0.2678, 1.9252]], device='cuda:0') print(t1.is_cuda) # True class M(nn.Module): def __init__(self): super().__init__() self.l1 = nn.Linear(1,2) def forward(self, x): x = self.l1(x) return x model = M() # not on cuda model.to(dev) # is on cuda (all parameters) print(next(model.parameters()).is_cuda) # True
This is all complicated, and if you grasp it, you’ll be able to deal with it quickly and with less troubleshooting.
Answered by prosti
Check GPU for PyTorch on the office site and the get started page as follows:
import torch torch.cuda.is_available()
Reference: PyTorch|Get Start
Answered by TimeSeam
Post is based on https://stackoverflow.com/questions/48152674/how-to-check-if-pytorch-is-using-the-gpu