Overview
PyTorch 1.10, released on October 22, 2021, adds CUDA Graphs support and improves type promotion for more reliable computations.
Main Features
CUDA Graphs
CUDA Graphs capture a sequence of GPU operations and replay them without CPU overhead, significantly reducing latency for inference models.
python
import torch
# CUDA Graph capture
# x = torch.randn(1000, device='cuda')
# graph = torch.cuda.CUDAGraph()
# Warmup
# with torch.cuda.graph(graph):
# y = x * 2 + 1
# Replay without CPU overhead
# graph.replay()
# CPU equivalent example
x = torch.randn(1000)
y = x * 2 + 1
print(f'Shape: {y.shape}, Mean: {y.mean():.2f}')Type promotion
Type promotion rules are aligned with NumPy for more predictable results when mixing types (int, float, complex) in tensor operations.
python
import torch
# Improved type promotion
a = torch.tensor([1, 2, 3], dtype=torch.int32)
b = torch.tensor([1.5, 2.5, 3.5], dtype=torch.float32)
# Result is promoted to float32
c = a + b
print(f'Type: {c.dtype}') # torch.float32
print(f'Values: {c}') # [2.5, 4.5, 6.5]
# Scalar promotion
d = a * 2.0
print(f'Type: {d.dtype}') # torch.float32