Blackwell GPU Support

PyTorch 2.7 adds NVIDIA Blackwell GPU (B100/B200) support via CUDA 12.8. The Blackwell architecture brings FP4, higher memory bandwidth, and 5th-gen Tensor Cores, directly benefiting large model training and inference.

Benchmark Example

python 
import torch

# Check Blackwell support
print(torch.cuda.get_device_name(0))  # NVIDIA B200
print(torch.cuda.get_device_capability(0))  # (10, 0)

# Simple benchmark
x = torch.randn(8192, 8192, device='cuda', dtype=torch.bfloat16)
y = torch.randn(8192, 8192, device='cuda', dtype=torch.bfloat16)

# Warm-up
for _ in range(10):
    _ = x @ y
torch.cuda.synchronize()

# Measure
import time
start = time.perf_counter()
for _ in range(100):
    _ = x @ y
torch.cuda.synchronize()
elapsed = time.perf_counter() - start
print(f'100 matmuls 8192x8192 BF16: {elapsed:.2f}s')

Sources