Overview

Hugging Face Transformers 4.25, released on December 15, 2022, integrates FlashAttention to accelerate Transformer training and inference.

Main Features

FlashAttention

FlashAttention optimizes attention computation by reducing memory accesses, offering significant speedups on long sequences.

python 
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    'gpt2',
    attn_implementation='flash_attention_2',  # if available
)
tokenizer = AutoTokenizer.from_pretrained('gpt2')

inputs = tokenizer('Hello', return_tensors='pt')
outputs = model.generate(**inputs, max_length=30)
print(tokenizer.decode(outputs[0]))

New models

New architectures are supported, including multimodal vision and language models.

python 
from transformers import pipeline

# Image classification
classifier = pipeline('image-classification')
# result = classifier('photo.jpg')

# Question answering
qa = pipeline('question-answering')
result = qa(
    question='What is the framework?',
    context='Transformers is a framework by Hugging Face.',
)
print(result['answer'])

Sources