Overview

Hugging Face Transformers 4.20, released on June 20, 2022, introduces GPTQ quantization to reduce language model size.

Main Features

GPTQ quantization

GPTQ quantizes language models to 4 or 8 bits with minimal quality loss, significantly reducing memory footprint.

python 
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = 'gpt2'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Text generation
inputs = tokenizer('Hello, I am', return_tensors='pt')
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))

New models

New pre-trained models are added, including architectures optimized for text generation and classification.

python 
from transformers import pipeline

# Classification pipeline
classifier = pipeline('sentiment-analysis')
result = classifier('This library is amazing!')
print(result)  # [{'label': 'POSITIVE', 'score': 0.99}]

# Summarization pipeline
summarizer = pipeline('summarization')
text = 'Long article text here...'
# summary = summarizer(text, max_length=50)

Sources