HF Transformers 4.42: Llama 3.1

Transformers 4.42 adds Llama 3.1 support with extended 128k token context.

Overview

Transformers 4.42, released on July 15, 2024, adds support for Llama 3.1 and its multilingual variants.

Main Features

Llama 3.1 support

Llama 3.1 models (8B, 70B, 405B) are integrated with extended context support (128k tokens) and improved multilingual capabilities.

python

from transformers import pipeline

generator = pipeline(
    'text-generation',
    model='meta-llama/Llama-3.1-8B-Instruct',
)
result = generator(
    'Explain the Pythagorean theorem simply.',
    max_new_tokens=100,
)
print(result[0]['generated_text'])

Sources

Transformers 4.42.0 Release