What it is
Transformers is a Python library developed by Hugging Face that provides state-of-the-art pre-trained models for Natural Language Processing (NLP) tasks such as text classification, translation, summarization, question answering, and more.
Transformers provides pre-trained models that can be used directly for inference or fine-tuned on custom datasets. It supports PyTorch, TensorFlow, and JAX backends. The library also includes tokenizers, pipelines, and trainer APIs for streamlined workflows.
Installation
pip install transformersGetting started
The smallest useful thing you can do with it, and what each part means.
from transformers import pipeline
classifier = pipeline('sentiment-analysis')
result = classifier('I love using Hugging Face Transformers!')
print(result)from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
inputs = tokenizer('Hello, Hugging Face!', return_tensors='pt')
print(inputs)Advanced usage
Where the library earns its place over a simpler alternative.
from transformers import AutoModelForSequenceClassification, Trainer, TrainingArguments, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = AutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)
# Prepare dataset, tokenize, and create Trainer object
training_args = TrainingArguments(output_dir='./results', num_train_epochs=1, per_device_train_batch_size=8)
trainer = Trainer(model=model, args=training_args, train_dataset=train_dataset, eval_dataset=eval_dataset)
trainer.train()from transformers import pipeline
qa_pipeline = pipeline('question-answering')
context = 'Hugging Face is creating a Transformers library.'
question = 'Who is creating Transformers?'
result = qa_pipeline(question=question, context=context)
print(result)from transformers import pipeline
generator = pipeline('text-generation', model='gpt2')
result = generator('Once upon a time', max_length=50)
print(result)from transformers import pipeline
summarizer = pipeline('summarization')
text = 'Hugging Face Transformers provides thousands of pre-trained models to perform tasks on texts, images, and audio.'
summary = summarizer(text, max_length=50, min_length=25, do_sample=False)
print(summary)Errors and fixes
The failures you are most likely to hit, and what actually resolves them.
- OSError: Model name 'xyz' was not found
- Check the model name on Hugging Face Hub or ensure it is correctly spelled.
- RuntimeError: CUDA out of memory
- Reduce batch size or move computations to CPU if GPU memory is insufficient.
- ValueError: Shape mismatch
- Ensure input tokens match the expected input shape for the model.
Best practices
- Use pipelines for quick inference without dealing with tokenization and model objects directly.
- Fine-tune pre-trained models on custom datasets for better performance on specific tasks.
- Leverage GPU acceleration for large models and batch inference.
- Use the `AutoModel` and `AutoTokenizer` classes to easily switch between architectures.
- Keep track of model versions to ensure reproducibility.
Background
Why it exists, and what it was reacting to.
Hugging Face released Transformers in 2018 to simplify the use of transformer-based models like BERT, GPT, RoBERTa, and T5. The library provides easy access to pre-trained models and tokenizers, enabling developers and researchers to leverage powerful NLP models without extensive training or setup.