Fine-Tune Hugging Face Models for Text Classification

Training text classifiers has traditionally required a lot of setup, coding, and hyperparameter tuning. But thanks to Langformers, you can now fine-tune powerful Transformer models (encoder-only models / masked lanuage models) like BERT, RoBERTa, or MPNet for your custom classification tasks with just a few lines of code — all while keeping full flexibility and control.

This guide will walk you through how to quickly set up and train a text classifier using Langformers, customize training settings, and use your trained model for predictions (inference).

Setting Up

First, make sure you have Langformers installed in your environment. If not, install it using pip:

pip install -U langformers

Now, let's dive into the code!

# Import langformers
from langformers import tasks

Define Training Configuration

Before training, you can specify how you want the model to be fine-tuned. This includes parameters like maximum input length, number of epochs, logging frequency, and early stopping patience.

Here’s an example configuration:

# Define training configuration
training_config = {
    "max_length": 80,
    "num_train_epochs": 1,
    "report_to": ['tensorboard'],
    "logging_steps": 20,
    "save_steps": 20,
    "early_stopping_patience": 5,
    # ...
}

The configuration is fully customizable, so you can tweak it based on your needs or even override defaults for advanced training setups.

Initialize the Model

Now, it's time to initialize your model using Langformers' create_classifier method. You’ll provide the model name (such as "roberta-base"), the path to your training dataset (in CSV format), and the column names for the input texts and their labels.

Example:

# Initialize the model
model = tasks.create_classifier(
    model_name="roberta-base",          # Hugging Face model name or local path
    csv_path="/path/to/dataset.csv",    # Path to your CSV file
    text_column="text",                 # Name of the text column
    label_column="label",               # Name of the label column
    training_config=training_config     # Pass the training configuration
)

Behind the scenes, Langformers takes care of:

• Loading the pretrained model
• Processing the dataset
• Detecting the number of classes automatically
• Preparing the model for fine-tuning

Important Note: Langformers expects the labels in your dataset to be human-readable strings (like "positive", "neutral", "negative") rather than numeric codes (like 0, 1, 2). This makes your model more intuitive and easier to use during inference.

Train the Classifier

Once everything is set up, fine-tuning your model is as simple as calling train():

# Start fine-tuning
model.train()

During training, the model will log progress, save checkpoints, and use early stopping as specified. At the end of training, the best-performing model along with its configurations is automatically saved to disk.

Load and Use the Classifier

After training, you can easily load your saved classifier using load_classifier and start making predictions:

# Import langformers
from langformers import tasks

# Load the trained classifier
classifier = tasks.load_classifier("/path/to/classifier")

# Classify new texts
results = classifier.classify([
    "I don't like this movie. Worst ever.",
    "I loved this movie!"
])

print(results)

Langformers will output the predicted class for each input text.

Happy fine-tuning and inferencing! 🚀

View official documentation here: https://langformers.com/train-text-classifiers.html