Skip to content

Zero-Shot Text Classification

Classify text into labels you choose at call time, with no training data and no fine-tuning: support tickets into queues, articles into topics, feedback into themes. Change the labels and the classifier follows.

Quick example

try (var classifier = DebertaZeroShotClassifier.builder().build()) {
    List<TextClassification> result = classifier.classify(
        "My invoice shows the same charge twice.",
        List.of("billing", "outage", "feature request", "account access"));
    // billing 0.97, account access 0.02, outage 0.01, feature request 0.00
}

Full example

import io.github.inference4j.nlp.DebertaZeroShotClassifier;
import io.github.inference4j.nlp.TextClassification;
import io.github.inference4j.nlp.ZeroShotTextClassifier;
import java.util.List;

public class TicketRouter {
    public static void main(String[] args) {
        List<String> queues = List.of("billing", "outage", "feature request", "account access");

        try (ZeroShotTextClassifier classifier = DebertaZeroShotClassifier.builder()
                .hypothesisTemplate("The customer is asking about {}.")
                .build()) {
            for (String ticket : List.of(
                    "The service has been unavailable since this morning and nothing loads.",
                    "Please add a dark mode to the app.")) {
                TextClassification top = classifier.classify(ticket, queues).get(0);
                System.out.printf("%-75s -> %s (%.0f%%)%n", ticket, top.label(), top.confidence() * 100);
            }
        }
    }
}

How it works

The classifier is a natural language inference (NLI) model: given a text and a statement, it judges whether the text entails the statement. Each candidate label becomes a statement through the hypothesis template — "This text is about billing." — and the label's score is how strongly the text entails it. This is how HuggingFace's zero-shot-classification pipeline works, and inference4j's scores match it.

All of a text's labels are scored in one batch, so one call costs one model run per text, whatever the number of labels (up to the batch's memory).

Single-label and multi-label

Mode Scores Use it when
multiLabel(false) (default) A softmax over the labels: they compete and sum to 1 Exactly one label applies (routing, topic)
multiLabel(true) Each label judged on its own (entailment against contradiction), 0 to 1 Several labels, or none, may apply (tagging)
try (var tagger = DebertaZeroShotClassifier.builder().multiLabel(true).build()) {
    tagger.classify("I can't log in after resetting my password, and I was charged for a plan I cancelled.",
        List.of("billing", "account access", "outage"));
    // account access 0.97, billing 0.27, outage 0.03
}

With a single candidate label, the score is always computed the multi-label way: a label cannot compete with itself.

Writing labels and templates

  • Labels are read as plain language. "feature request" works better than "FR" or "feat_req".
  • The template matters. The default, "This text is about {}.", suits topics. A template written for your texts, such as "The customer is asking about {}." or "The sentiment of this review is {}.", often scores better. {} stands for the label.
  • Validate on your own data. Zero-shot accuracy varies a lot between tasks. Try a few templates and label wordings on a sample of labeled texts before relying on it.

Models

Preset Model Size AG News accuracy* Time per text*
base() (default) inference4j/deberta-v3-base-zeroshot-v2.0-c ~740 MB 64.0% 47 ms
large() inference4j/deberta-v3-large-zeroshot-v2.0-c ~1.7 GB 78.7% 150 ms

*400 texts from the AG News test split, four topics (World, Sports, Business, Science and Technology), default template, single-label; laptop CPU, four ONNX Runtime threads. On the author's broader benchmark of 28 classification tasks, the models reach a mean F1 of 0.619 (base) and 0.676 (large), against 0.497 for facebook/bart-large-mnli.

Both are Moritz Laurer's zero-shot v2.0 "-c" models: MIT-licensed and trained only on commercially-friendly data (synthetic data plus the MNLI and FEVER-NLI datasets).

Builder options

Method Type Default Description
.modelId(String) String inference4j/deberta-v3-base-zeroshot-v2.0-c HuggingFace model ID; presets base(), large()
.modelSource(ModelSource) ModelSource HuggingFaceModelSource Model resolution strategy
.sessionOptions(SessionConfigurer) SessionConfigurer default ONNX Runtime session config
.hypothesisTemplate(String) String "This text is about {}." How a label becomes a hypothesis; {} is the label
.multiLabel(boolean) boolean false Score labels independently instead of letting them compete
.maxLength(int) int 512 Maximum tokens per model run, text, hypothesis and special tokens included
.stride(int) int off Score long text in full, in windows sharing this many tokens
.windowAggregation(WindowAggregation) WindowAggregation mean() single-label, max() multi-label How window scores combine. Requires .stride(int)

Long text

By default, text that does not fit in maxLength tokens together with a hypothesis is cut at its end; the hypothesis is always kept whole, as in HuggingFace. With .stride(int), the whole text is scored in overlapping windows and the window scores combined: by default mean() for single-label (an overall verdict) and max() for multi-label (a label that applies anywhere in the text applies to the text).

try (var classifier = DebertaZeroShotClassifier.builder().stride(128).build()) {
    classifier.classify(longReport, List.of("finance", "legal", "engineering"));
}

Result type

TextClassification is a record with:

Field Type Description
label() String The candidate label
index() int The label's position in the list passed in
confidence() float The label's score (0.0 to 1.0)

Results are sorted by confidence, highest first.

Accuracy against HuggingFace

On the same ONNX model, inference4j's scores equal those of HuggingFace's tokenizer and post-processing to within 4 × 10⁻⁷. Compared with the zero-shot-classification pipeline running the original PyTorch weights, scores differ by up to 3 × 10⁻⁴ (more on texts near 512 tokens), the ordinary difference between ONNX and PyTorch arithmetic; the top label agreed on all 400 AG News texts and all test cases, and accuracy was identical.