Skip to content

Supported Models

All supported models are hosted under the inference4j HuggingFace organization and are automatically downloaded and cached on first use.

NLP

Capability Wrapper Default Model ID Size API
Text Classification DistilBertTextClassifier inference4j/distilbert-base-uncased-finetuned-sst-2-english ~260 MB TextClassifier
Prompt-Injection Detection DebertaPromptInjectionDetector inference4j/deberta-v3-base-prompt-injection-v2 ~740 MB PromptInjectionDetector, TextClassifier
Zero-Shot Text Classification DebertaZeroShotClassifier.base() inference4j/deberta-v3-base-zeroshot-v2.0-c ~740 MB ZeroShotTextClassifier
Zero-Shot Text Classification DebertaZeroShotClassifier.large() inference4j/deberta-v3-large-zeroshot-v2.0-c ~1.7 GB ZeroShotTextClassifier
Named Entity Recognition BertNerRecognizer inference4j/distilbert-NER ~260 MB NamedEntityRecognizer
Named Entity Recognition BertNerRecognizer inference4j/bert-base-NER ~431 MB NamedEntityRecognizer
Zero-Shot NER GlinerEntityRecognizer.small() inference4j/gliner-small-v2.1 ~610 MB ZeroShotEntityRecognizer, NamedEntityRecognizer
Zero-Shot NER (PII) GlinerEntityRecognizer.pii() inference4j/gliner-pii-base-v1.0 ~665 MB ZeroShotEntityRecognizer, NamedEntityRecognizer
Text Embeddings SentenceTransformerEmbedder inference4j/all-MiniLM-L6-v2 ~90 MB TextEmbedder
Punctuation & Casing PunctCapSegModel inference4j/punctuation-fullstop-truecase-english ~210 MB InferenceTask<String, List<String>>
Text Embeddings SentenceTransformerEmbedder inference4j/all-MiniLM-L12-v2 ~130 MB TextEmbedder
Text Embeddings SentenceTransformerEmbedder inference4j/all-mpnet-base-v2 ~430 MB TextEmbedder
Text Embeddings SentenceTransformerEmbedder inference4j/bge-base-en-v1.5 ~430 MB TextEmbedder
Text Embeddings SentenceTransformerEmbedder inference4j/gte-base ~430 MB TextEmbedder
Text Embeddings SentenceTransformerEmbedder inference4j/nomic-embed-text-v1.5 ~520 MB TextEmbedder
Search Reranking MiniLMSearchReranker inference4j/ms-marco-MiniLM-L-6-v2 ~90 MB SearchReranker
Text Generation OnnxTextGenerator.gpt2() inference4j/gpt2 ~500 MB TextGenerator
Text Generation OnnxTextGenerator.smolLM2() inference4j/smollm2-360m-instruct ~700 MB TextGenerator
Text Generation OnnxTextGenerator.smolLM2_1_7B() inference4j/smollm2-1.7b-instruct ~3.4 GB TextGenerator
Text Generation OnnxTextGenerator.tinyLlama() inference4j/tinyllama-1.1b-chat ~2.2 GB TextGenerator
Text Generation OnnxTextGenerator.qwen2() inference4j/qwen2.5-1.5b-instruct ~3 GB TextGenerator
Text Generation OnnxTextGenerator.gemma2() Gated — requires manual download ~5 GB TextGenerator
Summarization / Translation / Grammar FlanT5TextGenerator.flanT5Small() inference4j/flan-t5-small ~300 MB TextGenerator, Summarizer, Translator, GrammarCorrector
Summarization / Translation / Grammar FlanT5TextGenerator.flanT5Base() inference4j/flan-t5-base ~900 MB TextGenerator, Summarizer, Translator, GrammarCorrector
Summarization / Translation / Grammar FlanT5TextGenerator.flanT5Large() inference4j/flan-t5-large ~3 GB TextGenerator, Summarizer, Translator, GrammarCorrector
Text-to-SQL T5SqlGenerator.t5SmallAwesome() inference4j/t5-small-awesome-text-to-sql ~240 MB TextGenerator, SqlGenerator
Text-to-SQL T5SqlGenerator.t5LargeSpider() inference4j/T5-LM-Large-text2sql-spider ~4.6 GB TextGenerator, SqlGenerator
Summarization BartSummarizer.distilBartCnn() inference4j/distilbart-cnn-12-6 ~1.2 GB TextGenerator, Summarizer
Summarization BartSummarizer.bartLargeCnn() inference4j/bart-large-cnn ~1.6 GB TextGenerator, Summarizer
Translation MarianTranslator.builder() Required .modelId(...): inference4j/opus-mt-en-fr, opus-mt-en-de or opus-mt-en-es ~650 MB each TextGenerator, Translator
Grammar Correction CoeditGrammarCorrector.coeditBase() inference4j/coedit-base ~900 MB TextGenerator, GrammarCorrector
Grammar Correction CoeditGrammarCorrector.coeditLarge() inference4j/coedit-large ~3 GB TextGenerator, GrammarCorrector

Vision

Capability Wrapper Default Model ID Size API
Image Classification ResNetClassifier inference4j/resnet50-v1-7 ~100 MB ImageClassifier
Image Classification EfficientNetClassifier inference4j/efficientnet-lite4 ~50 MB ImageClassifier
Object Detection YoloV8Detector inference4j/yolov8n ~25 MB ObjectDetector
Object Detection Yolo26Detector inference4j/yolo26n ~25 MB ObjectDetector
Object Detection (Apache-2.0) RtDetrDetector.dfineSmall() inference4j/dfine-small-coco ~41 MB ObjectDetector
Object Detection (Apache-2.0) RtDetrDetector.dfineNano() inference4j/dfine-nano-coco ~15 MB ObjectDetector
Object Detection (Apache-2.0) RtDetrDetector.rtDetrV2() inference4j/rtdetr-v2-r18vd ~81 MB ObjectDetector
Text Detection CraftTextDetector inference4j/craft-mlt-25k ~80 MB TextDetector
Text Detection (lines) PaddleTextDetector.small() inference4j/pp-ocrv6-small-det ~10 MB TextDetector
Text Detection (lines) PaddleTextDetector.tiny() inference4j/pp-ocrv6-tiny-det ~2 MB TextDetector
Text Recognition PaddleTextRecognizer.small() inference4j/pp-ocrv6-small-rec ~21 MB TextRecognizer
Text Recognition PaddleTextRecognizer.tiny() inference4j/pp-ocrv6-tiny-rec ~4.5 MB TextRecognizer
OCR OcrPipeline Paddle detector + recognizer (small by default) ~31 MB InferenceTask<BufferedImage, List<TextLine>>
Depth Estimation DepthAnythingEstimator inference4j/depth-anything-v2-small ~95 MB DepthEstimator
Promptable Segmentation Sam2Segmenter inference4j/sam2.1-hiera-tiny ~155 MB PromptableSegmenter

Multimodal

Capability Wrapper Default Model ID Size API
Zero-Shot Classification ClipClassifier inference4j/clip-vit-base-patch32 ~595 MB ZeroShotClassifier
Image Embeddings ClipImageEncoder inference4j/clip-vit-base-patch32 ~340 MB ImageEmbedder
Text Embeddings (CLIP) ClipTextEncoder inference4j/clip-vit-base-patch32 ~255 MB TextEmbedder

Audio

Capability Wrapper Default Model ID Size API
Speech-to-Text MoonshineRecognizer inference4j/moonshine-streaming-small-en (default) ~140 MB SpeechRecognizer
Speech-to-Text MoonshineRecognizer inference4j/moonshine-streaming-tiny-en ~46 MB SpeechRecognizer
Speech-to-Text MoonshineRecognizer inference4j/moonshine-streaming-medium-en ~260 MB SpeechRecognizer
Speech-to-Text Wav2Vec2Recognizer inference4j/wav2vec2-base-960h ~370 MB SpeechRecognizer
Voice Activity Detection SileroVadDetector inference4j/silero-vad ~2 MB VoiceActivityDetector

Generative AI

Generative models use a separate module (inference4j-genai) and a different builder pattern. See Generative AI for details.

Capability Wrapper Model ID Size License
Text Generation TextGenerator inference4j/phi-3-mini-4k-instruct ~2.7 GB MIT
Text Generation TextGenerator inference4j/deepseek-r1-distill-qwen-1.5b ~1 GB MIT
Text Generation TextGenerator inference4j/qwen3-0.6b ~0.4 GB Apache-2.0
Text Generation TextGenerator inference4j/smollm3-3b ~2 GB Apache-2.0
Text Generation TextGenerator inference4j/phi-4-mini-instruct ~4.9 GB MIT
Speech-to-Text / Translation WhisperSpeechModel inference4j/whisper-tiny-genai ~225 MB MIT
Speech-to-Text / Translation WhisperSpeechModel inference4j/whisper-small-genai ~1.1 GB MIT
Vision-Language VisionLanguageModel inference4j/phi-3.5-vision-instruct ~3.3 GB MIT

Model reference

A comprehensive view of all supported models, organized by architecture:

Encoder-only (single-pass)

Model Tokenizer Wrapper Use Cases
DistilBERT SST-2 WordPiece DistilBertTextClassifier Sentiment analysis, text classification
DistilBERT NER WordPiece (cased) BertNerRecognizer Named entity recognition
BERT Base NER WordPiece (cased) BertNerRecognizer Named entity recognition
all-MiniLM-L6-v2 WordPiece SentenceTransformerEmbedder Semantic search, embeddings
all-MiniLM-L12-v2 WordPiece SentenceTransformerEmbedder Semantic search, embeddings
all-mpnet-base-v2 WordPiece SentenceTransformerEmbedder Semantic search, embeddings
BGE Base EN v1.5 WordPiece SentenceTransformerEmbedder Semantic search, embeddings
GTE Base WordPiece SentenceTransformerEmbedder Semantic search, embeddings
MiniLM-L-6 MS MARCO WordPiece MiniLMSearchReranker Search reranking
Punct-Cap-Seg English SentencePiece Unigram PunctCapSegModel Punctuation, true-casing, sentence segmentation
DeBERTa-v3 Base prompt-injection v2 SentencePiece Unigram (tokenizer.json) DebertaPromptInjectionDetector Prompt-injection detection
DeBERTa-v3 zero-shot v2.0-c (Base / Large) SentencePiece Unigram (tokenizer.json) DebertaZeroShotClassifier Zero-shot text classification (NLI)
GLiNER small v2.1, GLiNER PII base v1.0 SentencePiece Unigram (tokenizer.json) GlinerEntityRecognizer Zero-shot NER, PII detection

Decoder-only (autoregressive)

Model Tokenizer Wrapper Use Cases
GPT-2 BPE OnnxTextGenerator Text completion
SmolLM2-360M BPE OnnxTextGenerator Chat, instruction following
TinyLlama-1.1B SentencePiece BPE OnnxTextGenerator Chat, instruction following
Qwen2.5-1.5B BPE OnnxTextGenerator Chat, instruction following
Gemma 2-2B SentencePiece BPE OnnxTextGenerator Chat, instruction following

Encoder-decoder (autoregressive)

Model Tokenizer Wrapper Use Cases
Flan-T5 (Small / Base / Large) SentencePiece Unigram FlanT5TextGenerator Summarization, translation, grammar
T5-small-awesome-text-to-sql SentencePiece Unigram T5SqlGenerator Text-to-SQL (lightweight)
T5-LM-Large-text2sql-spider SentencePiece Unigram T5SqlGenerator Text-to-SQL (high accuracy)
DistilBART CNN 12-6 BPE BartSummarizer Summarization
BART Large CNN BPE BartSummarizer Summarization
MarianMT (opus-mt-*) SentencePiece BPE MarianTranslator Translation (fixed language pair)
CoEdIT (Base / Large) SentencePiece Unigram CoeditGrammarCorrector Grammar correction

Vision

Model Tokenizer Wrapper Use Cases
ResNet-50 N/A ResNetClassifier Image classification
EfficientNet-Lite4 N/A EfficientNetClassifier Image classification
YOLOv8n N/A YoloV8Detector Object detection
YOLO26n N/A Yolo26Detector Object detection
D-FINE (nano / small, COCO) N/A RtDetrDetector Object detection (Apache-2.0)
RT-DETRv2 ResNet-18 N/A RtDetrDetector Object detection (Apache-2.0)
CRAFT N/A CraftTextDetector Text detection in images
PP-OCRv6 (small / tiny) detection N/A PaddleTextDetector Text line detection
PP-OCRv6 (small / tiny) recognition Character dictionary (inference.yml) PaddleTextRecognizer Text recognition (OCR)
Depth Anything V2 Small N/A DepthAnythingEstimator Monocular depth estimation
SAM 2.1 Hiera Tiny N/A Sam2Segmenter Promptable segmentation (click or box → mask)

Multimodal

Model Tokenizer Wrapper Use Cases
CLIP ViT-B/32 BPE ClipClassifier Zero-shot image classification

Audio

Model Tokenizer Wrapper Use Cases
Moonshine v2 (tiny, small, medium; English) SentencePiece BPE MoonshineRecognizer Speech-to-text
Wav2Vec2 CTC Wav2Vec2Recognizer Speech-to-text
Silero VAD N/A SileroVadDetector Voice activity detection

Model compatibility

YOLOv8 / YOLO11

YoloV8Detector is compatible with both YOLOv8 and YOLO11 models — they share the same output layout ([1, 4+C, N]). It is not compatible with YOLOv5 (different layout with objectness column) or YOLO26 (NMS-free architecture).

EfficientNet variants

EfficientNetClassifier is tested against EfficientNet-Lite4 (TensorFlow origin, softmax built-in). For PyTorch-exported EfficientNet models that output raw logits, override with .outputOperator(OutputOperator.softmax()).

Custom models

Any ONNX-exported model works with the appropriate wrapper, as long as it follows the expected input/output layout. See the Custom Models guide for details.

Cache

Models are cached in ~/.cache/inference4j/ by default. Customize with:

  • System property: -Dinference4j.cache.dir=/path/to/cache
  • Environment variable: INFERENCE4J_CACHE_DIR=/path/to/cache

See Configuration for all options.

Planned models

Domain Model Status
Text TrOCR (handwriting recognition) Planned (#128)

See the Roadmap for details.