Skip to content

Spring Boot

inference4j provides a Spring Boot starter with opt-in auto-configuration for all supported models.

Requirements

Spring Boot 4.0 or later
Spring Framework 7.0 or later
Java 17 or later

Spring Boot 4 required as of inference4j 0.11.0

The starter targets Spring Boot 4. It will not load in a Spring Boot 3 application — Boot 4 relocated the actuator health types, so the two lines are binary-incompatible and cannot be served by one artifact.

If you are still on Spring Boot 3, stay on inference4j-spring-boot-starter:0.10.1, which is the last release built against Boot 3.4. It is frozen, not maintained — Spring Boot 3.x itself left open-source support on 30 June 2026, so migrating to Boot 4 is the recommended path. See Migrating from 0.10.x below.

Only the starter is affected. inference4j-core and every other module are plain Java libraries with no Spring dependency, and work unchanged on any Spring version or none at all.

Setup

Add the starter dependency:

implementation 'io.github.inference4j:inference4j-spring-boot-starter:${inference4jVersion}'
<dependency>
    <groupId>io.github.inference4j</groupId>
    <artifactId>inference4j-spring-boot-starter</artifactId>
    <version>${inference4jVersion}</version>
</dependency>

Configuration

Every model is opt-in — nothing is downloaded until you set enabled: true.

inference4j:
  nlp:
    text-classifier:
      enabled: true

All properties

Property Type Default Description
inference4j.metrics.enabled boolean true Use a Micrometer-backed RouterMetrics when a MeterRegistry is present. When false, or when Micrometer is absent, RouterMetrics falls back to a no-op instead of being removed
inference4j.nlp.text-classifier.enabled boolean false Enable DistilBERT text classifier
inference4j.nlp.text-classifier.model-id String inference4j/distilbert-base-uncased-finetuned-sst-2-english Model ID
inference4j.nlp.text-embedder.enabled boolean false Enable SentenceTransformer embedder
inference4j.nlp.text-embedder.model-id String (required) Model ID (no default — must be specified)
inference4j.nlp.search-reranker.enabled boolean false Enable MiniLM cross-encoder reranker
inference4j.nlp.search-reranker.model-id String inference4j/ms-marco-MiniLM-L-6-v2 Model ID
inference4j.nlp.named-entity-recognizer.enabled boolean false Enable BERT NER recognizer
inference4j.nlp.named-entity-recognizer.model-id String inference4j/distilbert-NER Model ID
inference4j.vision.image-classifier.enabled boolean false Enable ResNet image classifier
inference4j.vision.image-classifier.model-id String inference4j/resnet50-v1-7 Model ID
inference4j.vision.object-detector.enabled boolean false Enable YOLOv8 object detector
inference4j.vision.object-detector.model-id String inference4j/yolov8n Model ID
inference4j.vision.text-detector.enabled boolean false Enable CRAFT text detector
inference4j.vision.text-detector.model-id String inference4j/craft-mlt-25k Model ID
inference4j.audio.speech-recognizer.enabled boolean false Enable Wav2Vec2 speech recognizer
inference4j.audio.speech-recognizer.model-id String inference4j/wav2vec2-base-960h Model ID
inference4j.audio.vad.enabled boolean false Enable Silero VAD
inference4j.audio.vad.model-id String inference4j/silero-vad Model ID
inference4j.genai.text-generator.enabled boolean false Enable the onnxruntime-genai TextGenerator (requires inference4j-genai)
inference4j.genai.text-generator.model String phi-3-mini Preset: phi-3-mini, phi-4-mini, deepseek-r1-1.5b, qwen3-0.6b or smollm3-3b
inference4j.genai.text-generator.max-length / temperature / top-k / top-p number builder defaults Generation settings; unset keeps the builder's default
inference4j.genai.vision-language.enabled boolean false Enable VisionLanguageModel (requires inference4j-genai)
inference4j.genai.vision-language.model String phi-3.5-vision Preset: phi-3.5-vision
inference4j.genai.vision-language.max-length / temperature / top-k / top-p number builder defaults Generation settings
inference4j.genai.whisper.enabled boolean false Enable WhisperSpeechModel; it also becomes the SpeechRecognizer bean (requires inference4j-genai)
inference4j.genai.whisper.model-id String inference4j/whisper-small-genai Model ID (inference4j/whisper-tiny-genai for the small, fast one)
inference4j.genai.whisper.language String en Source language code
inference4j.genai.whisper.task String transcribe transcribe or translate (to English)
inference4j.genai.whisper.max-length int 448 Maximum tokens per 30-second chunk

Usage

Beans are registered by their interface type, so you inject the interface — not the concrete implementation:

@RestController
public class SentimentController {
    private final TextClassifier classifier;

    public SentimentController(TextClassifier classifier) {
        this.classifier = classifier;
    }

    @PostMapping("/analyze")
    public List<TextClassification> analyze(@RequestBody String text) {
        return classifier.classify(text);
    }
}

Generative models

Add inference4j-genai next to the starter, and the text generator, vision-language model and Whisper become available with the same opt-in properties:

inference4j:
  genai:
    text-generator:
      enabled: true
      model: phi-3-mini
      max-length: 512
    whisper:
      enabled: true
      model-id: inference4j/whisper-tiny-genai
public AssistantController(TextGenerator generator, SpeechRecognizer recognizer) { … }

Without inference4j-genai on the classpath, the inference4j.genai.* properties are ignored.

WhisperSpeechModel is a SpeechRecognizer, so with inference4j.genai.whisper.enabled=true it replaces Wav2Vec2 as the application's SpeechRecognizer bean, even if inference4j.audio.speech-recognizer.enabled is also set. Inject WhisperSpeechModel directly for Whisper-only options. For recordings longer than 30 seconds, wrap it yourself in a SegmentedSpeechRecognizer bean.

Presets cover the models with known chat templates. For any other genai model, define your own TextGenerator bean with modelSource and chatTemplate; it replaces the auto-configured one.

Overriding beans

All auto-configured beans use @ConditionalOnMissingBean, so you can replace any model with your own implementation:

@Configuration
public class CustomModelConfig {

    @Bean
    public TextClassifier textClassifier() {
        return DistilBertTextClassifier.builder()
            .modelId("your-org/your-model")
            .sessionOptions(opts -> opts.addCoreML())
            .build();
    }
}

When you define your own bean, the auto-configured one is skipped.

Health indicator

An actuator health indicator is included automatically when Spring Boot Actuator is on the classpath. It reports the number of registered InferenceTask and GenerativeTask beans (a bean that is both, like WhisperSpeechModel, counts once).

GET /actuator/health
{
  "status": "UP",
  "components": {
    "inference4j": {
      "status": "UP",
      "details": {
        "tasks": 3
      }
    }
  }
}

The health indicator is enabled by default. Disable it with:

inference4j:
  health:
    enabled: false

Example configuration

A typical production setup enabling sentiment analysis and semantic search:

inference4j:
  nlp:
    text-classifier:
      enabled: true
    text-embedder:
      enabled: true
      model-id: inference4j/all-MiniLM-L6-v2
    search-reranker:
      enabled: true
@Service
public class SearchService {
    private final TextEmbedder embedder;
    private final SearchReranker reranker;

    public SearchService(TextEmbedder embedder, SearchReranker reranker) {
        this.embedder = embedder;
        this.reranker = reranker;
    }

    public List<SearchResult> search(String query, List<String> candidates) {
        // Stage 1: embed and retrieve
        float[] queryEmb = embedder.encode(query);
        // ... cosine similarity ranking ...

        // Stage 2: rerank top candidates
        float[] scores = reranker.scoreBatch(query, topCandidates);
        // ... sort by score ...
        return results;
    }
}

Lazy loading

All inference4j beans are lazy by default — models are downloaded and ONNX sessions are created on first use, not at application startup. This keeps startup fast even when multiple models are enabled.

The trade-off is that the first request to each model may be slower due to the model download and session initialization.

Warming up specific models

If you need a model ready immediately (e.g., for latency-sensitive endpoints), inject it eagerly at startup with an ApplicationReadyEvent listener:

@Component
public class ModelWarmup {
    private final TextClassifier classifier;

    public ModelWarmup(TextClassifier classifier) {
        this.classifier = classifier;
    }

    @EventListener(ApplicationReadyEvent.class)
    public void warmup() {
        classifier.classify("warmup");
    }
}

This triggers the lazy bean initialization during startup, so the model is ready when the first real request arrives.

Migrating from 0.10.x

inference4j 0.11.0 moved the starter from Spring Boot 3 to Spring Boot 4. There is no Boot 3 branch — 0.10.1 is the last Boot 3 release and stays available on Maven Central.

1. Upgrade your application to Spring Boot 4. Spring's own migration guide recommends going to 3.5 first, then to 4.0. Boot 4 requires Java 17 or later.

2. Update the health indicator import, if you override it. The actuator health types moved modules in Boot 4. If you replace inference4j's default indicator with your own bean, change:

// Before (Spring Boot 3)
import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;

// After (Spring Boot 4)
import org.springframework.boot.health.contributor.Health;
import org.springframework.boot.health.contributor.HealthIndicator;

The interface itself is unchanged — HealthIndicator is still a @FunctionalInterface returning Health, so an existing lambda keeps working once the import is fixed.

3. Nothing else changes. Every inference4j.* property, bean name and default is identical to 0.10.x. If you do not override the health indicator, upgrading your Boot version is the only step.

If you cannot move to Spring Boot 4 yet

Pin the starter to the last Boot 3 release and let it float independently of the rest:

implementation 'io.github.inference4j:inference4j-spring-boot-starter:0.10.1'
implementation 'io.github.inference4j:inference4j-core:${inference4jVersion}'
<dependency>
    <groupId>io.github.inference4j</groupId>
    <artifactId>inference4j-spring-boot-starter</artifactId>
    <version>0.10.1</version>
</dependency>
<dependency>
    <groupId>io.github.inference4j</groupId>
    <artifactId>inference4j-core</artifactId>
    <version>${inference4jVersion}</version>
</dependency>

inference4j-core has no Spring dependency, so you can keep it current while the starter stays pinned. The 0.10.1 starter will not receive fixes.

Tips

  • Use model-id to swap models without changing code (e.g., switch from ResNet to EfficientNet).
  • The text embedder has no default model ID — you must specify one via model-id.
  • All beans implement AutoCloseable and are properly cleaned up on application shutdown.