OpenVINO 2026.4: MTP, DFlash, Node.js ASR and New Models

Intel's OpenVINO 2026.4 release arrives while AI workloads keep shifting, and efficient, flexible inference is in higher demand than ever. Teams are shipping a wider spread of products, ranging from multimodal assistants to live speech pipelines, and the choice of models to run grows each month. This version responds with three things: a long list of new supported models, faster inference through two acceleration features, and ASRPipeline for Node.js. Whether the target is a production service on an AI PC, an experiment on an NPU, or a first look at OpenVINO, something in here applies.

The sections below cover faster decoding first, then the speech pipeline for JavaScript, and finally the new model list.

Faster generation: MTP and DFlash

Multi-Token Prediction

OpenVINO GenAI now supports Multi-Token Prediction (MTP) speculative decoding for Gemma 4, Qwen 3.5 and Qwen 3.6, on both CPUs and GPUs.

Some background helps. Ordinary speculative decoding uses a separate, small draft model. The draft proposes several candidate tokens, and the main model checks them all in a single forward pass. That cuts the number of full-model calls needed to produce a sequence.

MTP changes where the guesses come from. No separate draft model is needed. Prediction heads built into the model itself generate several tokens at once, speculatively. Since those heads are trained together with the main model, their guesses tend to be more accurate. A higher acceptance rate means more tokens confirmed on every pass. The outcome is higher throughput and lower latency, output quality isn't compromised, and existing pipeline setups need no changes.

DFlash for Qwen on Intel GPUs

DFlash acceleration for Qwen models arrives as a preview in OpenVINO GenAI. It targets GPUs on Intel Core Ultra Series 3 processors. The feature bundles several GPU optimizations with visual-token support for multimodal workloads, which lowers latency for GenAI pipelines that mix text and image input. For anyone building vision-language pipelines, or any workload that pushes Qwen hard on Intel GPU hardware, expect a quicker time to first token and better overall throughput.

Speech recognition for Node.js

ASRPipeline first appeared in release 2026.3 as a shared interface for automatic speech recognition models. It spares developers from handling each model's own input and output conventions. With this release, the same pipeline reaches Node.js, which opens it to the many JavaScript developers writing server-side and desktop AI applications.

The Node.js version works with models such as Whisper and Qwen3-ASR. It exposes the same core features as the C++ and Python versions: streaming transcription, performance metrics, and a clean pipeline API that hides the low-level details.

Streaming deserves attention. Instead of buffering a full audio input before returning anything, a streaming pipeline delivers text as soon as it's available. Live captioning, voice assistants and any other application where users can see latency depend on that.

The setup should feel familiar to anyone who has used other OpenVINO GenAI pipelines in Node.js. A full sample exists in the project's samples directory for the 2026.4 release branch, and a short snippet follows:

import { ASRPipeline } from 'openvino-genai-node';

import { readFileSync } from 'node:fs';

import { decode } from 'node-wav';



const pipeline = await ASRPipeline(modelDir, "CPU");

const rawSpeechBuffer = readFileSync(audioFilePath);

const rawSpeech = decode(rawSpeechBuffer).channelData[0];

const result = await pipeline.generate(rawSpeech);

console.log(result.texts[0]);

New supported models

The release adds a broad group of models across speech, vision-language, speech recognition and image generation.

On CPU and GPU

• Gemma-3n comes from Google DeepMind and continues the Gemma family. It's designed for efficient multimodal inference, balancing capability against resource use, and it suits on-device assistants and vision-language apps where memory is tight.
• Kokoro-82M is a compact text-to-speech model producing natural-sounding voices at a fraction of the size of bigger TTS systems. Its 82 million parameters are small enough for real-time speech generation on a CPU or a GPU, with none of the baggage of a full-size model.
• Qwen3-VL-4B with Eagle 3 combines Alibaba's vision-language model with Eagle 3, a speculative decoding component, so multimodal workloads generate tokens faster with no extra setup.
• Qwen3-ASR brings speech recognition to the Qwen3 family. It's a capable transcription model for voice-driven features such as captioning, transcription and speech-enabled assistants.
• Gemma 4 12B, also from Google DeepMind, sits in the middle of the Gemma 4 lineup. It has 12 billion parameters and accepts text, image and audio input. It's meant for developers who need stronger reasoning and multimodal results than lightweight edge models give, while still running practically on local or resource-conscious setups.
• Hy-MT2-1.8B is a small machine translation model tuned for fast multilingual inference. At 1.8 billion parameters, it provides dedicated translation across many languages in a small footprint. Think localization pipelines, live translation inside an app, or edge devices where a dedicated translator beats a general-purpose LLM.
• DeepSeek OCR-2 handles document understanding and optical character recognition. It's built to pull information out of scanned documents, forms, invoices and complex layouts, turning page images into data that downstream processing and automation can use.
• Granite 4.0 H Micro is IBM's small hybrid model, mixing Mamba-2 layers with transformer ones for low-latency, efficient inference. IBM aims it at agentic workflows and local deployments where hardware is limited.

On the NPU

• Kokoro-82M also extends to the NPU, which gives Intel AI PCs power-efficient speech synthesis and spares the CPU and GPU the memory and power demands.
• FLUX.2-Klein-4B, a Black Forest Labs model, adds small-footprint image generation on the NPU, so image synthesis in real time becomes a realistic goal on limited hardware.

Day 0 support

The project says it wants to keep pace with the community, and this release backs that up with Day 0 support for Muse Glimmer and Qwen 3.8 27B. Experiments on Intel hardware can start as soon as those models are out. For further optimized, ready-to-deploy models and benchmarks, the OpenVINO Model Hub is the place to look.

Where to start

The right first experiment depends on the hardware at hand. On a CPU or GPU already serving Gemma 4 or a recent Qwen model, MTP is the lowest-effort change, since pipeline setups stay as they are. A machine with a Core Ultra Series 3 GPU running Qwen vision-language workloads is the place to try the DFlash preview. A JavaScript backend that needs transcription can adopt ASRPipeline with Whisper or Qwen3-ASR, and streaming output is the part to evaluate first when latency matters to users. On an AI PC, the NPU additions make Kokoro-82M and FLUX.2-Klein-4B worth a test when power draw is the constraint rather than raw speed.

Summing up the release

New models span CPU, GPU and NPU, so there's more to build and more places to run it. MTP and DFlash squeeze extra inference performance from hardware that's already in place. ASRPipeline in Node.js gives JavaScript developers a route into speech recognition. The package is available to download now.