parakeet-rs

Fast speech recognition and diarization with NVIDIA's Parakeet models, on ONNX Runtime or pure Rust with burn.

Models

ONNX Runtime (the default backend) needs one environment per process, shared by every model. Create it once; the snippets below use this config:

use parakeet_rs::ExecutionConfig;

let env = parakeet_rs::ort::init().build()?;
let config = ExecutionConfig::new().with_environment(env);

CTC (English-only):

use parakeet_rs::{Parakeet, Transcriber, TimestampMode};

let mut parakeet = Parakeet::from_pretrained(".", Some(config.clone()))?;

// Load and transcribe audio (see examples/raw.rs for full example)
let result = parakeet.transcribe_samples(audio, 1600, 1, Some(TimestampMode::Words))?;
println!("{}", result.text);

// Token-level timestamps
for token in result.tokens {
    println!("[{:.3}s - {:.3}s] {}", token.start, token.end, token.text);
}

TDT (Multilingual): 25 languages with auto-detection

use parakeet_rs::{ParakeetTDT, Transcriber, TimestampMode};

let mut parakeet = ParakeetTDT::from_pretrained("./tdt", Some(config.clone()))?;
let result = parakeet.transcribe_samples(audio, 16000, 1, Some(TimestampMode::Sentences))?;
println!("{}", result.text);

// Token-level timestamps
for token in result.tokens {
    println!("[{:.3}s - {:.3}s] {}", token.start, token.end, token.text);
}

EOU (Streaming): Real-time ASR with end-of-utterance detection

use parakeet_rs::ParakeetEOU;

let mut parakeet = ParakeetEOU::from_pretrained("./eou", Some(config.clone()))?;

// Prepare your audio (Vec<f32>, 16kHz mono, normalized)
let audio: Vec<f32> = /* your audio samples */;

// Process in 160ms chunks for streaming
const CHUNK_SIZE: usize = 2560; // 160ms at 16kHz
for chunk in audio.chunks(CHUNK_SIZE) {
    let text = parakeet.transcribe(chunk, false)?;
    print!("{}", text);
}

Nemotron (Streaming): Cache-aware streaming ASR with punctuation. Two variants share the same API — point from_pretrained at whatever directory holds the ONNX files and the loader auto-detects which variant it is:

• English-only 0.6B — verbatim English, preserves disfluencies (um, uh). Best for transcription where every spoken word matters.
• Multilingual 3.5 0.6B — 40 language-locales across 3 tiers (19 transcription-ready, 13 broad-coverage, 8 that need fine-tuning to reach production quality based on the NVIDIA). Polished output (proper casing/punctuation, drops disfluencies). Same speed and size as the English-only model.

use parakeet_rs::{Nemotron, NemotronMode};

let mut model = Nemotron::from_pretrained(path, Some(config.clone()))?;

// Multilingual variant: optionally pick a target language. Defaults to "auto"
// (the model picks the language itself). Pass a specific code when you know
// the language it's strictly more accurate. No-op when an English-only model is loaded.
if model.mode() == NemotronMode::Multilingual {
    model.set_target_lang("es-ES")?; // also: "ja-JP", "tr-TR", "auto", ...
}

// Process in 560ms chunks for streaming
const CHUNK_SIZE: usize = 8960; // 560ms at 16kHz
for chunk in audio.chunks(CHUNK_SIZE) {
    let text = model.transcribe_chunk(chunk)?;
    print!("{}", text);
}

Cohere Transcribe (Offline Multilingual): 14 languages, punctuation & ITN toggles (yes, "parakeets🦜" talk about more than just NVIDIA right?? :-P)

parakeet-rs = { version = "0.4", features = ["cohere"] }

use parakeet_rs::CohereASR;

let mut model = CohereASR::from_pretrained("./cohere", Some(config.clone()))?;

// audio: Vec<f32>, 16kHz mono (long-form supported)
let text = model.transcribe_audio(&audio, "en", true, false)?; // lang, pnc, itn
println!("{}", text);

See examples/cohere.rs for a runnable demo.

Multitalker (Streaming Multi-Speaker ASR): Speaker-attributed transcription

parakeet-rs = { version = "0.4", features = ["multitalker"] }

use parakeet_rs::MultitalkerASR;

let mut model = MultitalkerASR::from_pretrained(
    "./multitalker",             // encoder, decoder, tokenizer
    "nemotron3_diar_v3.onnx",    // Nemotron-3 (Sortformer v3) for diarization
    Some(config.clone()),
)?;

for chunk in audio.chunks(17920) {  // ~1.12s at 16kHz
    let results = model.transcribe_chunk(chunk)?;
    for r in &results {
        println!("[Speaker {}] {}", r.speaker_id, r.text);
    }
}

See examples/multitalker.rs for full usage with latency modes.

Nemotron-3 Diarization (Sortformer v3): Streaming diarization, up to 8 speakers

parakeet-rs = { version = "0.4", features = ["sortformer"] }

use parakeet_rs::sortformer::{DiarizationConfig, Sortformer, StreamingProfile};

let mut diarizer = Sortformer::with_config(
    "nemotron3_diar_v3.onnx",
    Some(config.clone()),
    DiarizationConfig::default(),
)?;
let segments = diarizer.diarize(audio, 16000, 1)?;
for seg in segments {
    println!("speaker_{} [{:.2}s - {:.2}s]", seg.speaker_id,
        seg.start as f64 / 16_000.0, seg.end as f64 / 16_000.0);
}

// Streaming/real-time: diarize_chunk() / feed() preserve state across calls.
let segments = diarizer.diarize_chunk(&audio_chunk_16k_mono)?;

// Latency presets from NVIDIA's model card:
// offline() 30.4s (default), low_latency() 1.04s, very_low_latency() 0.64s, ultra_low_latency() 0.32s
diarizer.set_profile(StreamingProfile::low_latency())?;

See examples/diarization.rs for a runnable example (pass low, very-low or ultra to try the latency presets) and examples/streaming_diarization.rs for real-time feed/flush.

See scripts/export_diar_sortformer.py for exporting the ONNX (dual-resolution, self-describing metadata) with custom streaming parameters.

Setup

CTC: Download from HuggingFace: model.onnx, model.onnx_data, tokenizer.json. Or export it yourself with scripts/export_parakeet_ctc.py.

TDT: Download from HuggingFace: encoder-model.onnx, encoder-model.onnx.data, decoder_joint-model.onnx, vocab.txt. Or export it yourself with scripts/export_parakeet_tdt.py.

Parakeet Ultra (optional TDT fine-tune): Moondream's post-trained parakeet-tdt-0.6b-v3 with better accuracy. Download from HuggingFace: same files as TDT, loads with ParakeetTDT (CC-BY-4.0). Or export it yourself with scripts/export_parakeet_tdt.py --ultra.

Orukeet (optional TDT fine-tune): the pinned Hugging Face INT8 export works with ParakeetTDT. The downloader verifies a release manifest and all required file hashes, including licenses, then prints the cached model directory:

python3 -m pip install huggingface-hub
model_dir=$(python3 scripts/download_orukeet.py)
cargo run --release --example orukeet -- "$model_dir" audio.wav
# After installation, no network is needed:
model_dir=$(python3 scripts/download_orukeet.py --offline)

This uses the existing local TDT runtime; it does not add streaming support or change defaults. The ~672 MB weights are licensed under CC BY-SA 4.0, as recorded in the downloaded LICENSE-WEIGHTS; NOTICE.md preserves the NVIDIA/Parakeet attribution. Model downloads use normal Hugging Face accounting through the required verification manifest. Audio stays local; cached files cause no counting requests. Accuracy numbers on the model card describe its NeMo evaluation, not a benchmark of this Rust runtime.

EOU: Download from HuggingFace: encoder.onnx, decoder_joint.onnx, tokenizer.json. Or export it yourself with scripts/export_realtime_eou_120m.py.

Nemotron (English-only): Download from HuggingFace: encoder.onnx, encoder.onnx.data, decoder_joint.onnx, tokenizer.model. Or export it yourself with scripts/export_nemotron_streaming.py. (int8 / int4)

Nemotron (Multilingual 3.5): Download from HuggingFace: encoder.onnx, encoder.onnx.data, decoder_joint.onnx, tokenizer.model. Or export it yourself from the base model with scripts/export_nemotron_streaming_multilingual.py.

Unified: Download from HuggingFace: encoder.onnx, encoder.onnx.data, decoder_joint.onnx, tokenizer.model. Or export it yourself with scripts/export_parakeet_unified.py.

Multitalker: Download from HuggingFace: encoder.onnx + encoder.onnx.data and decoder_joint.onnx (fp32), or encoder.int8.onnx and decoder_joint.int8.onnx, plus tokenizer.model (also needs the Nemotron-3 diarization ONNX below). Or export it yourself with scripts/export_multitalker.py.

Cohere Transcribe: Download from HuggingFace: encoder_model.onnx (+ .onnx_data*), decoder_model_merged.onnx (+ .onnx_data), tokenizer.json (FP32, FP16, INT8, INT4 variants available)

Diarization (Nemotron-3 / Sortformer v3): Download from HuggingFace: nemotron3_diar_v3.onnx. Or export it yourself from the base model with scripts/export_diar_sortformer.py.

Quantized versions available (int8). All files must be in the same directory.

The scripts/export_*.py scripts export every model above from NVIDIA's NeMo checkpoints (except Cohere, which comes from Optimum), with PyTorch's dynamo exporter at ONNX opset 23.

GPU support (auto-falls back to CPU if fails):

parakeet-rs = { version = "0.4", features = ["cuda"] }  # or tensorrt, webgpu, directml, migraphx or other ort supported EPs (check cargo features)

use parakeet_rs::{Parakeet, ExecutionConfig, ExecutionProvider};

let config = ExecutionConfig::new()
    .with_environment(parakeet_rs::ort::init().build()?)
    .with_execution_provider(ExecutionProvider::Cuda);
let mut parakeet = Parakeet::from_pretrained(".", Some(config))?;

Advanced session configuration via ort SessionBuilder:

let config = ExecutionConfig::new()
    .with_custom_configure(|builder| builder.with_memory_pattern(false));

If your app also uses ort directly, use the same environment for both: ONNX Runtime allows only one per process. parakeet_rs::ort is the ort version parakeet-rs is built with.

Pure-Rust backend (experimental)

All models can also run on burn, without ONNX Runtime, on the CPU or GPU. Same model files (fp32), matching transcripts. Cohere needs its fp32 export there (about 8.4 GB, held in memory).See details:v0.4.0 release notes

parakeet-rs = { version = "0.4", default-features = false, features = ["metal"] }  # "wgpu" for other GPUs, "burn" for CPU only

let config = ExecutionConfig::new().with_execution_provider(ExecutionProvider::BurnWgpu);
let mut parakeet = ParakeetTDT::from_pretrained("./tdt", Some(config))?;

Other GPU features: vulkan, burn-cuda and burn-rocm. For burn on the CPU, apple-amx (Apple Silicon) or x86-v4 (AVX-512) speed up matrix multiplication. See examples/burn.rs.

Features

• CTC: English with punctuation & capitalization
• TDT: Multilingual (auto lang detection)
• Parakeet Ultra: Moondream's post-trained TDT, same API (ONNX)
• EOU: Streaming ASR with end-of-utterance detection
• Nemotron: Cache aware streaming ASR (600M params,EN only)
• Unified: Offline + buffered streaming RNNT ASR (600M params, EN only)
• Multitalker: Streaming multi-speaker ASR with speaker-kernel injection (ONNX)
• Cohere Transcribe: Offline multilingual ASR (14 languages, long-form supported) (ONNX)
• Nemotron-3 Diarization (Sortformer v3): Streaming speaker diarization (up to 8 speakers) (ONNX)
• Token-level timestamps (CTC, TDT)

Notes

• Audio: 16kHz mono WAV (16-bit PCM or 32-bit float)
• CTC/TDT models have ~4-5 minute audio length limit. For longer files, use streaming models or split into chunks

License

Code: MIT OR Apache-2.0

FYI: The Parakeet ONNX models (downloaded separately from HuggingFace) by NVIDIA. This library does not distribute the models.