Granite Speech 4.1 2B by IBM

1 2B by IBM is a speech-language model designed for multilingual automatic speech recognition (ASR) and bidirectional automatic speech translation (AST). It supports English, French, German, Spanish, Portuguese, and Japanese, with additional translation capabilities for English to Italian and English to Mandarin. The model is intended for enterprise applications that require processing of speech inputs, particularly those involving transcription and translation across multiple languages.

This model was trained on 174,000 hours of audio from public corpora and synthetic datasets, including resources tailored for Japanese ASR, keyword-biased ASR, and speech translation tasks. 1 2B introduces a dual-head CTC encoder that outputs both graphemic and byte pair encoding (BPE) representations, enhancing transcription accuracy for multilingual ASR. It incorporates frame importance sampling to focus on informative segments of audio and supports prompt-based punctuation and truecasing for ASR and AST in all supported languages, including German noun capitalization. The model also offers improved keyword list biasing to better recognize names, acronyms, and technical jargon.

Performance evaluations were conducted on public benchmarks, with particular focus on English ASR as well as multilingual ASR and AST tasks. Metrics such as punctuation error rate (PER) and capitalization F1 (Cap-F1) were reported across various datasets, demonstrating high accuracy in punctuation and capitalization, especially for German. The model’s keyword list biasing capability was evaluated using F1 scores for transcribed keywords during ASR tasks.

0 license. 1 or higher) and is compatible with vLLM. The model weights are accessible through GitHub, and usage examples are provided for both online and offline inference. The tool is positioned within the class of multilingual speech recognition and translation models, tailored for enterprise deployment.