This repository hosts an AWQ 4-bit quantized version of the Mistral Small 24B Instruct 2501 model. It allows users to run this large instruction-tuned language model locally with significantly lower VRAM requirements than the original. The model supports the standard Mistral chat template and is compatible with the Hugging Face Transformers library.
Mistral Small 24B Instruct 2501 is a Foundation models & chat product. It focuses on enabling efficient local deployment of a 24B parameter instruction-tuned model with reduced memory footprint. It is built as an open-source project for developers. Mistral Small 24B Instruct 2501 is open source under the MIT license. It runs on the web, the command line, and API.
It is developed by stelterlab, and the product first shipped in 2023. The GitHub repository has been archived. It operates in a well-populated space: PulseGate tracks 5 similar tools. Key capabilities include 4-bit Quantization, Instruction Tuned, and Large Context.
Latest indexed changes and source events
stelterlab/Mistral-Small-24B-Instruct-2501-AWQ verified by the PulseGate indexer
⚠ Archived
Other apps tracked under the same category.
