This repository hosts an AWQ 4-bit quantized version of the Mistral Small 24B Instruct 2501 model. It allows users to run this large instruction-tuned language model locally with significantly lower VRAM requirements than the original. The model supports the standard Mistral chat template and is compatible with the Hugging Face Transformers library.
Mistral Small 24B Instruct 2501 sits in PulseGate's Text generation category. It focuses on enabling efficient local deployment of a 24B parameter instruction-tuned model with reduced memory footprint. Mistral Small 24B Instruct 2501 is an open-source project aimed at developers. Mistral Small 24B Instruct 2501 is open source under the MIT license. Mistral Small 24B Instruct 2501 is available on the web, the command line, and API.
stelterlab builds and maintains Mistral Small 24B Instruct 2501, and it first shipped in 2023. The GitHub repository has been archived. It operates in a well-populated space: PulseGate tracks 5 similar projects. Key capabilities include 4-bit Quantization, Instruction Tuned, and Large Context.
Summary written by a language model from the project’s public pages.
What PulseGate has recorded for this listing
Closest matches by what these projects do