This is a quantized version of Mistral-7B-Instruct-v0.2 using the AWQ method to 4-bit precision. It enables efficient local inference of the instruction-tuned Mistral model on consumer hardware with reduced VRAM usage while maintaining high performance. The model is hosted on Hugging Face and can be used with the Transformers library or compatible inference engines.
In the Foundation models & chat space, Mistral 7B Instruct takes a focused approach. It focuses on running large language models with reduced memory and compute requirements on local hardware. Mistral 7B Instruct is an open-source project aimed at developers. The project is open source (AGPL-3.0). Mistral 7B Instruct is available on the web, the command line, and API.
It is developed by TheBloke, and the product first shipped in 2022. The project is developed in the open on GitHub with 47.5k stars and 140 commits in the last 90 days. PulseGate's similarity index places it among 6 comparable tools. Among its 3 catalogued features are 4-bit Quantization, Instruction Tuned, and Mistral Architecture.
Latest indexed changes and source events
TheBloke/Mistral-7B-Instruct-v0.2-AWQ verified by the PulseGate indexer
Other apps tracked under the same category.
