This is a 4-bit GPTQ quantized version of the Phi-3-mini-4k-instruct model. It enables efficient local inference of a capable small language model on consumer hardware. The model supports instruction following and can be used with standard Transformers pipelines or other inference frameworks.
In the Foundation models & chat space, Phi 3 Mini 4k Instruct takes a focused approach. It focuses on running capable language models with significantly reduced memory and compute requirements on local hardware. Phi 3 Mini 4k Instruct is an open-source project aimed at developers and researchers. The project is open source (Open Source). It runs on the web, the command line, and API.
kaitchup builds and maintains Phi 3 Mini 4k Instruct, and the product first shipped in 2024. Among its 4 catalogued features are 4-bit quantization, instruction tuned, and efficient inference.
Latest indexed changes and source events
kaitchup/Phi-3-mini-4k-instruct-gptq-4bit verified by the PulseGate indexer
Other apps tracked under the same category.
