Meta Llama 3 8B Instruct

MaziyarPanahi/Meta-Llama-3-8B-Instruct-GGUF is a repository on Hugging Face that hosts GGUF quantized versions of Meta's Llama 3 8B Instruct model. It supplies multiple quantization files to support efficient local inference on a range of hardware.

The available files include Meta-Llama-3-8B-Instruct.IQ1_M.gguf, Meta-Llama-3-8B-Instruct.IQ1_S.gguf, Meta-Llama-3-8B-Instruct.IQ2_XS.gguf, Meta-Llama-3-8B-Instruct.IQ3_XS.gguf, Meta-Llama-3-8B-Instruct.IQ4_XS.gguf, Meta-Llama-3-8B-Instruct.Q2_K.gguf, Meta-Llama-3-8B-Instruct.Q3_K_L.gguf, Meta-Llama-3-8B-Instruct.Q3_K_M.gguf, Meta-Llama-3-8B-Instruct.Q3_K_S.gguf, Meta-Llama-3-8B-Instruct.Q4_K_M.gguf, Meta-Llama-3-8B-Instruct.Q4_K_S.gguf, Meta-Llama-3-8B-Instruct.Q5_K_M.gguf, and Meta-Llama-3-8B-Instruct.Q5_K_S.gguf. These different levels allow users to select trade-offs between model size and performance. The repository also contains a chat template that defines formatting for messages with roles, beginning and end of text tokens, and assistant prompts.

It is delivered as downloadable GGUF files through the Hugging Face platform. The total file size across all variants is listed as 16069403840 bytes. This format enables compatibility with inference engines that support GGUF, such as those used for running large language models locally.

The repository forms part of the class of foundation models provided in quantized GGUF format for open-source use. No pricing, licensing details, or specific target audience beyond general access on Hugging Face are stated.