Qwen3 4B Alternatives

GGUF quantized versions of the Qwen3.5-2B model, optimized by Unsloth for fast local inference. Compatible with llama.cpp, Ollama, and other GGUF runtimes. Suitable for edge devices or low-memory environments while retaining strong language modeling performance.

This repository provides GGUF quantized weights for the 0.6 billion parameter Qwen3 model. It is optimized for use with Unsloth, supporting fast fine-tuning and inference. The model includes advanced features such as tool calling and is designed for users who want a lightweight yet powerful open LLM that runs locally.

A quantized 4B parameter version of the Qwen3 model prepared for use with Unsloth. It supports efficient inference and fine-tuning on consumer hardware using bitsandbytes 4-bit quantization. The model includes optimized chat templates and tool-calling capabilities.

This is a GGUF-quantized release of Qwen3-235B-A22B, a massive mixture-of-experts language model optimized for local execution. It supports advanced reasoning, tool calling, and follows a specific chat template. Provided by Unsloth, it enables efficient inference of one of the largest open models using tools like llama.cpp.

This is the GGUF quantized format of Alibaba's Qwen3-4B large language model. It enables efficient local inference using tools like llama.cpp or LM Studio. The model includes support for tool calling and follows a specific chat template for structured interactions.

Qwen3.5-122B-A10B-GGUF is a quantized release of the Qwen3.5-122B-A10B foundation model hosted on Hugging Face by Unsloth. It is provided in GGUF format for local inference. The model supports processing of text, image, and video inputs through a chat template that handles multimodal content. Its template includes specific tokens such as vision_start, vision_end, image_pad, and video_pad, along with logic for counting vision elements and raising exceptions for unsupported cases like videos in system messages. The template also accommodates tool calling by formatting available functions when tools are supplied in the messages. It is delivered as a repository on the Hugging Face platform containing GGUF quantized files. The page includes a chat template implementation in a macro-based format that processes message lists, handles different content types, and generates appropriate system prompts for tool use. The model belongs to the class of foundation models. No pricing, licensing details, or target audience beyond the repository context are stated.

Qwen3.6-27B-GGUF is an open-source large language model distributed in GGUF format for local inference. It is designed for AI researchers and developers who require a powerful language model that can be run on local hardware for experimentation and development.

unsloth/Qwen3-14B-unsloth-bnb-4bit is a 4-bit quantized version of Qwen3-14B created with Unsloth optimizations. It supports dramatically faster fine-tuning and inference while maintaining high accuracy. The model includes advanced function calling and tool-use capabilities via its chat template.

This is a GGUF-quantized version of the Qwen3-4B-Instruct model optimized by Unsloth. It supports instruction following, tool calling, and efficient local inference. The model is distributed on Hugging Face for use with llama.cpp and similar runtimes.

Qwen3.5-27B-GGUF is an open-source large language model released in GGUF format for local inference and experimentation. It is suitable for AI researchers and developers who require access to model weights and the ability to run models on their own hardware.

Qwen3-30B-A3B-GGUF provides GGUF quantized weights for the Qwen3 30B-A3B model, enabling efficient local execution with tools like llama.cpp. It supports advanced features including tool calling and follows a specific chat template for multi-turn conversations. The model is intended for developers who want to run powerful language models offline.

This is a quantized 4-bit version of the Qwen3-8B large language model prepared by Unsloth. It includes optimized inference code, a chat template with tool-calling support, and is designed for local execution with significantly lower memory requirements than the original model. It is distributed on Hugging Face for developers building LLM applications.

This is a GGUF quantized version of the Qwen3-30B-A3B-Instruct model optimized for local use. It includes support for tool calling and follows an instruction-tuned format. The model is distributed by Unsloth and is compatible with popular GGUF inference engines.

Qwen3-8B-GGUF is the GGUF-quantized format of Alibaba's Qwen3-8B model. This version enables efficient local inference on consumer hardware using tools like llama.cpp or Ollama. It retains strong language understanding and generation capabilities while supporting tool calling and structured prompting formats.

Qwen3.5-4B is a compact multimodal model from the Qwen series, optimized by Unsloth for faster inference. It supports both text and vision inputs with a custom chat template. The model is distributed on Hugging Face and can be used with standard transformers or Unsloth libraries.

Qwen3-14B-GGUF contains GGUF format files for the 14 billion parameter Qwen3 model, enabling efficient local execution using tools like llama.cpp. It supports advanced features such as tool calling and follows a specific chat template for multi-turn conversations. This allows developers to run a powerful open LLM on consumer hardware without relying on cloud APIs.

Qwen3.5-0.8B-GGUF is an open-source foundation language model distributed in GGUF format for local inference. It enables developers and researchers to run advanced text generation tasks on their own hardware without relying on cloud APIs.

This GGUF-quantized version of Qwen3-VL-2B-Instruct enables efficient on-device or local-server multimodal inference. The model can process both text and images, supports tool calling, and is optimized for local use with tools such as llama.cpp. It is ideal for developers building vision-enhanced AI applications without relying on cloud APIs.

Qwen3.6-35B-A3B-GGUF is an open-source, quantized large language model distributed in GGUF format for local inference and research. It enables developers and researchers to run advanced language models on their own hardware, supporting experimentation and customization. The model is freely available for use and modification.

Qwen3.5-35B-A3B-GGUF is an open-source large language model designed for local inference and research. It enables developers and researchers to run text generation tasks efficiently on their own hardware with open weights.

Qwen3-32B-bnb-4bit is a bitsandbytes 4-bit quantized version of the Qwen3-32B model, optimized for use with the Unsloth library. It supports advanced features such as tool calling and is designed for efficient fine-tuning and inference on GPUs with limited VRAM.

unsloth/Qwen3-VL-4B-Instruct-GGUF is a quantized 4B parameter vision-language model from the Qwen3 family. It supports image understanding, visual reasoning, and instruction following. The GGUF format enables efficient local inference on consumer hardware using compatible runtimes.

zlaabsi/Qwen3.6-27B-OTQ-GGUF provides GGUF quantized weights of the Qwen3.6 27B model, enabling efficient local inference with tools like llama.cpp. The model supports multimodal inputs and is suitable for developers who want to run powerful language and vision models on their own machines without cloud dependency.

Qwen3.6-14B-A3B-FableVibes-GGUF is a GGUF quantized fine-tune of the Qwen3.6 model tuned for fable and storytelling tasks. It allows local execution using tools like llama.cpp or LM Studio. The model is available on Hugging Face for creative text generation applications.

Qwen3-Coder-Next-GGUF is a collection of GGUF quantized files for the Qwen3-Coder-Next series of coding models, published by Unsloth. It enables efficient local inference of powerful coding LLMs on CPUs and GPUs using tools like llama.cpp. The models support advanced code generation, tool use, and follow a specialized chat template for programming tasks.

Qwen3.5-35B-A3B-GGUF is an open-source large language model distributed by Unsloth on Hugging Face. It supports local inference, multiple quantizations, and is designed for AI researchers and developers seeking to run LLMs on their own hardware.

This repository contains GGUF quantized files for the Qwen3-4B model, enabling efficient local inference with tools such as llama.cpp. It includes support for tool calling and follows the Qwen chat template. The models are intended for developers seeking lightweight, locally runnable large language models.

This is a GGUF-quantized version of the Qwen3.6-35B-A3B model, optimized for efficient local inference using tools like llama.cpp. It supports multimodal inputs and is designed for developers who want to run powerful language models on standard hardware without relying on cloud APIs.

Qwen2.5-VL-7B-Instruct-GGUF is a repository on Hugging Face that supplies GGUF quantized files of the Qwen2.5-VL 7B vision-language model. It is intended for local inference of a multimodal model capable of processing both images and video alongside text. The repository is provided by unsloth. The files include variants such as Qwen2.5-VL-7B-Instruct-BF16.gguf and Qwen2.5-VL-7B-Instruct-IQ4_NL.gguf. A chat template is defined that handles messages containing text, image, or video content by inserting specific vision and image or video pad tokens. The template supports counting multiple images or videos in a conversation and adds optional identifiers such as "Picture 1:" or "Video 1:" before the visual tokens. It uses special tokens including vision_start, vision_end, image_pad, video_pad, im_start, and im_end. The total file size across the GGUF files is 15237851776 bytes. These quantized models are distributed for use with compatible inference engines that accept the GGUF format. The repository forms part of the broader collection of foundation models available on the platform. No pricing, licensing terms, or additional deployment details beyond the GGUF files and chat template are stated.

This is an Unsloth-optimized 4-bit quantized version of Alibaba's Qwen3-VL-2B-Instruct vision-language model. It supports understanding both images and text, making it suitable for multimodal tasks. The bnb-4bit format allows efficient local inference on modest hardware while preserving vision-language capabilities.

unsloth/Qwen3-VL-4B-Instruct-unsloth-bnb-4bit is a quantized 4B parameter vision-language model based on Qwen3-VL. Optimized by Unsloth for fast inference and fine-tuning, it supports image understanding, document parsing, and multimodal chat. The model uses BitsAndBytes 4-bit quantization.

This is a 4-bit quantized version of the Qwen3-4B-Instruct model created with Unsloth and bitsandbytes. It supports efficient inference and includes a chat template for tool calling and instruction following. The model is hosted on Hugging Face and can be used with the Transformers library.

This is a 4-bit quantized version of the Qwen3-VL-4B vision-language model, optimized by Unsloth using bitsandbytes. It allows efficient multimodal inference combining text and image inputs on resource-constrained devices. The model includes a specialized chat template for vision tasks and is distributed via Hugging Face for local use with pip installable libraries.

Qwen3.5-9B-GGUF is a community-provided collection of GGUF-quantized files for the Qwen3.5-9B language model. It enables efficient local execution using tools such as LM Studio, llama.cpp, or Ollama. The repository is targeted at developers and enthusiasts who want to run a capable open-source LLM on their own machines without relying on cloud APIs.

Qwen3-0.6B-GGUF is a quantized version of Alibaba's Qwen3 0.6 billion parameter language model provided in GGUF format for use with llama.cpp and other local inference tools. It includes a chat template supporting tool calling and is suitable for resource-constrained environments. The model is hosted on Hugging Face and can be used for text generation and conversational tasks.

An 8 billion parameter model from the Qwen3 family optimized by Unsloth. It includes native support for function calling and follows a chat template suitable for agentic workflows. The model is distributed on Hugging Face and can be used with Transformers or converted to GGUF for local LLM runners.

This is a 4-bit quantized version of the Qwen2.5-3B-Instruct model prepared with Unsloth and bitsandbytes. It enables developers to fine-tune LLMs significantly faster and with much lower memory usage compared to standard methods. The model is hosted on Hugging Face and is intended for efficient continued pre-training or instruction tuning.