Hermes 4 70B Alternatives

This repository hosts a 6-bit quantized version of the Hermes-4-70B model in MLX format, optimized for Apple silicon (M-series chips). It includes advanced system prompts for deep thinking and tool use, enabling high-quality local inference on Macs without requiring high-end GPUs.

Hermes-4-70B-MLX-5bit is a 5-bit quantized version of the Hermes-4 70B model, optimized for the MLX framework on Apple silicon. It includes a custom chat template supporting function calling and extended reasoning modes. The model is designed for local inference with significantly reduced memory requirements while maintaining strong performance.

This is a 4-bit quantized version of the Hermes-4 70B model optimized for the MLX framework on Apple silicon. It supports advanced reasoning, tool use, and long-context conversations. The model is distributed on Hugging Face for local deployment using LM Studio or compatible runtimes.

This is a 4-bit quantized version of the LFM2-24B model using the MLX framework, optimized for Apple silicon devices. Hosted by the LM Studio community, it enables efficient local inference of a large language model on Macs. The model includes a comprehensive chat template supporting tools and system prompts.

Hermes-4-14B-AWQ-4bit is a quantized variant of the Hermes 4 model optimized for lower memory usage. It includes support for extended thinking, function calling, and structured output. Designed for developers who need high-capability open models that can run efficiently on consumer or enterprise hardware.

This repository hosts an 8-bit quantized MLX version of the LFM2-24B model optimized for Apple Silicon. It includes a custom chat template with system prompt and tool support. The model is designed for local inference on Macs using the MLX framework and is distributed via Hugging Face.

A 4-bit quantized version of the LFM2.5 1.2B Instruct model optimized for the MLX framework on Apple Silicon. It is designed for local inference on Macs and includes a chat template suitable for instruction following. The model is distributed via Hugging Face for use with MLX and LM Studio.

GLM-4.7-Flash-MLX-8bit is a community-quantized version of the GLM-4 large language model using 8-bit precision and optimized for the MLX framework on Apple silicon. It supports tool calling and can be used locally through libraries such as Transformers or MLX. The model is hosted on Hugging Face for easy download and integration into local AI applications.

This is a 6-bit quantized version of the GLM-4.7-Flash model, prepared by the LM Studio community for efficient inference using the MLX framework on Apple devices. It includes support for tool calling and follows a standard chat template. The model is hosted on Hugging Face for easy local deployment.