Mesh LLM

Mesh LLM is a distributed model inference tool for running large language models across multiple nodes. It is described as a way to run bigger models without buying bigger GPUs, by splitting large models into smaller pieces so memory-bound GPUs can handle smaller layer slices on right-sized nodes.

The service supports two mesh modes. In router mode, the model field selects a host and requests are routed by model. In split mode, one large model is divided into layer ranges that are processed as a pipeline across several machines. The page also says that tokens can stream back to the client, and shows an OpenAI-compatible API with a local base URL, so existing tools can point at localhost while Mesh handles routing or splitting.

Mesh LLM is presented for use across a homelab, a business, or the internet. The examples and interface reference use with tools such as goose, vscode, opencode, and pi.dev, as well as any OpenAI client. It also includes a mesh planner for partitioning layers and a live model catalog drawn from Hugging Face, where models can be browsed and copied as org/repo quantized references for single-machine or multi-machine use.

Beyond inference, Mesh LLM includes a pluggable architecture for distributed agents. Plugins declare what they provide in a manifest, and the runtime starts them, routes calls, and exposes capabilities over MCP, HTTP, inference, and mesh events. The page also mentions blobstore state persistence, blackboard agent coordination, configuration settings, metrics, data, tools, inference backends, model serving, HTTP routes, REST endpoints, and mesh events channels and gossip.

Installation is shown with a shell command, curl -fsSL https://meshllm.cloud/install.sh | bash, followed by mesh-llm setup. The page does not state pricing or licensing information.