Mesh LLM is an open-source infrastructure tool that allows developers and ML engineers to run large language models by distributing inference across multiple machines or GPUs. It provides an OpenAI-compatible API endpoint, supports sharding model layers, and enables running models that exceed the memory of a single device. Mesh LLM is suitable for homelabs, businesses, and research environments seeking to maximize hardware utilization for LLM workloads.
Latest indexed changes and source events
Mesh-LLM/mesh-llm discovered by the PulseGate indexer
Other apps tracked under the same category.
