mycellm_

mycellm_ is an open source platform for distributed inference with large language models (LLMs), enabling users to chat with open models for free using compute resources contributed by a global network of GPU nodes. The system is designed to avoid single points of failure by routing requests across multiple peers, ensuring continued availability even if individual nodes go offline. Users can contribute their own idle GPU resources to the network and earn credits, which can then be spent to run larger models within the system, establishing a credit-based exchange for compute power.

The platform offers a drop-in OpenAI-compatible API, allowing users to switch from other providers by changing a single environment variable. dev. mycellm_ also features a Sensitive Data Guard, which scans outgoing prompts on-device for sensitive information such as API keys, passwords, and personally identifiable information; if high-severity matches are detected, prompts are automatically routed to a local model so that sensitive data never leaves the user's device.

Users can participate in the network without signing up, and the platform can be installed via pip, Docker, or a one-liner script on macOS, Linux, or Windows. Additionally, there is an iOS and iPadOS app that allows iPhones or iPads to function as nodes within the network, with M-series iPad Pros capable of running models at notable speeds. The platform provides a web dashboard accessible at localhost:8420 for managing chat and network participation.

0 open source license, allowing users to audit, fork, and extend the codebase. The platform emphasizes transparency, community-driven scaling, and the absence of vendor lock-in. Users can also run their own private swarms or federate with trusted peers, maintaining control over membership, models, and data routing across networks.