sparkrun

Sparkrun is a command-line interface tool designed for launching, managing, and stopping large language model (LLM) inference workloads on one or more NVIDIA DGX Spark systems. It eliminates the need for traditional cluster management solutions such as Slurm or Kubernetes, aiming to simplify the orchestration of AI inference tasks across specialized hardware.

A key feature of sparkrun is its recipe system, where YAML configurations define model, container, runtime, and default settings. Users can override these parameters at launch, avoiding the need to manually search for configuration files. The tool manages container orchestration, model distribution, and networking automatically, streamlining the deployment process. For scaling across multiple DGX Sparks, sparkrun provides multi-node tensor parallelism, with each system contributing a GPU, and it handles InfiniBand/RDMA and NCCL configuration to support efficient distributed workloads.

Sparkrun also offers integration with git-based recipe registries, enabling users to share and collaborate on recipes through both community and private repositories, and to search across all available registries. Its VRAM estimation feature auto-detects model architecture from HuggingFace, helping users determine whether a configuration fits on a single DGX Spark or requires multiple nodes before launching. The tool supports multiple inference runtimes, including vLLM, SGLang, llama.cpp, and atlas, all accessible through a consistent CLI and recipe format.

Additional capabilities include rich shell tab completion for Bash, Zsh, and Fish, allowing instant completion of commands, recipe names, cluster names, and options. An AI-assisted inference management feature is available via the Claude Code Plugin, which learns the user's cluster and enables conversational management of running, monitoring, and stopping workloads.

Installation is performed with a single command that sets up the managed environment, configures tab completion, and initiates a setup wizard to guide cluster configuration. Sparkrun is distributed under the Apache 2.0 license and is available on PyPI. It is part of the Spark Arena ecosystem and is powered by scitrera.ai.