llm-inspector Alternatives

llm4s is an open-source CLI tool inspired by K9s, designed to provide observability and debugging for LLM and MCP servers. It offers a terminal UI for monitoring server activity, integrating with MCP, and assisting AI developers in managing and troubleshooting their deployments.

llm-cal is an open-source command-line tool that helps AI engineers and infrastructure planners calculate hardware requirements for LLM inference. It supports architecture and engine version awareness, providing honest and detailed compatibility assessments.

llm-benchmark-runner is an open-source CLI tool that allows developers and researchers to benchmark the inference latency and throughput of large language model APIs. It supports OpenAI-compatible endpoints and provides detailed performance metrics for model evaluation.

llm-interview is an open-source command-line tool that allows developers to submit and manage LLM interview history directly from their terminal. It is designed for users who need to track or submit interview data programmatically. The tool is suitable for developers and technical users who work with LLM interview workflows.

llmlint-cli is an open-source command-line linter that leverages large language models to enforce code-quality checks that traditional linters cannot express. It integrates with coding harnesses to provide advanced, AI-driven code review for developers.

CLI for benchmarking LLM inference servers (vLLM, SGLang, llama.cpp)

llm-speed is an open-source CLI tool that benchmarks the inference speed of large language models (LLMs) across various hardware setups and hosted APIs. It provides reproducible, community-verified results for AI researchers and developers seeking to optimize model performance.

Intercept and analyze LLM traffic from AI coding tools

inferguard is a CLI tool for read-only diagnostics and benchmarking of large language model serving frameworks such as vLLM, SGLang, Dynamo, and llm-d. It helps infrastructure engineers analyze serving performance and identify bottlenecks. The tool is open source and designed for advanced LLM deployment environments.

llmeter is an open-source CLI tool that enables developers and researchers to profile the latency and throughput of large language models (LLMs). It provides cross-platform support for performance testing and optimization of AI workloads.

llm-ekg is an open-source CLI tool that monitors the health of large language models by detecting degradation and hallucinations using mathematical and signal processing techniques. It is designed for AI researchers and ML engineers who need robust, language-agnostic model evaluation.

llm-autotune is an open source command-line tool that optimizes local large language model (LLM) inference on platforms like Ollama, LM Studio, and MLX. It reduces latency and memory usage, making local LLM deployment more efficient for developers.

WatchLLM is a command-line tool that intercepts every save operation made by AI coding agents, running deterministic AST-level security checks in milliseconds. It blocks dangerous code before it touches disk, providing developers with a safety layer when using AI-assisted coding tools. Ideal for teams and individuals integrating AI agents into their development workflow.

llm-usage-meter is an open-source CLI tool that enables developers to check their remaining credits and quotas across various LLM providers directly from the terminal. It is lightweight and has zero dependencies.

inferbench-cli is an open-source CLI tool for benchmarking local LLM inference speed and providing hardware configuration advice for omlx and llama.cpp. It helps developers and researchers optimize AI model performance on their own machines.

llm-usage-report is an open-source CLI tool that parses logs from LLM APIs (Anthropic, OpenAI, Google) to generate detailed reports on token usage and costs. It supports budget alerts and is designed for developers and teams managing LLM API expenses.