nvidia-nat-profiler is an open-source CLI tool for profiling and evaluating the performance of AI agents in the NVIDIA NeMo Agent Toolkit. Below are 11 llm eval & observability apps with similar functionality to nvidia-nat-profiler, matched by what each product actually does — not ranked or scored. Explore each to find the closest fit for your use case.
nvidia-nat-opentelemetry is an open-source CLI tool that integrates OpenTelemetry observability into the NVIDIA NeMo Agent Toolkit, allowing AI developers and DevOps engineers to monitor and collect metrics from their agent workflows.
nvidia-nat-config-optimizer is an open-source CLI tool for optimizing workflow configurations and prompts in the NVIDIA NeMo Agent Toolkit. It is aimed at AI developers seeking to streamline agent-based workflows and improve prompt engineering.
Subpackage for evaluation in NVIDIA NeMo Agent Toolkit
nvidia-nat-app is an open-source CLI toolkit by NVIDIA that provides reusable performance primitives for building and optimizing agentic applications. It is framework-agnostic and designed for developers working on AI-driven or agent-based systems.
nvidia-nat-redis is an open-source CLI tool that integrates Redis as a memory backend for the NVIDIA NeMo Agent Toolkit, enabling persistent storage and memory management for AI agents. It is aimed at AI developers and infrastructure engineers.
nvidia-nat-s3 is an open-source subpackage for the NeMo Agent Toolkit that adds S3-compatible storage integration, allowing AI agents to store and access data efficiently. It is distributed via PyPI under the Apache-2.0 license.
nvidia-nat-nemo-customizer is an open-source Python package that integrates the NeMo Customizer into the NeMo Agent Toolkit, allowing developers to fine-tune AI models for custom applications. It is designed for AI practitioners and researchers working with NeMo agents and model customization workflows.
nvidia-nat-phoenix is an open-source CLI subpackage that integrates Arize Phoenix observability features into the NeMo Agent Toolkit. It helps AI developers monitor and evaluate agent workflows more effectively.
nvidia-nat-agno is an open-source subpackage that integrates Agno into the NeMo Agent Toolkit, enabling developers to build, manage, and enhance AI agents with retrieval-augmented generation and other advanced features.
nvidia-nat-a2a is an open-source subpackage for the NeMo Agent Toolkit that adds A2A Protocol integration, enabling AI agents to communicate and interoperate using standardized protocols. It supports retrieval-augmented generation and is distributed via PyPI under Apache-2.0.
nvidia-nat-mysql is an open source subpackage for the NVIDIA NeMo Agent Toolkit, enabling seamless MySQL database integration for AI agent development. It provides CLI and API support for developers building data-driven AI agents.
