KVzap Mlp Llama 3.1 8B Instruct
nvidia/KVzap-mlp-Llama-3.1-8B-Instruct is a variant of the Llama 3.1 8B model that incorporates KVzap, a method for pruning the key-value cache using a lightweight MLP to predict importance scores. This accelerates both prefilling and decoding phases of LLM inference while maintaining model quality. It is hosted on Hugging Face and can be used with the Transformers library.
Open SourceApache-2.0WebAPI
1.2kstars
167forks
4features
2024since
Overview
4 featuresKVzap Mlp Llama 3.1 8B Instruct is an Other AI project. It focuses on reducing the computational cost and memory usage of large language model inference through intelligent KV cache pruning. KVzap Mlp Llama 3.1 8B Instruct is an open-source project aimed at machine learning engineers and researchers. KVzap Mlp Llama 3.1 8B Instruct is open source under the Apache-2.0 license. KVzap Mlp Llama 3.1 8B Instruct is available on the web and API.
NVIDIA builds and maintains KVzap Mlp Llama 3.1 8B Instruct, and it first shipped in 2024. The project is developed in the open on GitHub with 1.2k stars and 14 commits in the last 90 days. Key capabilities include KV Cache Pruning, Fast Inference, and llama-based. It exposes integrations via a public API.
Summary written by a language model from the project’s public pages.
- KV Cache Pruning
- Fast Inference
- Llama-based
- Instruction Tuned
Tags
llama-3kv-cache-pruninginference-optimizationnvidia-model
AI capabilities
Inference: LocalWeights: Open
Built with & integrations
Hosting
AWS
AI providers
meta_llama
Connectors
API
Runs on
BrowserAPI-only
Detected from
- AWS
- x-amz-cf-id header · x-amz-cf-pop header · via header
- meta_llama
- llama- in the HTML
Trust & compliance
License
Apache-2.0
Verified signals
Indexing history
1What PulseGate has recorded for this listing
- Indexed5 Aug · 09:38 UTCnvidia/KVzap-mlp-Llama-3.1-8B-Instruct verified against its public sourceSource: PulseGate · Open
Frequently asked questions about KVzap Mlp Llama 3.1 8B Instruct
- What does KVzap Mlp Llama 3.1 8B Instruct do?
- KVzap Mlp Llama 3.1 8B Instruct focuses on reducing the computational cost and memory usage of large language model inference through intelligent KV cache pruning. It is catalogued under Other AI on PulseGate.
- Who should use KVzap Mlp Llama 3.1 8B Instruct?
- KVzap Mlp Llama 3.1 8B Instruct is an open-source project built for machine learning engineers and researchers.
- Is KVzap Mlp Llama 3.1 8B Instruct free?
- Yes — KVzap Mlp Llama 3.1 8B Instruct is open source under the Apache-2.0 license and free to use.
- What platforms does KVzap Mlp Llama 3.1 8B Instruct run on?
- KVzap Mlp Llama 3.1 8B Instruct runs on the web and API.
- Is KVzap Mlp Llama 3.1 8B Instruct still active?
- The GitHub repository shows 14 commits in the last 90 days.
- What projects are similar to KVzap Mlp Llama 3.1 8B Instruct?
- Similar projects tracked by PulseGate include Llama 3.1 8B Instruct, Llama 3.1 8B Instruct FP8 KV, and Llama 3.3 70B Instruct.Llama 3.1 8B InstructLlama 3.1 8B Instruct FP8 KVLlama 3.3 70B Instruct
- Who makes KVzap Mlp Llama 3.1 8B Instruct?
- KVzap Mlp Llama 3.1 8B Instruct is developed by NVIDIA, based in the United States.
- When did KVzap Mlp Llama 3.1 8B Instruct launch?
- KVzap Mlp Llama 3.1 8B Instruct first shipped in 2024.
Similar apps
Closest matches by what these projects do