This is an intermediate training checkpoint from the TinyLlama project, which aims to pretrain a 1.1B parameter model on 3 trillion tokens. The model follows the architecture and tokenizer of Llama but is significantly smaller. It serves as a resource for researchers and developers interested in small language models.
TinyLlama 1.1B Intermediate Step 1431k 3T sits in PulseGate's Foundation models & chat category. It focuses on creating a compact, efficient language model that can run on resource-constrained devices while maintaining reasonable performance. It is built as an open-source project for developers. TinyLlama 1.1B Intermediate Step 1431k 3T is open source under the Apache-2.0 license. It runs on the web, the command line, and API.
TinyLlama builds and maintains TinyLlama 1.1B Intermediate Step 1431k 3T, and the product first shipped in 2023. The GitHub repository has been archived. Key capabilities include text generation, small language model, and pretraining checkpoint.
Latest indexed changes and source events
TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T verified by the PulseGate indexer
⚠ Archived
Other apps tracked under the same category.
