This repository contains a minimal, randomly initialized GLM-4 MoE (Mixture of Experts) model for causal language modeling. It is maintained by the TRL (Transformers Reinforcement Learning) internal testing organization and is used to test training, inference, and optimization code paths.
Tiny Glm4MoeForCausalLM sits in PulseGate's Foundation models & chat category. It focuses on enabling unit tests and pipeline validation for GLM-4 style Mixture-of-Experts architectures. Tiny Glm4MoeForCausalLM is an open-source project aimed at developers. The project is open source (Apache-2.0). The product ships for the web, the command line, and API.
Behind Tiny Glm4MoeForCausalLM is trl-internal-testing, and the product first shipped in 2020. The project is developed in the open on GitHub with 18.9k stars and 496 commits in the last 90 days. Among its 3 catalogued features are mixture of Experts, test model, and Causal LM.
Latest indexed changes and source events
trl-internal-testing/tiny-Glm4MoeForCausalLM verified by the PulseGate indexer
Other apps tracked under the same category.
