KALAVAI

KALAVAI is a protocol developed by Murai Labs for constructing a single routable model from independently trained AI specialist models. The approach addresses the challenge of combining specialists that have diverged from a shared base model, focusing on when and how such fusion is effective. KALAVAI frames the fusion process as a routing problem, where the fused model learns to select among specialists based on their divergence from the base model.

The protocol requires all specialists to originate from the same checkpoint, which ensures enough representational compatibility for post-hoc fusion. During training, there is no communication between specialists; instead, a lightweight routing mechanism is applied after training is complete. The system employs joint inference, meaning all specialists are active during inference rather than relying on single-expert dispatch, which is noted to fail due to specialists forgetting information outside their domains. KALAVAI introduces a measurable criterion—mean specialist divergence from the base model—that predicts the value of cooperative fusion before full evaluation. This divergence governs the potential gain from fusion, as specialists must differ meaningfully from the base model for the routing mechanism to be effective.

The protocol also explores the impact of freezing early layers (frozen anchors) during longer training horizons to maintain routing compatibility while allowing specialists to develop useful domain-specific expertise. Findings from experiments show that the fusion mechanism is applicable across various model sizes and that specialists tend to excel in their own domains, forming a diagonal cross-domain structure. Router confidence in the system approaches a hard switch, but still maintains coverage through joint inference. The importance of freeze depth increases with training duration, and primary evidence for the protocol’s effectiveness comes from perplexity and routing benchmarks rather than large benchmark movements.

Artifacts related to KALAVAI, including a research paper, code, and models, are available through platforms such as arXiv, GitHub, and Hugging Face. The protocol is positioned within the broader Murai Labs architecture and relates to systems focused on adaptive complexity routing and auditable decision traces.