thaw-vllm is an open-source CLI tool and Python library that snapshots running LLM inference sessions, including weights and KV cache, and hydrates them into multiple divergent children. It supports parallel agent workflows, RL rollouts, and is compatible with vLLM and SGLang.
In the Other AI space, thaw-vllm takes a focused approach. It enables efficient parallelization and branching of LLM inference sessions for advanced agent workflows and RL rollouts. thaw-vllm is an open-source project aimed at AI researchers and developers working with LLM inference. The project is open source (Apache-2.0). thaw-vllm is available on the web and the command line.
thaw-ai builds and maintains thaw-vllm, and the product first shipped in 2026. The project is developed in the open on GitHub with 102 commits in the last 90 days. Across PulseGate's embedding index, thaw-vllm has few near neighbours, marking it as relatively distinct. Among its 6 catalogued features are session snapshotting, KV cache management, and parallel agent branching.
Latest indexed changes and source events
Other apps tracked under the same category.
