Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM

This technical guide explains how to serve the 753-billion parameter GLM-5.3 model across two Nvidia DGX stations using vLLM and tensor parallelism. It details the hardware requirements and configuration needed to manage model weights that exceed the memory capacity of a single GPU.
Why it matters
As AI models grow in size, distributed inference techniques are becoming essential for developers and researchers working with frontier-scale models.
Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations connected over ConnectX-8 RDMA serve one model through a single endpoint, with tensor parallelism splitting the model weights and computation across both GPUs.
The NVIDIA GLM-5.3 NVFP4 model, a quantized version of Z.ai ’s mixture-of-experts (MoE) model, has 753 billion parameters, with 40 billion activated per token. Sparse activation reduces computation, but all model weights still need to remain available because each token can be routed to different experts. At about 464 GB, the checkpoint exceeds the 252 GB of HBM3e on a single GB300 GPU. This tutorial uses two DGX Stations to keep the weights in GPU memory, with CPU weight offload disabled.
Before starting, connect the two Stations with the Connect Two DGX Stations playbook .
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in