InferX Compute Sizing

Deployment

Federation tier

GPU compute node

Cooling

Sizing basis

GPU nodes

Compute racks

Workload

Model

Precision

Concurrent requests

Prompt tokens

Output tokens

Service level

Time per output token (TPOT) target

Caps batch size. 50 ms is roughly 20 tokens/sec per user — comfortable for chat.

Time to first token (TTFT) target

1000

Tensor parallel width

Storage

WEKApod Nitro nodes POSIX/S3/AI tier. 8 per the regional design, 24 in the 2 MW core design. Size capacity on the storage page.