vLLM on Google Cloud TPU: A Model Size vs Chip Cheat Sheet (With Interactive Tool)

Picking a Cloud TPU slice for vLLM inference involves three decisions that most tutorials skip over: how much HBM your model actually needs at runtime, how many chips to shard across, and whether the cost is justified for your workload. Get it wrong in either direction and you're either OOMing on st