What size model can 141GB of H200 memory handle?
One H200 with 141GB can roughly serve a 70B-parameter model in FP16. An 8-GPU system with about 1.1TB total memory supports 400B-class models for multi-GPU inference or mid-scale training. With FP8/FP4 quantization, tensor parallelism and longer context, the practical model size changes and should be calculated from the model structure.
