Supermicro V100 rig eyes local LLMs

This is a clever salvage-build idea, but it is much more likely to become a fun hardware project than a clean cost/perf king.

• –Supermicro’s own docs show the SYS-4028GR-TVRT as the 8x Tesla V100 SXM2, 300 GB/s NVLink box; TXRT is the P100-era sibling, and TRT is a PCIe-gpu chassis.
• –If you end up with PCIe V100s instead of SXM2 modules, the whole NVLink premise changes, so the exact GPU form factor matters as much as the price.
• –128 GB of aggregate VRAM sounds huge, but it is still sharded memory across eight cards, so model parallelism, interconnect efficiency, and software support will decide real-world speed.
• –Custom water cooling can make the thermals work, but it also adds leak risk, maintenance headaches, and another layer of failure on top of already old enterprise hardware.
• –The value case is strongest if you want a tinkering lab and can tolerate rough edges; if you want a low-friction local inference box, simpler modern hardware is usually the saner buy.