Large GPU jobs fail for many small infrastructure reasons, so reliability has to be continuous rather than reactive. Netflix built a layered system that validates nodes before jobs start, monitors hardware during runs, and automatically drains, reboots, resets, or replaces bad machines. The result is more usable GPU capacity and fewer long training or inference runs lost to platform faults.