nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Tue_May_27_02:21:03_PDT_2025 Cuda compilation tools, release 12.9, V12.9.86 Build cuda_12.9.r12.9/compiler.36037853_0 python: 3.13.5 torch: 2.10.0 torch.cuda.is_available: True torch.version.cuda: 12.9 torch.backends.cudnn.version: 91301 allow_tf32_matmul: False allow_tf32_cudnn: True float32_matmul_precision: highest device: NVIDIA H200 NVL sm_count: 132 total_memory_gb: 139.8 Linux h01 4.18.0-553.51.1.el8_10.x86_64 Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 52 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Vendor ID: AuthenticAMD Model name: AMD EPYC 9355 32-Core Processor CPU family: 26 Model: 2 Thread(s) per core: 2 Core(s) per socket: 32 Socket(s): 2 Stepping: 1 Frequency boost: enabled CPU(s) scaling MHz: 79% CPU max MHz: 4413,2319 CPU min MHz: 1500,0000 BogoMIPS: 7099,82 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx_vnni avx512_bf16 clzero irperf xsaveerptr wbnoinvd amd_ppin cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid bus_lock_detect movdiri movdir64b overflow_recov succor smca fsrm avx512_vp2intersect flush_l1d amd_lbr_pmc_freeze Virtualization: AMD-V L1d cache: 3 MiB (64 instances) L1i cache: 2 MiB (64 instances) L2 cache: 64 MiB (64 instances) L3 cache: 512 MiB (16 instances) NUMA node(s): 8 NUMA node0 CPU(s): 0-7,64-71 NUMA node1 CPU(s): 8-15,72-79 NUMA node2 CPU(s): 16-23,80-87 NUMA node3 CPU(s): 24-31,88-95 NUMA node4 CPU(s): 32-39,96-103 NUMA node5 CPU(s): 40-47,104-111 NUMA node6 CPU(s): 48-55,112-119 NUMA node7 CPU(s): 56-63,120-127 total used free shared buff/cache available Mem: 2,2Ti 276Gi 1,0Ti 716Gi 947Gi 1,2Ti Swap: 0B 0B 0B available: 8 nodes (0-7) node 0 cpus: 0 1 2 3 4 5 6 7 64 65 66 67 68 69 70 71 node 0 size: 289874 MB node 0 free: 167786 MB node 1 cpus: 8 9 10 11 12 13 14 15 72 73 74 75 76 77 78 79 node 1 size: 290299 MB node 1 free: 510 MB node 2 cpus: 16 17 18 19 20 21 22 23 80 81 82 83 84 85 86 87 node 2 size: 290299 MB node 2 free: 229734 MB node 3 cpus: 24 25 26 27 28 29 30 31 88 89 90 91 92 93 94 95 node 3 size: 290299 MB node 3 free: 274468 MB node 4 cpus: 32 33 34 35 36 37 38 39 96 97 98 99 100 101 102 103 node 4 size: 290299 MB node 4 free: 316 MB node 5 cpus: 40 41 42 43 44 45 46 47 104 105 106 107 108 109 110 111 node 5 size: 290255 MB node 5 free: 169071 MB node 6 cpus: 48 49 50 51 52 53 54 55 112 113 114 115 116 117 118 119 node 6 size: 290299 MB node 6 free: 3125 MB node 7 cpus: 56 57 58 59 60 61 62 63 120 121 122 123 124 125 126 127 node 7 size: 290236 MB node 7 free: 222867 MB node distances: node 0 1 2 3 4 5 6 7 0: 10 12 12 12 32 32 32 32 1: 12 10 12 12 32 32 32 32 2: 12 12 10 12 32 32 32 32 3: 12 12 12 10 32 32 32 32 4: 32 32 32 32 10 12 12 12 5: 32 32 32 32 12 10 12 12 6: 32 32 32 32 12 12 10 12 7: 32 32 32 32 12 12 12 10 PLATFORM_ID="platform:el8" GPU 0: NVIDIA H200 NVL (UUID: GPU-7fa8e8be-11cd-b316-5ee9-e17285b36fdf) Thu Apr 23 11:02:33 2026 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 575.51.03 Driver Version: 575.51.03 CUDA Version: 12.9 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA H200 NVL On | 00000000:03:00.0 Off | 0 | | N/A 28C P0 68W / 600W | 0MiB / 143771MiB | 0% Default | | | | Disabled | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ ==============NVSMI LOG============== Timestamp : Thu Apr 23 11:02:34 2026 Driver Version : 575.51.03 CUDA Version : 12.9 Attached GPUs : 1 GPU 00000000:03:00.0 Product Name : NVIDIA H200 NVL Product Brand : NVIDIA Product Architecture : Hopper Display Mode : Requested functionality has been deprecated Display Attached : Yes Display Active : Disabled Persistence Mode : Enabled Addressing Mode : None MIG Mode Current : Disabled Pending : Disabled Accounting Mode : Disabled Accounting Mode Buffer Size : 4000 Driver Model Current : N/A Pending : N/A Serial Number : 1320125038453 GPU UUID : GPU-7fa8e8be-11cd-b316-5ee9-e17285b36fdf Minor Number : 0 VBIOS Version : 96.00.B7.00.02 MultiGPU Board : No Board ID : 0x300 Board Part Number : 900-21010-0040-000 GPU Part Number : 233B-894-A1 FRU Part Number : N/A Platform Info Chassis Serial Number : N/A Slot Number : N/A Tray Index : N/A Host ID : N/A Peer Type : N/A Module Id : 4 GPU Fabric GUID : N/A Inforom Version Image Version : 1010.0230.00.02 OEM Object : 2.1 ECC Object : 7.16 Power Management Object : N/A Inforom BBX Object Flush Latest Timestamp : 2026/04/22 18:05:07.525 Latest Duration : 72905 us GPU Operation Mode Current : N/A Pending : N/A GPU C2C Mode : Disabled GPU Virtualization Mode Virtualization Mode : None Host VGPU Mode : N/A vGPU Heterogeneous Mode : N/A GPU Reset Status Reset Required : Requested functionality has been deprecated Drain and Reset Recommended : Requested functionality has been deprecated GPU Recovery Action : None GSP Firmware Version : 575.51.03 IBMNPU Relaxed Ordering Mode : N/A PCI Bus : 0x03 Device : 0x00 Domain : 0x0000 Base Classcode : 0x3 Sub Classcode : 0x2 Device Id : 0x233B10DE Bus Id : 00000000:03:00.0 Sub System Id : 0x199610DE GPU Link Info PCIe Generation Max : 5 Current : 5 Device Current : 5 Device Max : 5 Host Max : 5 Link Width Max : 16x Current : 16x Bridge Chip Type : N/A Firmware : N/A Replays Since Reset : 0 Replay Number Rollovers : 0 Tx Throughput : 773 KB/s Rx Throughput : 631 KB/s Atomic Caps Outbound : N/A Atomic Caps Inbound : N/A Fan Speed : N/A Performance State : P0 Clocks Event Reasons Idle : Active Applications Clocks Setting : Not Active SW Power Cap : Not Active HW Slowdown : Not Active HW Thermal Slowdown : Not Active HW Power Brake Slowdown : Not Active Sync Boost : Not Active SW Thermal Slowdown : Not Active Display Clock Setting : Not Active Clocks Event Reasons Counters SW Power Capping : 130843134691 us Sync Boost : 0 us SW Thermal Slowdown : 0 us HW Thermal Slowdown : 0 us HW Power Braking : 0 us Sparse Operation Mode : Disabled FB Memory Usage Total : 143771 MiB Reserved : 615 MiB Used : 0 MiB Free : 143157 MiB BAR1 Memory Usage Total : 262144 MiB Used : 1 MiB Free : 262143 MiB Conf Compute Protected Memory Usage Total : 0 MiB Used : 0 MiB Free : 0 MiB Compute Mode : Default Utilization GPU : 0 % Memory : 0 % Encoder : 0 % Decoder : 0 % JPEG : 0 % OFA : 0 % Encoder Stats Active Sessions : 0 Average FPS : 0 Average Latency : 0 FBC Stats Active Sessions : 0 Average FPS : 0 Average Latency : 0 DRAM Encryption Mode Current : N/A Pending : N/A ECC Mode Current : Enabled Pending : Enabled ECC Errors Volatile SRAM Correctable : 0 SRAM Uncorrectable Parity : 0 SRAM Uncorrectable SEC-DED : 0 DRAM Correctable : 0 DRAM Uncorrectable : 0 Aggregate SRAM Correctable : 0 SRAM Uncorrectable Parity : 0 SRAM Uncorrectable SEC-DED : 0 DRAM Correctable : 0 DRAM Uncorrectable : 0 SRAM Threshold Exceeded : No Aggregate Uncorrectable SRAM Sources SRAM L2 : 0 SRAM SM : 0 SRAM Microcontroller : 0 SRAM PCIE : 0 SRAM Other : 0 Retired Pages Single Bit ECC : N/A Double Bit ECC : N/A Pending Page Blacklist : N/A Remapped Rows Correctable Error : 0 Uncorrectable Error : 0 Pending : No Remapping Failure Occurred : No Bank Remap Availability Histogram Max : 3072 bank(s) High : 0 bank(s) Partial : 0 bank(s) Low : 0 bank(s) None : 0 bank(s) Temperature GPU Current Temp : 28 C GPU T.Limit Temp : 59 C GPU Shutdown T.Limit Temp : -8 C GPU Slowdown T.Limit Temp : -2 C GPU Max Operating T.Limit Temp : 0 C GPU Target Temperature : N/A Memory Current Temp : 26 C Memory Max Operating T.Limit Temp : 0 C GPU Power Readings Average Power Draw : 68.43 W Instantaneous Power Draw : 68.38 W Current Power Limit : 600.00 W Requested Power Limit : 600.00 W Default Power Limit : 600.00 W Min Power Limit : 200.00 W Max Power Limit : 600.00 W GPU Memory Power Readings Average Power Draw : N/A Instantaneous Power Draw : N/A Module Power Readings Average Power Draw : N/A Instantaneous Power Draw : N/A Current Power Limit : N/A Requested Power Limit : N/A Default Power Limit : N/A Min Power Limit : N/A Max Power Limit : N/A Power Smoothing : N/A Workload Power Profiles Requested Profiles : N/A Enforced Profiles : N/A Clocks Graphics : 345 MHz SM : 345 MHz Memory : 3201 MHz Video : 765 MHz Applications Clocks Graphics : 1785 MHz Memory : 3201 MHz Default Applications Clocks Graphics : 1785 MHz Memory : 3201 MHz Deferred Clocks Memory : N/A Max Clocks Graphics : 1785 MHz SM : 1785 MHz Memory : 3201 MHz Video : 1410 MHz Max Customer Boost Clocks Graphics : 1785 MHz Clock Policy Auto Boost : N/A Auto Boost Default : N/A Voltage Graphics : Requested functionality has been deprecated Fabric State : N/A Status : N/A CliqueId : N/A ClusterUUID : N/A Health Bandwidth : N/A Route Recovery in progress : N/A Route Unhealthy : N/A Access Timeout Recovery : N/A Processes : None Capabilities EGM : disabled GPU0 NIC0 NIC1 NIC2 NIC3 CPU Affinity NUMA Affinity GPU NUMA ID GPU0 X SYS SYS SYS SYS 12-13,76-77 1 N/A NIC0 SYS X PIX SYS SYS NIC1 SYS PIX X SYS SYS NIC2 SYS SYS SYS X PIX NIC3 SYS SYS SYS PIX X Legend: X = Self SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI) NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU) PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge) PIX = Connection traversing at most a single PCIe bridge NV# = Connection traversing a bonded set of # NVLinks NIC Legend: NIC0: mlx5_0 NIC1: mlx5_1 NIC2: mlx5_2 NIC3: mlx5_3