Is Amphenol Overpass Replaced by PCB? – All you need to know about GB200 NVSwitch tray redesign issue
Recently, there has been widespread rumor about the need to redesign the GB200 NVSwitch tray. Without further ado, let's briefly outline the context of the entire incident below.
The Background:
The ODM (Foxconn) responsible for manufacturing the NVSwitch tray for GB200 feedbacked to NVIDIA that after filling the tray with all the overpasses, the liquid cooling pipes were difficult to arrange within the limited space. Additionally, if a cable within the overpass failed, it was extremely difficult to pinpoint which cable was problematic, hence requiring the replacement of the entire overpass, and replacing the overpass is more expensive than replacing the PCB.
The New Design Plan:
The overpasses in the NVSwitch tray are divided into two different cables. The red one is connected to the compute trays within the same rack, while the blue one is connected to the NVSwitch trays in adjacent racks (see the image below). The blue cables will be removed in the new NVSwitch tray design, leaving only the red cables. By upgrading the PCB CCL material and reducing the overpasses by half, the liquid cooling pipes can be laid out smoothly, solving the heat dissipation issue.
The NVSwitch tray PCB will be changed from a 24-layer M7 HDI board to a 22-layer M8 through-hole board (PTH). Originally, the HDI board was primarily supplied by Unimicron (60%) and secondary supplied by Victory Giant (40%). The new through-hole board will be primarily supplied by Wus Printed Circuit (60%) and secondary supplied by TTM Technologies (40%).
Why Change the PCB Suppliers?
Unimicron, the original primary supplier, has experience with HDI but lacks experience with multi-layer through-hole board.
Wus and TTM are already the primary and secondary suppliers of NVIDIA's switch PCBs. The new NVSwitch tray board is very similar to the switch board, so NVIDIA prefers to continue using the same suppliers.
The Timeline:
July: NVIDIA began discussions on the modification plan.
Early September: The first version of the new design board released.
By the end of October: Expected to complete verification of the new PCB board.
ODM needs two months to assemble the system, so the earliest time the new GB200 system can be delivered to customers for validation is towards the end of December.
If all goes well, mass production of the GB200 system can start after the Chinese New Year, around late February/early March next year.
The Benefiting Companies:
EMC (2383 TT): The exclusive CCL supplier for the GB200 NVSwitch tray. The original M7 CCL cost $2000~$2500 per piece, while the new M8U CCL costs $4000~$4500 per piece.
Nitto Boseki (3110 JP): The original M7 CCL (890k) used the first-generation low-dk fiberglass yarn (NE glass), while the new M8U CCL (892k2) uses the second-generation low-dk fiberglass yarn (NER glass). The "U" in M8U stands for Ultra, and the "k2" in 892k2 refers to the second-generation low-dk glass. Upgrading from NE glass to NER glass doubles the glass fiber ASP. (For more details, please refer to my previous article on Nittobo:Nittobo (3110 JP) -- The King of Electronic Fiber Glass)
Union Tool (6278 JP): The designated mechanical drill supplier for NVIDIA. While HDI vias are generally made using laser drilling, through vias in through-hole boards (PTH) can only be made using mechanical drilling. This is because:
Laser-drilled vias naturally form a trapezoidal shape, wider at the top and narrower at the bottom, but through-hole vias require consistent diameter throughout the entire via, which can only be achieved with mechanical drilling.
Through-hole vias need to pass through fiberglass layers, which cannot be drilled with lasers. Different materials deform at different rates when heated (i.e. different thermal expansion coefficients), and the high heat generated by laser drilling can cause uneven hole diameters when passing through different materials. The only disadvantage of mechanical drilling is that it cannot achieve as small a diameter as laser drilling. However, Union Tool's high-end mechanical drill product (ULF) can achieve a diameter of 0.2 mm, almost comparable to laser drilling, which is why they were favored by NVIDIA.
Wus Printed Circuit (2316 TT) & TTM Technologies (TTMI US): The primary and secondary PCB suppliers for the new GB200 NVSwitch tray.
The Negatively Affected Companies:
Unimicron (3037 TT): The company lacks experience in producing multi-layer through-hole boards.