Quick Take
要点速览： GPU RFQs that list only servers omit the fabric that makes multi-node training work. A complete InfiniBand BOM needs QM9700 leaf/spine count, ConnectX-7 HCA OPNs, an OSFP DAC/AOC matrix, correct NS2F/NS2R airflow, and spares. Size host ports first, then freeze the cable plan before PO — fabric lead times often trail GPU crates.
Why InfiniBand Fabric BOMs Fail Before Go-Live
速答： Most failures are incomplete lines, not wrong GPU models: missing HCAs, wrong airflow SKUs, undersized spine uplinks, or cables ordered after racks are already bolted. Freeze node count and ports-per-node first, then size switches and optics as one package. Treat fabric lines as critical path equal to GPU nodes — incomplete networking BOMs strand capital on the dock.
Search interest for NVIDIA Quantum-2 QM9700 and ConnectX-7 remains high because buyers who already secured H100, H200, B200, or B300 allocation suddenly discover the cluster still needs a non-blocking NDR fabric. Ordering nodes without a matched InfiniBand BOM is one of the fastest ways to miss go-live: switches, HCAs, and OSFP cables have their own ATP and length constraints.
This guide is a procurement-oriented fabric BOM playbook. It complements our AI cluster sizing guide, B300 deployment SOP, and rack power and cooling planning. Shanghai TaoHan Industrial Co., Ltd. supplies NVIDIA QM9700, ConnectX-7, and related GPU infrastructure with configuration support — request a free fabric BOM risk assessment.
| Failure pattern | What happens | BOM fix |
|---|---|---|
| Nodes-only RFQ | Crates arrive; no HCAs or leafs | Require fabric lines on every GPU quote |
| Wrong airflow | NS2F ordered for C2P hot aisle | Lock NS2F (P2C) vs NS2R (C2P) per row |
| Leaf without spine | Multi-rack scale-out saturates uplinks | Size spine ports for target oversubscription |
| Cable after install | Wrong length / missing splitters | Publish OSFP matrix with rack elevations |
QM9700 Building Blocks Buyers Must Spec
速答： Spec each Quantum-2 leaf or spine with the full OPN (managed), airflow direction, power cords, rail kit, and software entitlement path. Do not treat "QM9700" as one purchasable line — airflow and management options change the SKU and the lead time.
Port geometry
Each QM9700 exposes 64×400Gb/s NDR ports through 32 OSFP connectors (two logical ports per cage). Designs that need denser 200G fan-out use NVIDIA-qualified splitter cables to present up to 128×200G — only when the HCA and topology support it.
Airflow is a hard SKU choice
NS2F power-to-connector (P2C / forward) and NS2R connector-to-power (C2P / reverse) are not interchangeable in the field without a return. Match hot-aisle / cold-aisle orientation before RFQ, or you will burn weeks on RMA.
Management and fabric services
Plan MLNX-OS management access, out-of-band Ethernet, and optional UFM (Unified Fabric Manager) licensing as separate BOM lines. Telemetry and job-aware congestion control are not free add-ons you invent after cutover.
ConnectX-7 Selection for GPU Nodes
速答： Order InfiniBand NDR ConnectX-7 OPNs that match the server's PCIe generation and slot count. Dual-port cards simplify cabling for dense nodes; confirm each SKU is InfiniBand — Ethernet-only ConnectX-7 variants will not attach to QM9700 as host channel adapters. Cross-check every OPN suffix against your BOM before signing.
| Decision | What to lock in the BOM | Why it matters |
|---|---|---|
| Protocol | InfiniBand NDR (not Ethernet-only) | Wrong OPN = dead leaf port |
| Ports | Single vs dual-port HCA | Affects cable count and redundancy |
| Form factor | PCIe HHHL / FHHL / OCP per chassis | Physical fit and thermal clearance |
| Quantity | HCAs × nodes (+ hot spares) | Fabric cannot absorb missing adapters |
For multi-rail designs (common in DGX / HGX-class nodes), count every fabric rail as a separate HCA port in the leaf math. Related GPU selection context: B300 vs B200 and B300 vs H100.
Leaf / Spine Sizing From Node Count
速答： Host ports ÷ usable leaf downlinks ≈ leaf count; then size spine ports for your oversubscription target (often 1:1 for training fabrics). Always reserve management, ISL, and growth ports — a "full" 64-port math on paper fails on day one.
Step 1 — Host port demand
host_ports = nodes × IB_ports_per_node. Example: 32 nodes × 1×400G HCA = 32 host ports; 32 nodes × 8 rails is a different fabric class entirely.
Step 2 — Leaf capacity
On each QM9700, reserve uplinks to spine (and optional ISLs). If 48 ports are downlinks and 16 are uplinks, one leaf supports 48 host ports at that ratio — not 64.
Step 3 — Spine and growth
Spine count follows uplink ports × leaves, divided by spine downlink budget, at the chosen non-blocking ratio. Add at least one growth leaf slot in the BOM narrative even if you do not buy it yet — ATP for Quantum-2 can lag GPU delivery.
| Cluster sketch | Host ports (example) | Illustrative leaf need* |
|---|---|---|
| 8-node pilot, 1×400G each | 8 | 1 leaf (oversized but simplest) |
| 32-node row, 1×400G each | 32 | 1 leaf if uplinks fit; else 2 |
| 64-node pod, 1×400G each | 64 | 2+ leaves + spine |
| Multi-rail HGX / DGX | nodes × rails | Leaf/spine math scales with rails |
*Illustrative only — final counts depend on uplink ratio, dual-homing, and management ports. Share your node sketch for a validated BOM.
OSFP Cable and Optics Matrix
速答： Every OSFP cage needs a typed length: short DAC inside the rack, ACC/AOC for row spans, and optics only when reach demands it. Publish a matrix keyed by rack elevation before PO — cable rework after GPU install is expensive and slow.
| Link type | Typical use | BOM note |
|---|---|---|
| OSFP DAC | In-rack leaf to HCA | Cheapest; length-limited |
| OSFP ACC / AOC | Row / aisle spans | Match NVIDIA-qualified lengths |
| Optical OSFP | Longer reach / MMP | Budget DOM/FEC validation time |
| Splitter (400→2×200) | Density fan-out | Only with compatible HCAs |
Count cables as host links + leaf-to-spine uplinks + spares (typically 5–10%). Missing five DACs can idle an entire leaf just as effectively as a missing switch.
Lead-Time and Compatibility Risks in 2026
速答： Fabric ATP often trails GPU crates. Parallel-order switches, HCAs, and cables; verify InfiniBand OPNs; lock airflow; and keep a spare kit. Treat quote windows like any other enterprise hardware — do not assume fabric pricing holds while compute quotes move.
- Allocation skew: GPU and Quantum-2 lines can clear on different calendars — place fabric POs in the same approval cycle as nodes.
- OPN confusion: Ethernet ConnectX-7 and InfiniBand ConnectX-7 share branding; the wrong line item will not train.
- Airflow RMA: NS2F / NS2R mistakes consume the same calendar time as a new factory order.
- Optics policy: Support contracts may require NVIDIA-qualified modules — document the policy before substituting.
- Power / cooling dependency: High-density GPU rows still need facility readiness; see the power and cooling guide.
Buyer Checklist Before You RFQ
速答： Attach node count, ports per node, leaf/spine sketch, airflow map, and OSFP length matrix to the RFQ package. Incomplete fabric packages get incomplete quotes — and incomplete quotes miss go-live even when GPU crates already cleared customs. Attach rack elevations and length assumptions so cable SKUs do not become a second RFQ after hardware ships.
- Node model and quantity frozen (H100 / H200 / B200 / B300 / other)
- InfiniBand ports per node and dual-homing policy written
- QM9700 leaf and spine counts with uplink oversubscription target
- NS2F vs NS2R decided per rack row
- ConnectX-7 OPN class (IB NDR, form factor, single/dual) listed
- OSFP DAC / ACC / AOC / optics matrix with lengths
- Management (OOB, UFM) and spare kit percentages included
- Go-live date and acceptable ATP risk noted
真实应用场景
速答： Map the three scenarios below to your workload, facility, and commercial constraints before freezing CapEx. Equal-BOM RFQs should cite which scenario drives the shortlist.
Scenario 1: Multi-Thousand-GPU LLM Pretrain (Collective Bound)
Workload Profile: Dense AllReduce / All-to-All phases dominate step time across thousands of GPUs.
Architectural Fit: Quantum-2 InfiniBand with ConnectX-7 NICs still wins when congestion control and credit-based flow are non-negotiable; RoCEv2 needs mature PFC/ECN ops maturity.
Scenario 2: Ethernet-Standardized Campus DC (Ops Bound)
Workload Profile: NetOps team only staffs Ethernet; IB skill is scarce.
Architectural Fit: 800G RoCEv2 can win on staffing—if the BOM funds lossless tuning, telemetry, and buffer sizing, not just switch stickers.
Scenario 3: Hybrid Inference + Train Pods (Topology Bound)
Workload Profile: Train fabric must stay IB while inference fronts sit on Ethernet.
Architectural Fit: Keep fabrics separate in the RFQ; do not force one overlay to serve both latency profiles.
常见问题
速答： A fabric BOM is switches + HCAs + cables sized to node ports. QM9700 offers 64×400G NDR via 32 OSFP cages. ConnectX-7 InfiniBand OPNs attach to those leaves. Most outages come from incomplete lines or wrong airflow — not from GPU FLOPS.
A complete bill of materials for an NVIDIA Quantum-2 fabric: QM9700 switches, ConnectX-7 host adapters, OSFP DAC/AOC or optics, management accessories, and spares — sized to GPU node count and the non-blocking target.
64 non-blocking 400Gb/s NDR ports via 32 OSFP cages. Qualified splitter cables can present up to 128 ports at 200Gb/s when the design requires fan-out.
Yes — InfiniBand NDR ConnectX-7 HCAs are the standard attachment. Confirm the OPN is InfiniBand (not Ethernet-only), matches the server form factor, and covers every fabric rail you intend to use.
Yes whenever possible. Fabric ATP and cable custom lengths often trail GPU crates. Parallel approval avoids idle capital sitting in crates while networking is still on order.
Submit node count, ports per node, and any draft switch list via 提交 BOM / Contact TAOHAN or email info@021taohan.com. You receive a validated QM9700 / ConnectX-7 / cable package with availability signals from a global free-port hub.
Key Takeaway
核心结论： Price the fabric as one BOM (switches, HCAs and cables together) rather than three line items appended after the GPU nodes. Freeze host port count first, specify NS2F or NS2R before the RFQ, publish the OSFP matrix, and keep fabric ATP inside the compute approval cycle. Shanghai TaoHan Industrial Co., Ltd. supplies NVIDIA QM9700 switches and ConnectX-7 HCAs with competitive pricing.