Blogs -NVIDIA’s GB200s for up to 27 Trillion Parameter Models: Scaling Next-Gen AI Superclusters

NVIDIA’s GB200s for up to 27 Trillion Parameter Models: Scaling Next-Gen AI Superclusters


March 21, 2025

author

I/O Fund

Team

Supercomputers and cutting-edge AI data centers are fueling the artificial intelligence (AI) revolution. Large-scale systems need comprehensive builds that are increasingly integrated to meet the evolving demands of complex workloads. As AI applications become more sophisticated, the need for infrastructure that's not only incredibly powerful but also energy-efficient is growing exponentially. Innovations like NVIDIA’s GB200 are designed to deliver the scalability needed for next-generation AI superclusters.  

At the 2025 NVIDIA GPU Technology Conference (GTC), VP and Chief Architect of Systems, Mike Houston, and Senior Director of Applied Systems Engineering, Julie Bernauer, discussed large-scale systems design principles in their May 18 presentation, “Next-generation at Scale Compute in the Data Center.”

NVIDIA’s First Rack-Scale Product is the GB200 Superchip

The NVIDIA Grace Blackwell 200 (GB200) Superchip combines two Blackwell GPUs and one Grace CPU. It’s NVIDIA’s first rack-scale product. The NVIDIA GB200 NVL72 is a configuration and rack-scale, liquid-cooled AI computing platform, which is purpose-built for AI training and inferencing, handling up to 27 trillion parameters for generative AI models. The GB200 includes base components like Grace Hopper compute trays, NVLink switches (a connector in the middle of the rack linking all GPUs) and cable cartridges (literally miles of cables in the back to tie everything together). The design includes quantum switches for InfiniBand (a high-speed network for linking clusters) and spectrum switches for Ethernet.

AI 101: What are Clusters and Superclusters?

Clusters 101: are a network of independent computers (called nodes) connected by a high-speed network. A cluster serves as a unified resource, as they are separate machines configured to work together to act as a single powerful computing system. They are often used for parallel processing, which breaks down a large task into smaller parts distributed across the nodes, enabling faster processing than just a single computer could do. A key benefit of a node is high availability, meaning if one node (computer) fails, the other nodes can take over its workload, ensuring that the system remains operational. High-performance compute (HPC) clusters are used for tasks like research, scientific simulations and AI training.

Join thousands of investors who trust I/O Fund’s expert stock analysis on AI, semiconductors, cryptocurrency, and adtech — sign up for free! Click here!

Superclusters 101: are very large clusters that may be comprised of hundreds to thousands of GPUs through many data centers. For example, Elon Musk’s xAI supercomputer Colossus, powered by 100,000 NVIDIA GPUs, is definitely a supercluster.

DGX started as single machines for AI but evolved into clusters for AI training. Pre-training can involve superclusters, but post-training can still involve 16,000 GPUs with smaller setups for fine-tuning and inference using trained AI to answer questions.

NVIDIA AI and HPC platform architecture diagram featuring the GB200 NVL72 SuperPOD, designed for large-scale AI training and high-performance computing.

NVIDIA AI & HPC Platform architecture diagram, featuring GB200 NVL72 SuperPOD.
Source: NVIDIA

Optimizing the Benefits of Rack-Scale Architecture with GB200

NVIDIA’s GB200 NVL72 is a rack-scale system. Rack-scale designs a whole rack as one big, coordinated unit, not just random machines stuck together. Rack scale refers to integrating and compressing systems that may span across multiple servers, storage and networking devices onto a single server rack. GB200 can replace or consolidate a large number of GPU compute servers. This provides many benefits, including:

  • Improved GPU Density: The GB200 NVL72 contains 72 Blackwell GPUs, and 36 Grace CPUs interconnected with NVLink, NVIDIA’s proprietary high-speed (130 TB/s) signaling interconnect that enables all 72 GPUs and 36 CPUs to act as a single massive GPU. It's designed to offer exceptional performance in AI training and inference for large language models (LLMs).
  • Performance: The GB200 delivers up to 720 petaFLOPs for AI training and 1.4 exaFLOPs for inference. Since all components are within proximity in a single rack, communication between components has much lower latency, which is especially beneficial in data-intensive tasks, reducing bottlenecks and improving data throughput.
  • Increased Efficiency: Rack-scale architecture allows for better utilization of hardware by pooling resources to optimize performance. Consolidating resources within a single rack reduces the need for separate units, saving space and power in the data center.
  • Easier Management: Centralized management of the entire rack's resources simplifies setup and maintenance, also enabling automation tools for scaling, provisioning and monitoring to reduce manual interventions.
  • Cost Efficient: Fewer servers, storage, networking equipment, physical space, cooling, and energy usage save money. As IO Fund discussed in its article “AI Power Consumption: Rapidly Becoming Mission-Critical," the GB200 is “expected to consume 2,700W”, which can add dramatically to operating expenses, especially without rack-scale architecture.
  • Future Proofing: Rack-scale architecture enables the integration of evolving technologies as components can be switched out, repaired and upgraded, enabling more adaptability for future growth.
  • Unified Power and Cooling: Housing multiple components within a single rack reduces the complexity of cooling systems and improves energy efficiency to lower operational costs.

Scaling Up AI Factories with DGX SuperPOD, Reference Architecture and Fabric

At the 2025 NVIDIA GPU Technology Conference (GTC), NVIDIA unveiled its next-generation DGX SuperPOD AI infrastructure. In the “Next-generation at Scale Compute in the Data Center” presentation, VP and Chief Architect of Systems, Mike Houston, and Senior Director of Applied Systems Engineering, Julie Bernauer, spoke about

The SuperPOD is NVIDIA’s all-in-one HPC solution designed to handle the massive computational needs of AI models and simulations. Grace Blackwell nodes are the building blocks of the SuperPOD. When scaling up clusters and superclusters, there are three factors to consider. Reference architecture is comprised of pre-tested system designs that serve as a blueprint for new data center deployments to ensure optimal installation and performance, accelerating time to the first token.

Fabric refers to the data center’s network infrastructure that connects all the servers and devices enabling them to seamlessly communicate with each other to reduce latency between components, especially GPUs. Cooling is critical in large data centers. Liquid cooling is preferred to manage the heat produced by thousands of GPUs as it is much more efficient for high-density platforms. Future GPU architectures aim for higher density and more efficient connectivity to push the limits of AI computation.

The I/O Fund recently entered five new small and mid-cap positions that we believe will be beneficiaries of this AI spending war. We discuss entries, exits, and what to expect from the broad market every Thursday at 4:30 p.m. in our 1-hour webinar. For a limited time, get $110 off an Annual Pro plan with code PRO110OFF [Learn more here.]

Disclaimer: This is not financial advice. Please consult with your financial advisor in regards to any stocks you buy.

Recommended Reading:

head bg

Get a bonus for subscription!

Subscribe to our free weekly stock
analysis and receive the "AI Stock: 5
Things Nobody is Telling you" brochure
for free.

More To Explore

Newsletter

S&P 500 hits key 2025 target as bond market and consumer trends signal rising volatility and shifting stock-bond correlation.

2025 Market Outlook: Why Stocks and Bonds Are Signaling More Volatility

As the S&P 500 reaches a key bounce target, troubling signs in bonds and consumer behavior suggest this market rally may be on thin ice. I/O Fund’s Knox Ridley explains why volatility may intensify an

May 02, 2025
Illustration of an investor holding an umbrella, shielding from stock market volatility.

The Impact of Tariffs on the Stock Market: Q1 Preview

Rising tariffs are injecting significant uncertainty into the stock market, triggering daily volatility and forcing analysts to revise earnings estimates. Our Q1 preview dives into the potential impac

April 25, 2025
Aerial view of Tesla's new Model Y Juniper parked in lines. Courtesy of Tesla, Inc.

Tesla Stock Faces Recalibration of Growth Expectations

Tesla’s stock is now facing a recalibration of expectations after Q1’s delivery report missed by a wide margin. Q1’s analyst consensus has gone from $25.98B at the start of the year to $23.97B in earl

April 17, 2025
The bond market could break the stock market in 2025, as explored in ‘The Fed Can’t Save This One’ article.

The Fed Can’t Save This One: Why Bonds May Break the Stock Market in 2025

In early 2025, as markets rallied to new highs, we warned that divergence across key sectors signaled a looming correction. Now, with all major indexes in a technical bear market and bond market dysfu

April 11, 2025
Silhouette illustration of Larry Ellison, Oracle's CTO and executive chairman.

Oracle Stock Outlook: Revenue Could Double by FY2029, yet Targets Seem Lofty

Late in 2024, Oracle outlined an ambitious plan to nearly double its revenue by fiscal 2029, hinging on long-term growth in enterprise AI and cloud spending. Oracle sets itself apart from its hypersca

April 04, 2025
I/O Fund reports a 210% cumulative return, surpassing top tech ETFs and institutional portfolios with a 35% gain in 2024. Source: YCharts and InsiderMonkey.

I/O Fund Reports 210% Cumulative Return -- Ranking Above Wall Street's Best

In 2024, I/O Fund posted a 35% return, significantly outperforming popular tech ETFs, which recorded an 8% return over the same period. On a cumulative basis, the results translate to a remarkable 219

March 31, 2025
Chart showing retail investor losses compared to institutional investors, highlighting market volatility and the impact of high-frequency trading.

The Harsh Truth: Retail Investors Take the Brunt of Market Losses

Retail investors face significant disadvantages in the stock market, often underperforming institutional investors by a wide margin. Studies show that high-frequency trading firms dominate market acti

March 28, 2025
Illustration of a futuristic AI data center featuring NVIDIA’s GB200 Superchip

NVIDIA’s GB200s for up to 27 Trillion Parameter Models: Scaling Next-Gen AI Superclusters

Supercomputers and advanced AI data centers are driving the AI revolution, enabling breakthroughs in deep learning and large-scale model training. As AI workloads become increasingly complex, next-gen

March 21, 2025
NVIDIA Blackwell Ultra GPU unveiled at GTC 2025, revolutionizing AI and HPC with unprecedented efficiency and power.

NVIDIA Blackwell Ultra Fuels AI & HPC Innovation, Efficiency and Capability  

NVIDIA’s latest Blackwell Ultra GPU, unveiled at NVIDIA GTC 2025, is transforming AI acceleration and high-performance computing (HPC). Designed for the “Age of Reasoning,” these cutting-edge GPUs del

March 21, 2025
Nvidia CEO Jensen Huang discusses AI market dominance at GTC 2025, addressing demand concerns and future growth projections.

Nvidia CEO Predicts AI Spending Will Increase 300%+ in 3 Years

Nvidia has traversed choppy waters so far in 2025 as concerns have mounted about how the company plans to sustain its historic levels of demand. At GTC, Huang threw cold water on many of the Street’s

March 20, 2025
newsletter

Sign up for Analysis on
the Best Tech Stocks

https://bethtechnology.cdn.prismic.io/bethtechnology/e0a8f1ff-95b9-432c-a819-369b491ce051_Logo_Final_Transparent_IOFUND.svg
The I/O Fund specializes in tech growth stocks and offers in-depth research for Premium Members. Investors get access to a transparent portfolio, a forum, webinars, and real-time trade notifications. Sign up for Premium.

We are on social networks


Copyright © 2010 - 2025