How AI Data Centers Work: GPUs, HBM and Networking Explained

AI data center construction plans are appearing around the world at an extraordinary pace.

Amazon, Microsoft, Google, Meta and other technology giants are pouring enormous amounts of capital into AI infrastructure, while governments increasingly view data centers and the energy systems behind them as matters of national competitiveness.

J.P. Morgan estimates that capital spending by the five largest U.S. hyperscalers will reach about $697 billion in 2026.

But there is a growing gap between announcing an AI data center and actually bringing one online. Buying GPUs is only the beginning. An AI data center also needs high-bandwidth memory, ultra-fast networking, storage, enormous amounts of electricity, cooling infrastructure, substations, transformers, financing, permits and, increasingly, support from local communities.

The bottleneck in AI is moving from chips into the physical world.

What Makes an AI Data Center Different?

Traditional data centers have long supported websites, email, databases, cloud storage and enterprise applications. AI data centers face a different level of computational intensity.

Training a large AI model can require hundreds or thousands of GPUs working together as one enormous computing system. And the computation does not stop after training. Every time someone asks an AI assistant a question, summarizes a document, generates an image or creates a video, another round of computation takes place. This is known as inference.

JLL expects AI could account for roughly half of all data center workloads by 2030, with inference becoming the primary driver of AI-related data center growth.

Everyday use of AI is itself becoming a major driver of data center demand.

1. GPUs — The Computing Engines of AI

GPUs perform much of the heavy computation inside modern AI data centers. CPUs are designed to handle a broad range of tasks, while GPUs are particularly effective at performing large numbers of similar mathematical operations in parallel. That makes them well suited to the matrix calculations used by large AI models.

But faster GPUs do not automatically produce an equally faster AI system. Even the most powerful processor has to wait if the data required for its next calculation does not arrive in time. That is where HBM becomes critical.

2. HBM — Keeping Expensive GPUs Fed With Data

HBM stands for High Bandwidth Memory. A simple way to understand its role is to imagine a GPU as an extremely fast chef. A brilliant chef cannot work at full speed if ingredients arrive too slowly. HBM keeps large amounts of data close to the processor and delivers it at very high bandwidth, reducing the time the GPU spends waiting.

As AI models grow, the amount of data moving through the system also grows. AI chip competition is therefore not simply a race for more raw computing power. The ability to deliver enormous amounts of data to those processors quickly is also a major part of performance.

3. Networking — Turning Thousands of GPUs Into One System

Large AI models cannot always be processed efficiently by a single GPU. Hundreds or thousands of accelerators may divide the same workload and continuously exchange data. If one GPU finishes its calculation but has to wait for information from another, valuable computing capacity sits idle.

AI data centers therefore require networks with extremely high bandwidth and very low latency. Technologies such as high-speed Ethernet and InfiniBand play a critical role in allowing massive clusters of GPUs to operate as one computing system.

The importance of connectivity is already visible in the business opportunities surrounding AI infrastructure. Reuters reported that STMicroelectronics expects roughly 80% of its more than $2 billion in AI data center revenue in 2027 to come from chips used in fiber-optic data links.

The Core Structure of an AI Data Center

Component	Main Role	What Happens When It Becomes a Bottleneck
GPU	Performs AI computation	Processing capacity falls
HBM	Delivers high-speed data to GPUs	Expensive GPUs spend more time waiting
Networking	Moves data between GPUs and servers	Communication delays slow the cluster
Storage	Stores and supplies models and data	Data access becomes slower
Power	Keeps the facility operating	Expansion or operation is restricted
Cooling	Removes heat from high-density hardware	Performance and operating capacity are limited

The performance of an AI data center depends on how efficiently the entire system works together. A bottleneck in one part can leave expensive capacity elsewhere underused.

4. Power Is Becoming a Real-World Bottleneck

The rapid expansion of AI data centers is placing new pressure on electricity systems. According to the International Energy Agency, global data center electricity consumption is projected to more than double to around 945 TWh by 2030, slightly more than Japan consumes today. The IEA also expects data centers to account for nearly half of U.S. electricity-demand growth through 2030.

But generating enough electricity is only part of the challenge. Transmission lines, substations, transformers and distribution infrastructure all have to be built or upgraded. The IEA notes that while a data center can become operational in roughly two to three years, the broader energy system often requires much longer planning and construction timelines.

AI grows at software speed. Data centers are built at the speed of power plants and cities.

5. Data Centers Can Be Built Faster Than They Can Get Power

JLL’s 2026 Global Data Center Outlook shows that the average wait for a grid connection in primary data center markets now exceeds four years. By comparison, JLL reports an average global construction time of about 18 months for a 50 MW data center.

In other words, a company can potentially build the facility faster than it can secure the electricity required to operate it. This is changing how developers choose locations. JLL describes “speed to power” as the primary criterion driving data center site selection.

6. The 700 GW Problem and “Ghost Demand”

The rush to secure electricity has created another problem. Reuters reported that U.S. utilities have received data center power requests totaling more than 700 GW — more than ten times the estimated current power consumption of all U.S. data centers.

Texas halted new grid connections for data centers while authorities began examining whether proposed projects had credible ownership, financing and the ability to proceed. Similar measures have appeared elsewhere as utilities attempt to separate serious projects from speculative applications. The phenomenon has become known as “ghost demand.”

An announced data center is not the same thing as a data center that will actually be built. An investment announcement, a request for electricity, the start of construction, the installation of AI hardware and commercial operation are separate stages. Operational capacity is therefore becoming a more meaningful measure than announced capacity.

7. South Korea Faces the Same Power Challenge

The pressure is not limited to the United States. In September 2026, South Korea’s energy minister estimated that new AI data centers and the expansion plans of Samsung Electronics and SK Hynix could add 25 to 30 GW of electricity demand, roughly comparable to the output of around 20 nuclear reactors, according to Reuters.

The scale of the estimate shows why AI policy can no longer be separated from energy policy. Countries that want to become AI infrastructure hubs will increasingly have to plan computing capacity and electricity capacity together.

8. More Computing Power Also Means More Heat

Power creates another physical constraint: heat. As AI servers become more powerful and rack density rises, cooling them becomes more difficult.

Traditional data centers have relied heavily on air cooling, but increasingly dense AI systems are accelerating the adoption of liquid cooling and other advanced thermal-management technologies. Pipes, racks, electrical systems and even parts of the building have to be designed around much higher computing density.

Cooling can no longer be treated as something to solve after the servers arrive. It is becoming part of the original architecture of the AI data center.

9. Cutting-Edge AI Can Still End Up Waiting for a Transformer

Even after grid access is secured, developers still need transformers, generators, switchgear, UPS systems and cooling equipment. According to JLL’s 2026 Global Data Center Outlook, the average lead time for data center equipment is about 33 weeks globally. In the United States, the average is around 42 weeks — 83% longer than in 2019.

JLL also found that 57% of data center projects experienced construction delays of three months or more in 2025, while some developers are ordering critical materials as much as 24 months in advance.

There is an irony here. The world’s most advanced AI infrastructure can still be delayed by equipment such as transformers, switchgear and cooling systems. The technology may be digital, but its constraints are increasingly physical.

10. AI Data Centers Have Become a Massive Financing Business

AI data centers are not only technology projects. They are also enormous financial projects. JLL estimates that up to $3 trillion of investment will be required to support the global data center expansion through 2030.

Financing becomes more difficult when power availability, permitting or equipment delivery remains uncertain. J.P. Morgan identifies power availability, supply-chain constraints and permitting timelines as major execution risks that can materially extend project schedules and affect financing structures.

The financing market is also becoming more cautious. Reuters reported that AI-related debt issuance had reached nearly $500 billion by early August 2026, while delays involving electricity access, supply chains and political opposition were pushing lenders toward tougher terms and stronger safeguards.

AI enthusiasm remains enormous, but capital is increasingly asking whether a project can actually be delivered.

11. Community Support Is Becoming Part of the Infrastructure Equation

Data centers may power digital services, but the facilities themselves occupy real land in real communities. They consume large amounts of electricity, can require significant water resources, operate cooling and backup systems, and may require new transmission infrastructure.

The backlash has become increasingly visible in the United States. Reuters reported that 142 coordinated protests took place across 42 U.S. states in July 2026, reflecting concerns over resource use, local impacts and community consultation.

For developers, community support is no longer a secondary public-relations issue. It can directly affect permitting, financing and construction schedules.

12. Google Is Securing Power for Decades

Google’s September 2026 investment announcement in Finland illustrates how the AI infrastructure race is changing. Reuters reported that Google plans to invest €13 billion in Finnish AI infrastructure over two years, including three new data centers in northern Finland.

The more revealing part of the announcement may be the power agreement. Google signed a 22-year deal with Finnish energy company Fortum to purchase up to 50% of the output of one nuclear plant. It is Google’s first nuclear energy deal outside the United States.

Big technology companies are no longer competing only to secure GPUs. They are beginning to secure electricity itself for decades. As AI infrastructure expands, the boundary between the computing industry and the energy industry is becoming less distinct.

13. There Is a Long Distance Between Announcement and Operation

The slowdown in some AI data center projects does not necessarily mean demand for AI is disappearing. In many cases, demand is growing so quickly that companies are competing to reserve land, electricity and equipment before every part of a project has been finalized.

But a proposal still has to move through a long chain: site secured → power contracted → grid connection approved → permits obtained → equipment ordered → construction started → AI hardware installed → commercial operation.

A delay at any one of those stages can push a multibillion-dollar project back by months or years. That is why planned capacity and operational capacity should not be treated as the same thing.

14. The Real AI Bottleneck Is Changing

For the past several years, much of the AI race has focused on access to GPUs. GPUs remain critical, but the competition is entering a different stage.

The key issue is increasingly not how much computing hardware has been ordered, but how quickly that computing capacity can be brought online and how consistently it can be utilized.

Seen this way, HBM shortages, network congestion, grid-connection delays, transformer shortages and cooling constraints are not unrelated problems. They are different versions of the same problem: turning expensive theoretical computing capacity into usable AI capacity.

AI is expanding beyond semiconductors into energy, electrical infrastructure, construction, cooling and finance. The next competitive advantage may not come simply from owning the most GPUs. It may come from building an infrastructure system capable of keeping those expensive GPUs working instead of waiting.

A Better Way to Read AI Data Center Announcements

Large investment numbers will continue to dominate headlines. But a $10 billion or $20 billion announcement does not tell us how close a project is to actually running AI workloads.

The more useful indicators are whether the site has been secured, electricity has been contracted, the grid connection has been approved, permits are complete, construction has begun, critical equipment has been ordered and a realistic operating date has been established.

An announced data center and a data center capable of running AI are not the same thing.

FAQ

Is the GPU the most important part of an AI data center?

The GPU is the core computing engine, but it does not determine total performance by itself. Bottlenecks elsewhere in the system can reduce the utilization of even the most advanced processors.

Why is HBM so important?

As GPUs become faster, they require larger amounts of data to be delivered at very high speed. HBM reduces the amount of time processors spend waiting for that data.

Why are AI data center projects being delayed?

Major causes include grid-connection delays, shortages of critical electrical equipment, permitting timelines, rising construction and financing costs, and community opposition.

Will AI data center demand continue to grow?

Current forecasts point to substantial growth. The IEA expects global data center electricity consumption to more than double to around 945 TWh by 2030, with AI as the most important driver of that increase.

Conclusion

One major change is becoming clear in the AI data center race.

AI is no longer only a semiconductor competition.

Grid connections can take longer than the buildings themselves. Advanced AI projects can be delayed by transformers and cooling equipment. Technology companies are signing multi-decade power agreements to secure enough electricity for future computing capacity.

The faster AI develops, the more important the physical infrastructure behind it becomes.