AI-Ready Data Infrastructure: How Modular Data Centres Support High-Density AI Compute Demand

Artificial intelligence workloads are reshaping the data centre industry faster than any technology shift in the past three decades. The infrastructure requirements of AI training and inference are categorically different from those of traditional enterprise computing — different in power density, cooling approach, network architecture, and physical design. Data centres built for traditional workloads in 2018 are already inadequate for the AI computing requirements of 2026.

Bangladesh's AI adoption trajectory is accelerating. The country's banking sector is deploying AI for credit scoring, fraud detection, and customer segmentation. Telecommunications operators are using AI for network optimisation and predictive maintenance. Fintech and e-commerce platforms are implementing recommendation engines and dynamic pricing. Government agencies are exploring AI-powered public services. Each of these applications generates compute demand that traditional server infrastructure cannot efficiently support.

This article explains what AI-ready data infrastructure actually requires — technically and specifically — and how modular data centre approaches can provide that infrastructure efficiently and scalably.

The AI Compute Density Problem

Traditional vs. AI Workload Power Density

Traditional enterprise server workloads — file servers, database servers, application servers, email infrastructure — operate at power densities of 3–10 kW per rack. A fully loaded standard server rack containing 20–30 general-purpose servers consumes 5–8 kW. HVAC design for traditional data centres is built around this density range.

AI computing workloads are fundamentally different. GPU (Graphics Processing Unit) servers used for AI training and inference — NVIDIA H100, A100, and H200 server systems, for example — consume dramatically more power per unit:

A single NVIDIA H100 SXM5 GPU: approximately 700 W
A standard 8-GPU H100 server (DGX H100): approximately 10,200 W (10.2 kW) for the server alone
A rack of 4 DGX H100 servers: approximately 41 kW — exceeding the entire IT load of many traditional enterprise data centres
High-density AI computing deployments: 30–100 kW per rack is now common in hyperscale AI facilities; 100 kW+ per rack is emerging in specialised AI training clusters

Bangladesh enterprises beginning AI deployments may not immediately require 100 kW racks — but even modest AI server deployments (2–4 GPU servers) in a traditional server room with 8 kW rack design will overload that room's cooling infrastructure. Planning for AI-appropriate power density is a prerequisite for any organisation with AI ambitions.

Liquid Cooling: The Enabling Technology for High-Density AI

Why Air Cooling Fails at High Density

Traditional air cooling — forcing cool air through server chassis using fans — has a practical maximum density limit of approximately 25–30 kW per rack before the physics become unmanageable. Beyond this density, the volume of cool air required to carry away the heat generated by the rack exceeds what fan systems can move at acceptable velocity without creating noise and airflow turbulence problems.

GPU servers compound this problem because GPU chips produce concentrated heat in a small physical area — unlike CPUs that distribute heat more evenly across server chassis. A 700W GPU chip produces more heat per square centimetre of chip surface than air-cooling heatsinks can efficiently remove at the air velocities achievable in standard server chassis.

The data centre industry's solution is liquid cooling — using water or dielectric fluid as the primary heat transfer medium instead of air.

Direct Liquid Cooling (DLC)

Direct Liquid Cooling routes chilled water through cold plates mounted directly on GPU chips, CPUs, and memory modules. The cold plate — a precision metal block with internal water channels — is in direct contact with the chip surface, providing heat transfer orders of magnitude more effective than air convection.

Water-cooled components in a DLC system reject 50–90% of their heat directly to the cooling water circuit, dramatically reducing the heat load on the room-level air cooling system. A DLC-equipped 40 kW GPU rack may reject only 5–8 kW of heat to the room air (from components that cannot be liquid-cooled — power supplies, networking cards) rather than all 40 kW. The room air cooling system is sized for the residual heat load, not the total IT load.

DLC servers (from manufacturers including Dell EMC, Hewlett Packard Enterprise, and Lenovo — all offering liquid-cooled GPU server configurations) connect to a facility coolant distribution unit (CDU) that conditions the cooling water to the required supply temperature (typically 18–25°C for chip-level DLC) and manages the flow distribution to server racks.

Immersion Cooling

Immersion cooling is the highest-efficiency approach: servers are physically submerged in a tank of dielectric fluid (electrically non-conductive) that absorbs heat from all components simultaneously. Heat is transferred from the fluid to a water heat exchanger at the tank, and the warm water is then routed to a chiller or cooling tower.

Two immersion approaches exist:

Single-phase immersion: The dielectric fluid remains liquid throughout. Servers are submerged in tanks of mineral oil or engineered dielectric fluid (such as 3M Novec or Engineered Fluids EC series). The fluid circulates by convection and pump, absorbing heat and rejecting it via a heat exchanger.

Two-phase immersion: The dielectric fluid is specifically engineered to boil at low temperatures (35–60°C). Server heat causes the fluid to boil at the component surface, carrying heat via vapour to a condenser above the fluid surface, where it condenses back to liquid and returns. Two-phase cooling is more efficient than single-phase but requires more careful fluid management.

Immersion cooling is particularly effective for GPU cluster deployments: the entire 8-GPU server is submerged, and heat removal is uniform across all components. Power densities of 100 kW per immersion tank are achievable with appropriate infrastructure.

For Bangladesh's early-stage AI deployments, DLC is the more accessible starting point — it uses standard server room infrastructure (chilled water or CDU with water-cooled condenser) and avoids the more radical infrastructure change of immersion. As AI compute density grows, immersion cooling becomes increasingly relevant.

Power Density Planning for AI Infrastructure

Site Power Planning

Planning AI compute infrastructure requires calculating the total site power requirement, not just the IT load. A 4-rack AI cluster at 40 kW per rack requires:

IT load: 160 kW
Cooling overhead (DLC, PUE 1.3): 48 kW
UPS losses (assume 95% efficiency): 11 kW
Total site power requirement: approximately 219 kW

This site power must be available from the electrical supply — which requires adequate upstream capacity from the building's main distribution panel, appropriate UPS sizing (N+1 for at least 275 kVA), and generator capacity to support the total load.

Bangladesh's commercial building electrical infrastructure is frequently designed without AI compute density in mind. A building designed for standard office or data centre use may have a 500 kW main supply for the entire building — which a modest AI cluster could consume entirely. Early-stage AI infrastructure planning requires an electrical survey of available power capacity before committing to AI hardware purchases.

PDU Specification for High-Density Racks

Standard PDUs (Power Distribution Units) in server racks are rated for 16A or 32A at 230V (approximately 3.7 kW or 7.4 kW). A 40 kW AI rack requires specialised high-density PDUs: 3-phase 60A or higher, with individual outlet monitoring, remote switching capability, and high-ampacity outlets matching the GPU server's proprietary power connectors.

Specify PDUs before GPU server hardware — the connector types and phase configuration required by GPU servers are not universal and must match the facility's power distribution design.

Network Architecture for AI: InfiniBand and High-Speed Fabric

Why Standard Ethernet Is Insufficient for AI Training

AI training workloads require constant data exchange between GPU servers. During the forward and backward passes of neural network training, gradients (mathematical vectors representing model update information) are exchanged between all GPUs in the training cluster at high frequency. This GPU-to-GPU communication is the performance bottleneck for large AI model training.

Standard 10 GbE or 25 GbE Ethernet — adequate for traditional server workloads — introduces latency and bandwidth constraints that severely limit AI training efficiency. Large model training scales linearly (theoretically) with the number of GPUs — but only if the inter-GPU communication can keep pace with the compute rate.

InfiniBand is the interconnect technology that solves this problem. InfiniBand provides:

Bandwidth: 200 Gbps (HDR InfiniBand) or 400 Gbps (NDR InfiniBand) per port — versus 25–100 Gbps for high-end Ethernet
Latency: Sub-microsecond latency for RDMA (Remote Direct Memory Access) operations — versus 10–100 microseconds for TCP/IP Ethernet
RDMA support: Direct memory access between GPUs across the network without CPU involvement, dramatically reducing communication overhead

NVIDIA's NVLink and NVSwitch technologies provide even faster intra-server GPU communication (within a single server chassis), with InfiniBand providing inter-server communication in a cluster.

For Bangladesh enterprises building on-premise AI training infrastructure, InfiniBand network infrastructure is a prerequisite for GPU cluster scaling beyond 4–8 GPU units. A single 8-GPU server can train many models efficiently using NVLink alone; scaling to 16, 32, or 64 GPUs requires InfiniBand fabric.

High-Speed Ethernet for AI Inference

AI inference — using a trained model to make predictions on new data — has less demanding network requirements than training. Inference can typically be served over standard 25 GbE or 100 GbE Ethernet infrastructure with appropriate network design (low latency switches, Quality of Service configuration). This makes inference deployments more accessible to organisations without InfiniBand expertise.

Bangladesh Context: Growing AI Adoption in Banking and Telecom

Bangladesh's banking sector is the leading adopter of AI and ML applications among the country's industries. Bangladesh Bank's progressive data governance requirements (requiring in-country data processing for customer financial data) make on-premise AI infrastructure — rather than offshore cloud AI — the appropriate framework for many banking AI applications.

Current AI applications in Bangladesh banking include:

Credit scoring: ML models trained on transaction history, digital footprint, and behavioral data to assess creditworthiness for the large unbanked and underbanked population. These models require regular retraining as data grows — a CPU-based training process that could be accelerated 10–50× with GPU compute.

Fraud detection: Real-time inference models that assess each transaction against fraud patterns. The inference load scales with transaction volume and requires low-latency GPU inference infrastructure for sub-second response times.

Customer segmentation and targeting: ML models that segment customers for personalised product recommendations and marketing. Training these models on millions of customer records benefits significantly from GPU acceleration.

Bangladesh's telecommunications sector is similarly deploying AI for network performance prediction, customer churn prediction, and automated network management — use cases that benefit from GPU-accelerated training and inference.

Future-Proofing Your Data Centre for AI

Design Principles for AI Readiness

Even if an organisation does not have immediate AI infrastructure requirements, building AI readiness into current data centre designs costs relatively little and avoids expensive retrofits later:

Power density headroom: Design rack power distribution for at least 15–20 kW per rack, even if current equipment only requires 5–8 kW. The incremental cost of higher-rated PDUs and slightly heavier switchgear is modest.

Cooling flexibility: Include a chilled water loop or CDU connection point in the facility design, even if DLC is not deployed initially. Retrofitting chilled water infrastructure after construction is expensive and disruptive.

Network infrastructure: Install dark fibre between server rooms and network rooms for future InfiniBand runs. InfiniBand uses different cabling (optical fibre) than standard Ethernet in high-speed configurations; planning fibre pathways now avoids civil works later.

Modular architecture: Modular data centres specifically enable future AI density upgrades because additional cooling infrastructure can be added as modules rather than requiring facility renovation.

Physical space allocation: Reserve floor space adjacent to current IT deployments for AI-specific cooling equipment (CDUs, immersion tanks) that will require space when AI workloads are added.

SSAL's AI Infrastructure Expertise

SSAL designs modular data centre infrastructure for AI-ready deployments in Bangladesh. Our data centre engineering team has expertise in high-density power distribution, DLC infrastructure design, InfiniBand network planning, and the modular architecture that provides the scalability AI growth requires.

We work with organisations at every stage of AI infrastructure maturity — from the first GPU server in an existing server room to purpose-built AI compute facilities with liquid cooling and InfiniBand fabric.

Ready to assess your AI infrastructure readiness?

[Get Free Consultation] — Request an AI infrastructure assessment and gap analysis for your current data centre.

[Talk to Our Expert] — Discuss your AI compute plans and data centre readiness with a specialist who understands Bangladesh's regulatory and infrastructure context.