Summary
AWS and NVIDIA's August 2026 infrastructure expansion is easy to read as another GPU capacity headline: 2 million additional NVIDIA GPUs planned for AWS global infrastructure in 2027-2028, support for Vera CPUs, deeper networking work, and a 100,000-GPU federal AI factory commitment. The more durable engineering signal is different. At this scale, the critical design question is no longer simply how many accelerators fit in a fleet. It is how the surrounding server, network, storage, security, and operations planes keep those accelerators useful without turning host CPUs into infrastructure bottlenecks.
That is why the announcement matters alongside NVIDIA's BlueField-4 and "Scale-In" architecture work. Production AI factories are making DPUs a first-class server architecture component. They are not merely SmartNICs for offload or a cloud-provider optimization. They are becoming the control and data path boundary between tenant workloads and the shared infrastructure services that keep large AI systems secure, observable, connected, and fed with data.
The story is larger than GPU supply
AWS and NVIDIA said they plan to deploy an additional 2 million NVIDIA Blackwell Ultra, Rubin, and Rubin Ultra GPUs across AWS global infrastructure in 2027-2028. AWS also described plans for NVIDIA Vera CPU-based infrastructure, ongoing work around NVLink Fusion for future Trainium systems, and continued use of Nitro and Elastic Fabric Adapter as part of the production instance foundation.
Those details point to heterogeneous systems rather than a simple accelerator monoculture. Future AI capacity will blend GPUs, custom accelerators, high-performance CPUs, scale-up interconnects, Ethernet or InfiniBand fabrics, host isolation layers, and external data services. The operating model has to support training, inference, retrieval, simulation, agent orchestration, data preprocessing, and governance on the same broad infrastructure base.
For enterprise infrastructure teams, that means the practical constraint shifts from peak accelerator throughput to system balance. A cluster can have enough GPUs on paper and still underperform if storage traffic contends with model traffic, if telemetry is too coarse to diagnose tail latency, if tenant isolation depends on overloaded host software, or if host CPUs spend too much time moving packets, encrypting flows, and servicing virtual networking.
Why host CPUs become the wrong place for infrastructure work
Traditional cloud server design placed much of the flexible infrastructure layer in software on general-purpose CPUs. That model worked well for many web, database, and virtualization workloads because per-server network and storage rates were lower and workload behavior was more predictable.
AI factories stress that assumption. A single server may participate in high-volume collective communication, move training data or embeddings from storage, serve inference traffic, write checkpoints, expose observability, and enforce policy for multiple tenants or internal teams. Agentic systems add more north-south traffic because agents interact with tools, APIs, data stores, queues, search systems, and sandboxes continuously. The traffic pattern is not just model-to-model. It is application-to-data-to-tool-to-model, repeatedly.
When those services run mostly on the host CPU, infrastructure work competes with application work. On GPU servers, that competition is expensive because the host CPU often coordinates accelerator submission, memory registration, storage paths, and runtime services. On CPU-heavy agent systems, the same contention reduces useful capacity for code execution, orchestration, and data processing. Either way, a general-purpose host becomes a shared chokepoint for functions that should be isolated, measurable, and enforceable outside the workload domain.
Scale-In makes the access plane part of the AI system
NVIDIA's Scale-In framing is useful because it names a part of the architecture that is often treated as background plumbing. Scale-up links GPUs inside a node or rack-scale domain. Scale-out connects servers across racks. Scale-across extends clusters across sites. Context-memory systems address shared inference state. Scale-In focuses on the infrastructure around the compute domain: users, applications, storage systems, security services, provisioning, telemetry, and data access.
BlueField-4 is positioned as the DPU for that Scale-In layer. NVIDIA describes a host-independent processing domain with a 64-core Grace CPU, inline acceleration engines, LPDDR5X memory, PCIe Gen6 host connectivity, and an 800 Gb/s network interface. The important architectural point is not the spec sheet alone. It is that packet processing, RDMA, storage protocols, encryption, firewall rules, policy enforcement, telemetry, and service chaining can be handled in a separate infrastructure domain rather than inside the tenant host.
That separation gives operators a cleaner boundary. The AI workload sees accelerated compute and network access. The infrastructure team gets a place to apply policy, observe state, route storage traffic, and update services without trusting or loading the guest environment. In multi-tenant clouds and shared enterprise AI platforms, that boundary is increasingly valuable.
Storage and data movement are now server design issues
AI infrastructure discussions often over-index on training interconnects, but production systems also need sustained access to enterprise data. Retrieval-augmented generation, vector indexing, fine-tuning, analytics, checkpointing, and multimodal pipelines all create storage and data movement pressure that cannot be solved by GPU count.
A DPU-based access plane can accelerate NVMe-oF, file, object, RDMA, TCP, storage virtualization, and encryption paths while reducing host CPU overhead. That does not remove the need for careful storage design, but it changes where key services can run. Instead of treating storage access as an application-side burden, operators can place policy enforcement, flow classification, and data path acceleration at the server boundary.
This matters for availability as much as performance. If a training job stalls because checkpoint writes collide with retrieval traffic, or if inference latency spikes because storage flows have no isolation, the business impact is not theoretical. A production AI platform needs predictable service behavior under mixed workloads. DPUs give architects another control point for managing that behavior.
Security isolation has to scale with the cluster
The federal component of the AWS-NVIDIA announcement is notable because it highlights secure AI factories for sensitive government workloads, including infrastructure intended for IL6 and above. Even outside the public sector, enterprises are moving AI workloads closer to regulated data, customer records, source code, security telemetry, and proprietary research.
That makes isolation a core infrastructure requirement rather than a compliance afterthought. Host-independent DPUs can enforce network policy, segmentation, encryption, and telemetry from outside the workload they protect. This is not a substitute for identity, application security, or data governance, but it reduces reliance on in-host software controls that may be misconfigured, bypassed, or overloaded.
The operational advantage is consistency. If an AI platform spans GPU servers, CPU agent systems, storage nodes, and cloud services, security teams need policy models that apply across those domains. A programmable DPU layer, managed through frameworks such as DOCA, can provide a common enforcement and telemetry point at line rate.
What architects should evaluate now
The near-term lesson for CTOs and infrastructure architects is not "buy DPUs everywhere." It is to update server evaluation criteria for AI platforms. Accelerator count, memory capacity, and network speed remain essential, but they are incomplete.
Teams should ask where virtual networking, encryption, storage protocol handling, packet filtering, telemetry export, congestion controls, and tenant policy enforcement actually run. They should measure host CPU overhead during realistic mixed workloads, not just during idealized benchmark phases. They should test storage and retrieval traffic while GPUs are busy. They should validate failure isolation: what happens when an agent workload misbehaves, a tenant generates excessive north-south traffic, or a storage path becomes congested?
Procurement should also account for lifecycle operations. DPUs add firmware, software, observability, and operational complexity. A DPU strategy only helps if the organization can provision, patch, monitor, and debug that layer consistently. Otherwise, the architecture moves the bottleneck rather than removing it.
The enterprise takeaway
The AWS-NVIDIA expansion confirms that hyperscale AI infrastructure is moving toward heterogeneous, co-designed systems. GPUs remain the visible capacity unit, but the hidden differentiator is the server architecture surrounding them. CPUs, DPUs, scale-up links, scale-out fabrics, storage systems, and isolation layers have to be engineered as one production platform.
For enterprises building private AI clusters or negotiating cloud capacity, the important question is not whether they can access the newest accelerator generation. It is whether the infrastructure can keep accelerators fed, secure, isolated, and observable under real production pressure.
DPUs are becoming one answer to that question. They provide a dedicated infrastructure plane at the server boundary, close enough to enforce policy and move data at high speed, but separate enough to protect host capacity and tenant isolation. In the AI factory era, that makes them less like optional adapters and more like part of the server architecture itself.