Google splits TPU line into agent and security systems

Google has added two specialized systems to its eighth-generation TPU lineup: TPU 8i Max, which places two 8i chips in one package for long-context agent work, and TPU 8tx, which reuses 8t silicon for confidential, hardware-isolated inference. Neither has a published spec sheet, price, ship date, or named customer. Both are listed as "coming soon" on Google's Cloud TPU page.

The releases arrive roughly seven months after April's launch of the TPU 8t for training and TPU 8i for inference. That pace matters. Google says its modular approach delivers specialized systems "in a matter of months rather than years." TPUs have become a fast-growing business, with Google claiming nearly 100 customers, including Meta's recent move to TPUs.

What does the 8i Max change for agentic workloads?

On the surface, it is two 8i chips in a single package. That doubles theoretical compute, SRAM, and HBM bandwidth within the "existing rack footprint," and Google claims more compute per square foot and per watt. Derived figures put the part at about 20.2 FP4 PFLOPS, 768 MB of on-chip SRAM, and about 17.2 TB/s of HBM bandwidth.

The comparison with rivals is close. Nvidia's Rubin R200 is rated at 50 FP4 PFLOPS for inference with 22 TB/s of HBM4 per socket. AMD's MI455X sits at 40 FP4 PFLOPS. The compute gap is narrowing.

Bandwidth is the point. Google calls the bottleneck "the memory wall," and it bites "when dozens of AI agents work together across an entire software codebase." Larger context windows — frontier models are targeting 1 million tokens — mean larger KV caches, partly held in SRAM, and every new token must read that cache back from memory. Doubling HBM bandwidth is Google's route around this, in contrast to Nvidia's LPX approach of a second chip for decoding plus a storage tier for context.

Two software pieces support the same goal. TPU Sync, an open-source "model-state layer for TPU clusters," moves KV caches off chip HBM, "automatically tiering massive KV caches off the chip in milliseconds" to host DRAM and, experimentally, shared storage. Google says this cuts KV transfer latency by "more than 90%," with time-to-first-token gains varying by workload. The Agent Compute Engine, a Google Cloud service, pauses idle agent sessions and moves their working memory aside until needed.

Google credits its vLLM-TorchTPU stack with "state-of-the-art price-performance marks" on the InferenceX and AgentX benchmarks for agentic coding and long-context work. Independent numbers exist only for last-generation Ironwood. In SemiAnalysis InferenceX tests in September, running Qwen3.5 397B in FP8, Ironwood delivered 50.4% more tokens per dollar than Nvidia's B200 and 96.0% more than the B300 at 20 tokens per second per user. November results show Ironwood's FP8 beating Nvidia's FP4 GB300 NVL72 on the same model at 1.67x performance per dollar. There is no eighth-gen result yet.

Why does the 8tx swap Arm for AMD?

The 8tx is "built on the same silicon as TPU 8t" and is "optimized for secure and confidential inference" plus small-scale training. Isolation is built into the hardware to prevent snooping on sensitive data, including model weights. Data normally has to be unlocked in memory during processing; the isolation covers that step.

The target buyers are regulated enterprises and governments running frontier or proprietary models, where operators may want to "fine-tune models using their private data, distill large models into smaller ones, and train their agents" without moving sensitive data between clusters. Google frames this as "secure agent execution." The Pentagon and Google reportedly discussed TPU usage in classified settings in April, though the 8tx is not named.

The host change is notable. The 8tx uses 5th Gen AMD EPYC boards instead of the Arm Axion hosts on the 8t and 8i — the Axion having replaced x86 in April. The 8i runs one Axion per two TPUs; Ironwood ran one Xeon per four. Google gives no reason, but its Confidential VM list includes AMD and Intel chips with no Arm options, and its Private AI Compute platform already uses "an AMD-based hardware Trusted Execution Environment." The 8tx appears to be that capability, already running confidential serving on TPUs per a September technical brief, turned into a product. Alternatives include Nvidia's NVL72, with one security domain across 72 Rubin GPUs and 36 Vera CPUs, and Intel's TDX, which is adding trusted I/O for accelerators.

The 8tx also adopts the 8i's Boardfly network over the 8t's 3D torus. Google claims "up to 2x lower latency compared to TPU 8t" with almost no performance penalty, though the metric is unclear. Torus suits training, where chips exchange data with neighbors predictably. Mixture-of-experts and reasoning models send smaller messages between distant chips, and every hop adds delay. Boardfly needs at most seven hops against 16 at 1,024 chips.

What is still missing?

No HBM capacity figure, pod sizes, availability date, pricing, or customer list for either system. Attestation details — the customer's ability to verify confidentiality — are absent for the 8tx. Shipping two specialized systems in seven months shows Google can now turn variants out quickly. Proving them, with benchmarks and hardware in customers' hands, is the remaining task.