TensorBay launches 1MW AIPods for distributed private inference

By Grace Kim

TensorBay has launched AIPods, 1MW containerized AI nodes supporting up to 64 Nvidia HGX B300 systems, with deployment in 12 weeks.

The offering combines compute hardware with a distributed private inference platform (software coordinating model workloads across locations) and Nvidia Dynamo-based inference optimization. The package includes KV-aware routing (request placement based on key-value cache state) and disaggregated serving (separation of inference functions across computing resources).

What does this change for enterprise AI buyers?

For buyers, the practical change is geographic and operational. Enterprises can place capacity closer to users, expand region by region, retain control over models and data, and reduce reliance on centralized third-party inference providers, according to TensorBay. The offering supplies a distributed serving platform, not simply a container holding GPUs. The unit of purchase changes.

In a LinkedIn post announcing the launch, TensorBay said: “The real breakthrough isn’t simply putting GPUs inside a container. It’s making geographically distributed AI capacity operate as a coordinated inference platform.”

The company added: “Instead of concentrating everything in one massive facility, enterprises can deploy capacity closer to their users, expand region by region, retain control over their models and data, and reduce dependence on centralized third-party inference providers.”

At deployment level, the 1MW rating and 64-system ceiling give buyers a sizing framework for one installation. The 12-week period gives infrastructure teams a disclosed schedule for adding capacity. Together, those figures turn a distributed AI concept into a procurement unit with stated power, compute and rollout limits.

For organizations operating inference across several regions, the software layer addresses coordination across that footprint. KV-aware routing directs requests according to cache state, while disaggregated serving separates inference functions across resources. TensorBay frames those tools, alongside Dynamo-based optimization, as the means to make geographically separated capacity operate as one platform.

TensorBay, headquartered in San Jose, California, offers access to AI infrastructure from Tier III data centers in the US, Brazil, Europe and Asia Pacific. The company operates five locations. TensorBay claims more than 24,000 GPUs operate across that footprint.

That network gives the AIPod launch an established geographic base across the Americas, Europe and Asia-Pacific. TensorBay presents AIPods as a packaged hardware-and-software unit for that footprint rather than a standalone hardware box. The 24,000-GPU operating figure remains a TensorBay claim; the announcement gives the aggregate without a location-by-location breakdown.

The product enters a category already populated by several companies. Last month, LG launched an AI Box offering. ZTE also has a small AI container solution.

Against that competitive background, TensorBay has set out three commercial terms: 1MW per node, support for as many as 64 HGX B300 systems and deployment in 12 weeks. It has also attached a private, distributed inference platform rather than offering hardware alone.

For buyers, the practical test is whether the platform can coordinate capacity across regions while preserving control over models and data. TensorBay says it can, through the distributed platform, routing and serving tools.

Its claimed 24,000-GPU footprint and five operating locations supply the base for that model. The 1MW node and 12-week schedule define how buyers can add to it.