AI Chip Dynamics: Google TPUs Go Native in Meta's Infrastructure

Google has announced that its custom tensor processing units are now running AI workloads in production at Meta. This development was detailed in a post on Google's developer blog, which highlighted a new PyTorch-native version of the SGLang inference engine. Alongside this release, a matching open-source version of vLLM, another popular inference tool, was also made available. These updates represent a significant step in making Google's hardware a standard option for developers who primarily use the PyTorch framework.

For years, running these specific inference engines on Google's TPUs required the use of JAX, a different AI framework. This created a barrier for teams that had invested heavily in PyTorch. The newly released software versions remove this hurdle, allowing the hardware to function natively with PyTorch. This means that engineers can largely avoid rewriting their existing codebases, streamlining the process of deploying models on Google's infrastructure.

Zongwei Zhou, who leads the PyTorch-TPU effort at Meta, confirmed that the new SGLang-TorchTPU system is already live. He attributed the success to a close three-way collaboration involving Meta, Google, and the team behind SGLang. That team operates out of RadixArk, a startup founded by the creators of the SGLang engine. Zhou described the result as a remarkably mature serving stack that feels native to the PyTorch environment.

Jana van Greunen, a director of PyTorch engineering at Meta, emphasized the strategic importance of this flexibility. She noted that Meta cannot be constrained by a single hardware architecture. According to her, the new TorchTPU system enables the company to deploy its PyTorch workloads onto Google TPUs more efficiently. Google stated that it co-developed this software with Meta's PyTorch team to ensure seamless integration.

This is not the first time Meta has experimented with Google's chips. In the past, the company used PyTorch/XLA, a tool that acted as a bridge between PyTorch and TPUs. Google now labels this older tool as a predecessor to the new TorchTPU. Reports from last year indicated that Meta's research teams used these chips through the older bridge between 2020 and 2023 before canceling those specific contracts. The recent push for a native PyTorch solution appears to be a response to Meta's preference to stay within its existing software ecosystem rather than migrating to JAX.

While the exact location of Meta's production TPUs remains unclear, industry talks last year suggested a timeline involving Google Cloud TPU rentals in 2026 and potential purchases in 2027. Meta is reportedly now signed to a multi-year rental agreement. This move comes just one day after Amazon announced it was open-sourcing TorchNeuron, a similar native PyTorch backend for its own Trainium2 and Trainium3 chips. Both announcements were made at the same PyTorch Conference and appear designed to loosen the software hold that Nvidia has maintained in the AI market.

Performance benchmarks have been released to demonstrate the capabilities of the new setup. A tuned vLLM configuration serving the Kimi K3 model, which has 2.8 trillion parameters, reportedly achieves over 700 tokens per second for a single user on the seventh-generation Ironwood TPU. This figure comes from Inferact, a startup founded by the creators of vLLM that assisted Google in building the new software. The test utilized 16 Ironwood chips with speculative decoding, a technique that uses a smaller draft model to speed up inference. In comparison, the same setup on 16 Nvidia GB200s reached 452 tokens per second.

Further tests showed that the new kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the throughput of the GB200 baseline at small batch sizes, provided speculative decoding is not used. The development team at RadixArk stated that SGLang-TorchTPU will continue to add model support, starting with DeepSeek V4.1 Flash and Reflection Beam. The goal is to provide Day 0 support for new open models by the first quarter of 2027.

More details are expected to be shared by Google, Meta, and RadixArk at the SGLang Summit 2026. This event is scheduled for November 12 and 13 in San Francisco. As the AI industry continues to diversify its hardware options, these software updates signal a growing trend toward interoperability. Companies are increasingly looking for ways to run their models on various chips without being locked into a single vendor's software stack.

The shift reflects a broader market movement where major tech firms seek greater autonomy in their infrastructure choices. By making Google's TPUs more accessible to PyTorch users, the companies involved are lowering the barrier to entry for those who might previously have stuck with Nvidia. This could lead to a more competitive landscape in the coming years, as developers gain more flexibility in choosing the hardware that best fits their specific needs and budget constraints.