Google and Meta let PyTorch code run natively on Google's TPU chips

Google and Meta have released TorchTPU, a new open source project that lets PyTorch code run natively on Google's own AI chips, its TPUs. According to the developers, existing PyTorch code should run with trivial changes, for example by replacing "cuda" with "tpu" in the code. For companies that train or serve their own models, that makes Google Cloud a more realistic alternative to GPUs, though any gain has to be measured on real workloads.

The announcement was published on the PyTorch project blog on 20 October 2026 at 16:00 UTC, written by the TorchTPU team. The PyTorch account on X followed up at 17:45 UTC the same day, during the PyTorch Conference in San Jose.

From workaround to native backend

PyTorch is the most widely used framework for building and training AI models. Google's TPUs have historically been tied to Google's own frameworks, TensorFlow and JAX. In 2020 Google launched a first PyTorch version for TPUs, PyTorch/XLA. According to the blog post, it required developers to modify their source code, suffered from long compile times and did not support PyTorch's standard step-by-step ("eager") execution mode.

TorchTPU is built from scratch as a native PyTorch backend for TPUs. The work began at Google in mid-2025, in collaboration with the PyTorch team at Meta. The team writes that the runtime has no dependency on JAX, that it supports both eager execution and compilation with torch.compile, and that it works with the standard PyTorch tools for spreading training across many chips.

The team is explicit about the limits. A model should run with no change beyond choosing the device, but it will not reach peak performance out of the box. Shapes, sharding and operations in a model may be tuned for GPUs and need adjustment to get full speed.

What is in place

According to the announcement, TorchTPU supports around 450 core PyTorch operations. The team says it has verified that the 5,000 most downloaded models on Hugging Face run without adaptation, including Qwen, DeepSeek, Gemma, Llama, Mistral and Whisper. It integrates with vLLM and SGLang for serving models, and with TorchTitan and Ray for training.

The team writes that TorchTPU is already used in production for both training and inference by some of Google Cloud's largest customers. The customers are not named. The code is now publicly available on GitHub. The licence terms are not stated in the blog post.

The developers' own numbers

The blog post contains a range of performance figures. In training, Llama 3 405B reportedly reached 500 tokens per second per chip across 1,024 TPU v7x chips. For serving, Google says its upcoming combination of vLLM and TorchTPU delivered the lowest cost per million tokens in the InferenceX benchmark, 2.01 cents, down from 3.6 cents in September. All figures were selected and published by the developers themselves. They have not been independently verified, and the announcement does not say what TPU capacity costs an ordinary customer.

For the next 6 to 12 months, the team says the work will focus on hardening the software for production, supporting more model types and adding lower numerical precision for the next generation of chips.

More chips for the same code

The day before, on 19 October, Amazon presented a similar move for its own Trainium chips, TorchNeuron. Taken together, the announcements show the large cloud providers competing to run the same PyTorch code on their own chips. In our assessment, this may over time reduce the importance of which chip the code was originally written for.

What leaders should note

• Organisations that train or run their own models in PyTorch gain another option to negotiate with. Ask the team to test a real workload on TPUs before committing to more GPU capacity.
• The cost figures in the announcement are Google's own and apply to specific benchmarks. Model your own volumes and ask Google Cloud for actual pricing, since it is not stated.
• In the announcement, the TPUs run in Google Cloud. Moving workloads there may reduce dependence on a single chip supplier but increases dependence on a single cloud provider. Weigh both in the same decision.
• The project is in active development. Check the licence in the GitHub project and which parts are mature enough before putting anything into production.

Sources and media

• PyTorch: Introducing TorchTPU, blog post published 20 October 2026 at 16:00 UTC
• PyTorch on X: Google and Meta launch TorchTPU, post published 20 October 2026 at 17:45 UTC
• TorchTPU on GitHub
• Amazon: TorchNeuron runs PyTorch on AWS Trainium chips, published 19 October 2026
• Sources were checked on 20 October 2026.
• Illustration: AI-generated with Grok Bot / hogby.ai