Back to Devoured - November 11, 2026
DEVOURED
ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser

ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser

AI Pytorch
PyTorch models can now run directly on the browser GPU using ExecuTorch without requiring conversion to external formats like ONNX or GGUF.
What: The new ExecuTorch WebGPU backend allows models like Llama 3.2 and Whisper to run locally in the browser using WGSL compute shaders, providing 2–10x speed improvements over CPU-based WebAssembly execution.
Why it matters: Bringing native PyTorch inference to the browser removes the conversion bottleneck and privacy risks associated with server-side AI, while enabling high-performance local AI deployment.
Takeaway: Try the ExecuTorch WebGPU Playground to test your models directly in the browser and use the 'torch.export' toolchain to generate .pte files for web deployment.
Deep dive
  • Models are exported once to a .pte file and run on the client using the browser's native WebGPU API.
  • Performance benchmarks on an Apple M1 Pro show a 4–7x speedup for prefill and 2–10x for decode compared to XNNPACK CPU inference.
  • Supports a wide array of architectures, including large language models, vision-language models, and speech-to-text tasks.
  • The runtime uses WebAssembly for orchestration and WGSL compute shaders for execution, ensuring privacy by keeping all data in the client's device memory.
  • The system is designed for zero-installation deployment and cross-platform compatibility across Chrome, Firefox, and Safari.
Decoder
  • WGSL (WebGPU Shading Language): The native shading language for WebGPU, used to write compute shaders that run on the GPU.
  • Prefill: The stage in LLM inference where the model processes the initial prompt or input sequence to generate internal hidden states.
  • Decode: The stage in LLM inference where the model generates tokens one by one after the initial prompt has been processed.
Original article

TL;DR

  • PyTorch models now run in the web browser on the GPU through ExecuTorch, with no conversion to another framework’s format. You export with torch.export to an ExecuTorch .pte file, and the same file runs in any WebGPU browser via ExecuTorch’s new WebGPU backend.
  • It works across a broad range of models: LLMs (e.g. Llama 3.2, Qwen3, Gemma 3), vision-language (e.g. SmolVLM), segmentation, depth, object detection, super-resolution and speech-to-text (e.g. Whisper). Everything runs locally in the tab, and no data leaves the device. Try it in the ExecuTorch WebGPU Playground.
  • On the same machine, WebGPU is 4–7× faster than ExecuTorch’s CPU path (Webassembly + XNNPACK) on prefill and 2–10× faster on decode.

Why the Browser?

The web is the largest deployment surface with billions of users on Chrome, Firefox and Safari. Browser inference ensures privacy by keeping users’ prompts, model inputs, and outputs on the user’s device rather than the server. This reduces network latency, enables offline use-cases, and lowers server-side inference costs. With the ExecuTorch WebGPU backend, PyTorch models get a native path to GPU-accelerated inference in WebGPU-enabled browsers on desktop, laptop and mobile.

WebGPU, the modern successor to WebGL, provides native compute shaders, shared memory, storage buffers, and barriers, making it viable for high-performance ML inference directly in the browser. Today, running a PyTorch model in the browser usually means first converting it to another framework’s format, such as GGUF, ONNX or TFLite. The ExecuTorch WebGPU backend lets PyTorch models reach the browser without leaving the PyTorch toolchain.

Backend Design

Three principles guided the WebGPU backend’s design:

  • Architecture, export, and dynamic runtime. Models export through ExecuTorch’s torch.export() and WebGPUPartitioner and run via WGSL compute shaders. At runtime, the WebGPU runtime allocates GPU buffers and dispatches compute shaders into command buffers. Dynamic shapes are handled via allocate-at-max buffers with per-dispatch workgroup recalculation to enable batched prefill, and the runtime routes shapes to specialized WGSL kernels at runtime.
  • Same artifact across multiple devices and browsers. One model (.pte file) runs in WebGPU browsers such as Chrome and Safari, natively on Metal (macOS), Vulkan (Linux/Android) and D3D12 (Windows), translating WGSL to Metal Shading Language, SPIR-V or HLSL depending on the platform. For browser deployment, the dynamic C++ runtime compiles to Webassembly via Emscripten and emdawnwebgpu bridges its WebGPU API calls to any browser’s WebGPU JavaScript API.
  • Zero friction for the end user. With WebGPU, models run in the browser sandbox with no native installation, backend server, and no data leaving the device. For developers, deployment is a .pte file served over CDN on a web page. Pipeline compilation, memory management, and dispatch scheduling are all handled through the WebGPU backend, with all data staying in GPU device memory within the browser tab.

Performance

We benchmarked ExecuTorch WebGPU against ExecuTorch’s own Webassembly (wasm) CPU backend (XNNPACK, multi-threaded) on the same machine, using quantized Llama 3.2 1B and Qwen3-0.6B in Google Chrome on Apple M1 Pro. Each data point is the median of 5 runs after 3 warmups, across 128 to 8K tokens of context, with identical token IDs for both backends. Both are timed with wall-clock time, since GPU timestamps aren’t available for CPU execution. WebGPU uses 4-bit groupwise weights (group size 64) with fp32 activations; the CPU uses XNNPACK’s 4-bit weights with 8-bit dynamic activations.

Models in Practice

We’ve validated the WebGPU backend across models with very different architectures, demonstrating that the operator coverage generalizes beyond autoregressive text generation:

LLM Inference

  • Llama 3.2 1B (4-bit quantized)
  • Qwen2.5-0.5B (4-bit quantized)
  • Qwen3-0.6B (4-bit quantized)
  • Gemma 3 1B

Vision-Language Understanding

  • SmolVLM: image question answering and captioning

Vision

  • Depth Anything V2: monocular depth estimation
  • SAM 2 (SAM 2.1 Hiera Tiny): promptable image segmentation
  • D-FINE: object detection
  • Real-ESRGAN: super-resolution
  • Background Removal (MODNet): foreground segmentation

Voice

  • Whisper (base): speech-to-text
  • Phonon-31M-EN: text-to-speech

On-Device Training

  • LoRA and DiReFT fine-tuning for Llama 3.2 1B, Qwen2.5 0.5B and Qwen3 0.6B, aimed at personalized assistants and domain adaptation on-device

Try it: ExecuTorch WebGPU Playground

The ExecuTorch WebGPU Playground runs many of the models above, all on the same WebAssembly + WebGPU runtime. All the prompts, photos, camera frames and audio never leave your device. Each model’s Hugging Face repo includes the .pte files, a model card, and the exact export recipe.

What’s Next

The backend is still in early development. We’ll keep making the backend faster and more stable, and expand operator, data type, and model coverage.

Contributions / Feedback

ExecuTorch is open source, and we’d love your contributions to the WebGPU backend. If something breaks or a model you care about isn’t supported, open an issue or reach us on the ExecuTorch Discord. Contributions are welcome too, whether that’s new operators, new models or performance improvements.

Explore all 54 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi