Vibe Coding XR: Accelerating AI + XR prototyping with XR Blocks and Gemini

Large language models (LLMs) and agentic workflows are changing software engineering and creative computing. We are seeing a shift toward “vibe coding”, where LLMs turn human intent directly into working code. Tools like Gemini Canvas already make this possible for 2D and 3D web development. However, extended reality (XR) remains difficult to access. Prototyping in XR typically requires piecing together fragmented perception pipelines, complex game engines, and low-level sensor integrations.

Quick, vibe-coded prototypes can solve this problem. They help experienced developers test new UIs, 3D interactions, and spatial visualizations directly in a headset. This rapid validation can save days of work on ideas that might eventually be discarded. It also makes it easier to build interactive educational experiences that demonstrate natural science and mechanics.

Today, we are announcing Vibe Coding XR to bridge this gap. This workflow uses Gemini as a creative partner alongside our web-based XR Blocks framework. By combining Gemini’s long-context reasoning with specialized system prompts and curated code templates, the system handles spatial logic automatically. It translates natural language directly into functional, physics-aware Android XR apps in under 60 seconds.

Our team will present an onsite demonstration at the Google Booth at ACM CHI 2026. You can also try it out here today.

The Vibe Coding XR workflow

Over the last year, we have been iteratively designing and improving the Vibe Coding XR journey to be seamless and accessible. Here’s an example:

To facilitate easier testing, we also provide a “simulated reality” environment on the desktop Chrome. This allows creators to rapid-prototype and test interactions prior to deploying them on Android XR devices. Many advanced perceptual features such as depth sensing, hands interaction, and physics are best experienced on Android XR.