Real-Time AI Sound Generation on Arm: A Personal Tool for Creative Freedom

By Michael Gamble, Partner & Ecosystem Lead, Arm

As a software engineer and music producer, I’m always exploring how technology can expand creative expression. That curiosity recently led me to build a personal sound generation app that runs directly on-device—powered by an Arm-based CPU and open-source generative AI models. It’s fast, private, and enables me to generate studio-ready sounds from a simple prompt, all within seconds.

This project brings together the best of several worlds:

• The Stable Audio Open model from Stability AI, sourced from Hugging Face
• Execution powered by PyTorch and TorchAudio
• A fast, efficient pipeline that runs natively on Arm-based CPUs
• A seamless creative handoff to Ableton Live

A New Kind of Creative Companion

When I’m deep in a music project using Ableton Live, I don’t want to interrupt my workflow to dig through libraries or browse sound packs. I wanted a tool that could meet me where I am—right in the flow.

Now, I can simply describe the sound I’m imagining (“analog bassline,” “cinematic riser,” “lofi snare”), and within seconds, the generated .wav file appears in my Ableton browser. From there, I can tweak it, loop it, or turn it into an instrument.

Every sound is unique. No one else will generate exactly what I do. That sense of personal ownership fuels my creativity.

Powered by Arm: On-Device, On-Demand

This sound generator runs entirely on-device using Arm-based CPU technology—no GPU, no cloud inference, no latency. Thanks to Arm's efficiency and performance-per-watt, the app stays responsive even during multi-step diffusion runs.

The generation engine is built on:

• The Stable Audio Open model by Stability AI, available via Hugging Face
• PyTorch and TorchAudio for model inference and audio handling
• Optimized multithreaded execution for smooth CPU performance

Sample Code: Optimized CPU Generation

To maximize performance on Arm CPUs, I enabled full thread utilization:

# Use all available Arm CPU threads
torch.set_num_threads(os.cpu_count())