Sparse Neural FX

Overview

This is the project I worked on with Ryota Sato that won the Stanford EE 2026 design award and became a DAFx26 demo paper. Ryota is a model training and pruning unicorn and pruned a wavenet guitar amplifier model such that 90% of the AI computations are reduced and the neural guitar amp sounds basically identical.

We presented at DAFx26 and showed in a demo how the amp sounds the same as real guitar pedals while being very compute light. Made a custom audio USB dongle for our demo too.

PAPER — WaveNet-Style Guitar Amplifier Model Pruning for Real-Time iOS Deployment, Ryota Sato & Eli Silverstein, arXiv 2607.10086 · accepted as a DAFx 2026 demo.

The engine

The model. An 18-layer dilated-conv WaveNet, 16 channels, 2044-sample receptive field. Ryota prunes it in PyTorch to about 90% unstructured sparsity; it is exported to a custom weight format and my engine runs it.

Three backends, all fp32: Accelerate BLAS on the AMX units, dense NEON intrinsics, and a custom sparse NEON kernel that stores only the nonzero weights and vectorises over time.

Speed. About 65× over the original scalar C++. The sparse kernel is 1.9× faster than the best dense backend at 90% sparsity, 2.9× over the dense model. Unstructured pruning normally saves you nothing on real hardware; this turns it into real compute savings.

On an iPhone 16 Pro. Real-time factor 0.016, about 60× faster than the deadline, using around 12% of one core and 9 MB. Output matches the Python reference to int16 quantization.

Benchmarking

To know whether my kernels were actually good, I ran the same model through 16 inference backends on Apple silicon, Mac and iPhone: Core ML on CPU, GPU and Neural Engine (stateful and stateless), BNNSGraph fp32 and fp16, ONNX Runtime, LibTorch, RTNeural, Apple Sparse BLAS, and mine. Sparsity swept 0 to 100% in 5% steps, block size 64 to 2048.

Dispatch-bound, not compute-bound. Core ML has a fixed floor of about 250 µs per call, so the GPU and Neural Engine only beat the CPU sparse kernel at 1024-sample blocks and up, which is 21 ms of latency. For low-latency audio a well-written CPU kernel wins.

Two more. Sparse beats dense only above about 63% sparsity. fp16 BNNS is 2× faster at large blocks, at the cost of 1e-3 error.

The figures in the paper come out of this suite (matplotlib), and I rewrote the performance analysis after DAFx review.

C++17 · ARM NEON · Accelerate/AMX · Core ML · BNNSGraph · ONNX Runtime · LibTorch · RTNeural · Swift/SwiftUI · AVAudioEngine · PyTorch · matplotlib