Steerable discovery of neural audio effects

Christian J. Steinmetz and Joshua D. Reiss

Centre for Digital Music, Queen Mary University of London

Paper · GitHub · Colab

Abstract


Applications of deep learning for audio effects often focus on modeling analog effects or learning to control effects to emulate a trained audio engineer. However, deep learning approaches also have the potential to expand creativity through neural audio effects that enable new sound transformations. While recent work demonstrated that neural networks with random weights produce compelling audio effects, control of these effects is limited and unintuitive. To address this, we introduce a method for the steerable discovery of neural audio effects. This method enables the design of effects using example recordings provided by the user. We demonstrate how this method produces an effect similar to the target effect, along with interesting inaccuracies, while also providing perceptually relevant controls.

Demo


Examples


Below are a number of example effects generated following the process described in the paper.
Pre-trained models from different effect types are available for usage in the Colab notebook.

Reverb

Description c0 c1
Vocal
Clean vocal - -
Default reverb 0 0
Shorter reverb -2 1
Longer reverb -1 5
Distortion reverb -7 10
Electric Guitar
Clean electric guitar - -
Large room -7 10
Small room 1 1

Compressor

Description c0 c1
Drum kit
Clean drum kit - -
Default compression 0 0
Bassy compression 0.2 -1
More compression 0 0

Analog Delay

Description c0 c1
Gated Synth
Clean gated synth - -
Default delay 0 0
Gritty delay -3 -3
Metallic delay 10 0
Beat
Clean beat - -
Wide delay 0 0
Rumble delay -7 5
Train in the station 10 -5.5

Guitar Amplifier

Description c0 c1
Electric Guitar
Clean electric guitar - -
Amp slapback 0 0
Soft fuzz slap -1 -1
Tunnel 10 -10

Sound matching (Synth to Synth)

Description c0 c1
Piano
Clean piano - -
Long cascade 0 0
Fuzzy cascade -1 0
Heavenly Tunnel 6 6

Paper


Bibtex


                
  @inproceedings{steinmetz2021steerable,
    title={Steerable discovery of neural audio effects},
    author={Steinmetz, Christian J. and Reiss, Joshua D.},
    booktitle={5th Workshop on Machine Learning for Creativity and Design at NeurIPS},
    year={2021}}