Possible Mentors
-
Georg Holzmann
-
Thomas Grill
-
Jamie Bullock
Description
Vamp ( http://www.vamp-plugins.org/) is an audio processing plugin system, which extracts descriptive information from audio data — typically referred to as audio analysis plugins or audio feature extraction plugins. This project should implement a vamp host for pure data. Vamp is the primary analysis plugin format used by Sonic Visualiser and open-source editors like Ardour or Audacity.
Many vamp plugins are already available, for example:
-
a note onset detector,
-
beat tracker,
-
tempo estimator / tempot tracking,
-
pitch tracking / note tracking,
-
key estimator,
-
tonal change detector,
-
structural segmenter,
-
timbral and rhythmic similarity estimator,
-
chromagram,
-
constant Q spectrogram and MFCC calculation,
-
audio fingerprinting, ...
for a full list see http://www.vamp-plugins.org/download.html .
A vamp host for PD would need some more work than a simple DSP externals. The following things, amongst others, have to be considered (see also http://www.vamp-plugins.org/rationale.html):
-
Plugins have more control over their inputs than a typical real-time processing plugin. For example, they can indicate to the host their preferred processing block and step sizes, and these do not have to be equal.
-
Vamp plugins may ask to receive data in the frequency domain instead of the time domain. The host takes the responsibility for converting the input data using an FFT of windowed frames.
Therefore the vamp host external should abstract all the requirements of an individual plugin. This means that FFT and blocksize conversions should be done automatically in the external, making it as easy to use as possible on the patcher level.
Resources to start:
-
http://iem.kug.ac.at/pd/externals-HOWTO : a tutorial on how to write C externals for pure data
-
http://www.vamp-plugins.org/develop.html : information on how to develop a vamp host and plugin, API documentation and an example host
Required Skills
-
C/C++ programming
-
basic signal processing knowledge (e.g. what is a FFT and how to use it)
-
interest in audio analysis, music information retrieval