Sonify is a web experiment that reads an image, pixel by pixel, and plays it back as music. Load a picture, press Sonify, and listen.
01 What is Sonify?
Sonify uses the Web Audio API to synthesize sound directly from a picture. The canvas is divided into a grid; the average color of each block is mapped to pitch, gain and timbre, then sequenced into a short, looping phrase.
Adjust the image with the on-screen effects and the music changes with it — because the sound is read from the filtered pixels, not the original. Editing the picture is editing the song.
02 Why color & sound?
“The sound of colours is so definite that it would be hard to find anyone who would try to express bright yellow in the bass notes, or dark lake in the treble.” — Wassily Kandinsky, Concerning the Spiritual in Art
Sonification — turning data relations into sound you can perceive — has no single agreed paradigm. That said, Sonify’s mapping is based on creative intuition.
03 Parameter mappings
Each slider is wired to a specific musical parameter. Color and tone shape pitch and loudness; the red, green and blue channels each voice a different waveform; the effects become audio effects.
| Color & tone | Sound parameter |
|---|---|
| Hue | Pitch detune amount |
| Saturation | Voice separation |
| Lightness | Note gain |
| Contrast | Note gain |
| Brightness | Note gain |
| Channel | Waveform |
|---|---|
| Red | Square wave |
| Green | Sine wave |
| Blue | Triangle wave |
| Effect | Audio parameter |
|---|---|
| Blur | Reverberation amount |
| Sharpen | Amplitude attack / decay |
| Glow | Note gain |
| Noise | Delay amount |
04 Hear it back
Sonify