Pull the thread.
Open to collaborations, commissions and conversations.
An Image, Listened To is an interactive work that takes any image and turns it into sound and film. You touch a sphere of nine thousand particles, give it a picture, and the work listens to it: not by scanning it line by line, but by looking at it the way a person does. Every sound you hear has an address in the image, and the same image will always become the same piece.
Most image-to-sound methods read a picture like a score, left to right at a constant speed. This work builds a model of attention instead. A saliency map estimates where the eye is drawn; between ten and twenty-four fixations, depending on how much detail the image holds, are chosen in the order a viewer would find them, and the eye travels between them in saccades. How long it rests on each point depends on how strongly that point holds attention. That path is the time structure of the piece.
Everything in the image has a voice. Height becomes pitch, in a scale and root chosen by the image's dominant colour and light. Brightness becomes loudness, and darkness becomes silence. Edges shape the roughness and the articulation of each sound; the direction of lines turns into rising or falling melody; repeating textures pulse and grainy ones crackle. Sharp, focused areas sound close and dry, blurred ones distant and echoed. Even symmetry and proportions are heard, as mirroring and as the width of the stereo field.
Colour chooses how each sound is made. Every hue belongs to one of eight digital synthesis families: FM, wavefolding, pulse width, phase distortion, granular, additive sweep, bit reduction, and one built from the pixel's own 24 bits, turned into an on/off rhythm. Saturation decides how much of that voice you hear; grey stays a pure tone. Nothing imitates an acoustic instrument: every sound is made by a machine, from the picture.
The image also chooses how it is heard, by how much of it is calm. Continuous texture becomes micro-polyphonic clusters of sustained, slightly detuned voices. Dense, tangled forms become stochastic clouds of short gliding grains. Distinct subjects on a quiet ground become pointillist points of tone. And a mostly empty image lets chance operations in. The gaze always sets the timing; only the way each moment sounds changes.
Between two fixations the eye moves, and the piece listens to what it crosses. Smooth areas become filtered noise, like air; edges become transient clicks; abrupt changes of light become a quantised digital grit. There are no pitched glides here, and the movement always sits beneath the fixations, measured against them in every piece.
The second result is a film of the image being heard. A soft inverting lens travels the exact gaze path, turning the colours beneath it into their negative, while points of light mark each sound where it came from. Below, a particle equaliser of three braided strands follows the music, low frequencies on the left and high on the right.
There is no text anywhere in the work. Touch the sphere, choose an image, and a thin ring follows the real progress. When the work is done, the sphere divides into two smaller spheres: one plays the sound, and as it plays, its particles open and close with the music; the other opens the film. A small point between them brings everything back together.
The work is deterministic. Every sample of the sound is computed with basic arithmetic that every computer performs identically, and its seed comes from the image's pixels themselves, not from the file. I tested it in two different JavaScript engines, and the audio matched to the last bit. Give it the same image tomorrow, or in ten years, and you will hear the same piece.
The edition on Networked is the play edition: the sound and the film play inside the work, in your browser, with nothing uploaded anywhere. It is a single self-contained HTML file with no outside dependencies, made to last. A download edition will follow on my website. Bring it an image you love, and listen to what was already there.
YEKTA
Select
overthink(paint)ing
Open to collaborations, commissions and conversations.