I enjoy the idea that our senses are finite and that our brains filter most of what we experience out of our consciousness’ to make life liveable.

Plus, I think this will be a fun idea. – Let’s turn visual into audio.

In a nutshell, the app will take a snapshot every few frames, average out the colour values in the snapshot, and play a sound depending on the range of those colour values.

Colour values are sorted using the hexadecimal system

If we look at #e8bc0c, the warm yellow above, and break the hex number down into 2 red values, two green values, and two blue values, or #RRGGBB, we see that:

e8bc0c
redsgreensblues

But what does “e8” mean for the red, etc., values?

Each value in the hexadecimal system represents a value of 0 – 15 and is expressed from 0 – 9, and A – F:

000
101
202
303
404
505
606
707
808
909
100A
110B
120C
130D
140E
150F

That means the more understandable values would be:

(14 x 16) + 8 = 232/255(11 x 16) + 12 = 188/255(0 x 16) + 12 =12/255
RedsGreensBlues

Two hexadecimals multiplied together gives us a byte of information, 16 x 16 = 256, and because 0 is no colour we have 255 different amounts of red, green, and blue, where 255 is the most of that colour.

But where did that x 16 come from?

Well, hexadecimal is base 16 and not base 10 as the decimal system, so after 0F (15 in decimal), 10 would represent: (1 x 16) + 0.

So to get back to the app, we could convert hexadecimal to decimal format and then define light/dark colour value ranges for each sound/pitch as:

LightDark
< 128> 127
colour values in decimal

For colour ranges we can add some logic, eg. red’s range could look like:

Reds< 100
Greens> 25
Blues> 30
red value ranges

Yellow is particularly interesting as there are no yellow pixels, and is made of Red and Green, the higher the value the more yellow, and the more Blue we add to it the lighter the shade of yellow.

Reds> 210
Greens> 200
Blues< 95
yellow value ranges

That’s enough for this post, in the next post I’ll be looking at how to set up the app.

One Response

  1. So much possibility with synesthesia! I think it could work even better the other way round: turning songs into pictures, and then having the community comment on whether it captures the essence of the song well!

Leave a Reply

Your email address will not be published. Required fields are marked *