Hear what you see (pt.1)

I enjoy the idea that our senses are finite and that our brains filter most of what we experience out of our consciousness’ to make life liveable.

Plus, I think this will be a fun idea. – Let’s turn visual into audio.

In a nutshell, the app will take a snapshot every few frames, average out the colour values in the snapshot, and play a sound depending on the range of those colour values.

Colour values are sorted using the hexadecimal system

If we look at #e8bc0c, the warm yellow above, and break the hex number down into 2 red values, two green values, and two blue values, or #RRGGBB, we see that:

e8	bc	0c
reds	greens	blues

But what does “e8” mean for the red, etc., values?

Each value in the hexadecimal system represents a value of 0 – 15 and is expressed from 0 – 9, and A – F:

0	00
1	01
2	02
3	03
4	04
5	05
6	06
7	07
8	08
9	09
10	0A
11	0B
12	0C
13	0D
14	0E
15	0F

That means the more understandable values would be:

(14 x 16) + 8 = 232/255	(11 x 16) + 12 = 188/255	(0 x 16) + 12 =12/255
Reds	Greens	Blues

Two hexadecimals multiplied together gives us a byte of information, 16 x 16 = 256, and because 0 is no colour we have 255 different amounts of red, green, and blue, where 255 is the most of that colour.

But where did that x 16 come from?

Well, hexadecimal is base 16 and not base 10 as the decimal system, so after 0F (15 in decimal), 10 would represent: (1 x 16) + 0.

So to get back to the app, we could convert hexadecimal to decimal format and then define light/dark colour value ranges for each sound/pitch as:

Light	Dark
< 128	> 127

For colour ranges we can add some logic, eg. red’s range could look like:

Reds	< 100
Greens	> 25
Blues	> 30

Yellow is particularly interesting as there are no yellow pixels, and is made of Red and Green, the higher the value the more yellow, and the more Blue we add to it the lighter the shade of yellow.

Reds	> 210
Greens	> 200
Blues	< 95

That’s enough for this post, in the next post I’ll be looking at how to set up the app.