Sunday 8 March 2015

Coarse code representation in brain

Although neurons can be considered having active state or not, but brain at its bare neocortex layer  doesn't understand binary information or processing binary data. I mean the neocortex, for example doesn't represent a number like 6 as 0110 and ... It seems that the coarse coding is the way brain understands things, numbers, shapes, images, ... all has a coarse code representation.

Let us see how it works, first numbers. What we had and described in the previous example doesn't work here because our brain is not a binary processor, although we understand this kind of processing the understanding is a knowledge we have achieved via a hierarchical memory. So the simplest way will be like what we described in "Understanding Coarse Codes". Suppose we want to represent numbers from 0 to 9,999, we can do it by allocation 10,000 neurons each representing a number, but this is not that efficient. From the post we mentioned we have:

Number of neurons in u-v (fine resolution) coordinates: 10,000
Single axis high resolution: √100,000 = 100
Single axis low resolution : 5 // we simply consider it
Number of coordinates: Round(100/5) = 20
Number of neurons in low resolution coordinates: 20 * (5*5) = 500
Efficiency = 10,000/500 = 20 or 2,000%

So to represent 0 to 9,999 we need 500 neurons, in the form of 20 overlapped layer of coordinates each having 25 square or neurons. So just with 500 neurons having maximum active of 20 neurons which gives 20/500=4% we can identify a number between 0 to 99,999.



You can't have the imagination of what this means unless get deep into this. Let us have a much more simple and understandable example.

Remember our previous two overlapped coordinates, Green with 8 box and Red with 4 box in it. Consider them representing colors and shapes instead of something related to just a scalar value, like what we have in 2 bottom squares.

Now if you look at the top right square, showing the overlapped coordinates, we see for example activating Green#7 and Red#2 represents a orange circle, while  Green#7 and Red#3 represents a orange square.

There are some points in this example. First if you see in this kind of representation the things (or numbers) that each combination of these 2 layers show will be going to have similarities either in color or shape, if their active neurons are near each other or just some of them are alike. For example, if 000000010000000011000000 and 000000010000000010000000 represent two things, although they do not have fully equal bits, but since they have two similar active bits and there is just one difference between them these two representations are somehow similar things.

000000010000000011000000 
000000010000000010000000

In our simple example we can say that 0001000000010 which can be an orange square and 0010000000010 which can be an orange circle are both an orange thing and so on. So in our previous example in a 20 coordinate system if you have, say 10 or 15 similar active bits you can surely say that these two representations are somehow related and similar things.

0001000000010  // orange square
0010000000010  // orange circle

The other important thing we see in this graphical example is that we can't represent a situation for a dark-blue circle. The reason is we just showed things in flat plane, while in neurons or multi dimensional space (or network) we can have it too. Remember in one of the previous posts we talked about the bayesian networks, the connections between neurons are not as simple as what we see in this picture, I mean just for squares who share boarders. So dark blue square in a higher dimensional space or in a bayesian network like, can have overlap with circle too.