Understanding Coarse Codes

It turns out that human brain uses coarse coding to store information it gathers. In fact, this might be the easiest way to store this huge amount of information we gather as we explore the world and live through our lives. In a nutshell, we just store specifications of a the things we see or hear or touch and ... There are neurons in our neocortex each related to any of the concepts our senses understand like big, small, square, circle, red, green, hot, cold and ... The corresponding neurons of these concepts are linked to each other and build something like a Bayesian Network, there are even some mechanisms for the conditions and probability for moving on network edges!

Coarse coding helps the brain to have accuracy when measuring or comparing things, in fact if we have 10 thousands of concept related neurons to figure out a seen or voice or ... we may just need to set 10 or 100 or 200 of these neurons active not more, we will talk about these in later posts.



What is Coarse Coding?

Coarse Coding Idea

Accurate measuring of things is much more difficult than less accurate measuring. I mean, finding the high of people in meter is much more easier than finding the height in centimeter or millimeter. It is easier to say someone is less than a meter or between 1 and 2 meters or over 2 meters than finding his height in centimeters or millimeter. Even the instruments we use to find the length gets more expensive as the resolution goes up.

Our brain hasn't built for accurate measuring, instead with some ordinary measurement it always try to have best possible guess of what it sees or hears or ....  and from the capabilities we see of this amazing creature, I mean brain, we have to agree that this is a good way of measuring or comparing things.

Now look at the picture, it is obvious that finding the position of the ball in x-y coordinate is easier than u-v because the area of each square unit in x-y is 4 times than u-v coordinates. But obviously, the u-v coordinates gives a more accurate position of the ball.

But we can have that accurate position with some less accurate measuring system. Now look at the bottom picture, here we have 2 overlapping coordinates with same low resolution. As you see if we determine the position of the ball in both x-y and x'-y' we can get the position of the ball exactly as when we position it in u-v.

This is the core concept of what we can do with coarse coding. Instead of storing some high accurate information, we can store many less accurate information in a way the whole information gives us what we want. This is why we can recognize things easier or faster than computer programs.

Is it efficient?
Now the question is, how much is this storage efficient? Is the brain uses the best of its available neurons? The answer depends on the resolution we want or the number of dimensions, you can see the efficiency with the simple following calculations, which shows the number of neurons we need to support the required accuracy:

x-y:                   16 neurons 
u-v:                   64 neurons
x-y and x'-y':   16+16=32 neurons

So for this model the efficiency can be described as:  64/32 = 2 or 200%

If we want more resolution or accuracy we need to have x''-y'', x'''-y''' and ... then the efficiency comes to the following:

Number of neurons in u-v (fine resolution) coordinates: N
Single axis high resolution: √N
Single axis low resolution : L
Number of coordinates: Round(√N/L) 
Number of neurons in low resolution coordinates: Round(√N/L) * (L*L)
Efficiency = (N)/[Round(√N/L) * (L*L)]

So for N = 64, L = 4 we have:

Efficiency = (64)/[Round(√64/4) * (4*4)] = 64/(2 * 16) = 32

For N = 1,000,000 and L = 100 we have

Efficiency = (10^6)/[Round(√10^6/100) * (100*100)]  
Efficiency = (10^6)/(10 * 100 * 100) = 10 or 1000%

Just consider we have billions of neurons in our brain and see how efficient this design can be!!!