How do you perceive your Grandmother? It is still not known how the activity of neurons represent complex perceptual objects with many features. The theories are compelling and yet fall short.
The problem of representation is a critical one in systems neuroscience. How is the outside world represented inside our heads? A book uses letters. A computer uses numbers. A painting uses color pigments. What does the brain use? The problem of representation is critical to our understanding of the nature of a percept, thought or idea.
The closest cousin to the brain that we have is the digital computer. All representations in a computer end up with 1s and 0s. However, there is a hierarchy. For example, 1s an 0s create another code called ASCII, and this code represents individual characters of a written text, which then is combined into lines of code, entire files, full texts, novels, etc. A computer can also represent images and sounds, again starting from 1s and 0s and building up a hierarchy of coding schemes.
That way a grandmother can be represented as a word “grandmother” as an image of a grandmother, as a sound of a pronounced word grandmother, or even as a voice of a grandmother saying “Hi, I am a grandmother.”.
But when you perceive your grandmother, where exactly is she in the brain? A picture? A sound? a concept? What about when you remember your grandmother? How and where is this representation taking place?
This problem of representation in the brain is not easy to solve. We still do not have a satisfactory solution.
see related post From Neuron to Brain: The Perils of a Reductionist Approach
One influential idea is that the brain has a hierarchy of representations built from a large number of neurons distributed across that hierarchy. At each level of the hierarchy an increased activity of a neuron—what we call high firing rate—indicates something related to the grandmother. If this is a visual representation, then a neuron very low on that hierarchy may be active for a single pixel of the image (similar to the computer representation) but then other neurons would be active for combinations of such pixels.
Some neurons, sill very low on that hierarchy, would be responsible for detecting lines combined out of those pixels. But then others would operate higher on the hierarchy and would use those lines to detect specific features such as her eyes, nose, hair, glasses, and so on. Finally, neurons even higher on the hierarchy would combine those parts of a face to decide whether the image is a grandmother or maybe rather a young woman.
There may be even one single cell that responds at the end if there is a grandmother being seen. Traditionally, neuroscientists call this unique type of cells “grandmother cells”. But we are not quite sure whether they exist. If they exist, they are located in the infero-temporal cortex (Quiroga et al. 2005).
This hierarchical idea is so attractive and influential that some of the discoveries that initially suggested that this may be how the brain works (Hubel & Wiesel 1962) were awarded a Nobel Prize.
Also, this hierarchical idea is heavily used in artificial intelligence technologies. Deep learning techniques constitute basically a working implementation of such a hierarchy ending with grandmother cells.
But there is also a problem. This theory cannot explain a number of things that human brains can do. The theory only partly explains the problems of perception and imagination.
For example, we can see new objects that we have never seen before. And we can immediately understand what they are and how to interact with them. Consider the following drawing of a car.
We do not need a prewired network of hierarchical feature extraction to see a car in this image. Also, unlike today’s deep learning technology, we do not need hundreds or thousands of examples of such images for an initial training. We get it immediately.
It gets even better. We can switch our perception in an instant. Look back at the image above and test this yourself. Can you erase your perception of a car and replace it with a perception of a table with two chairs? I am sure you can. Again, you can do it in an instant.
The representational hierarchy of neurons cannot do that.
Our mental capabilities go even further than that. We can carry over the idea of a table and chair from the above image and see a classroom in this next image.
Or we can go back to our idea of a car and see a train in the same image.
We can just juggle different ideas in our minds as we please and with these different ideas we can see the world in different ways, see different things in the world, extract different relations between those things in the world and so on.
If we need it, we can even use a single dot to represent a grandmother, and yet have this representation be useful and relevant. For example, using dots we can indicate the trajectory of grandmother’s afternoon walk on a city map.
Our hierarchical theory of representation is helpless at such tasks.
There have been, of course, a number of attempts to expand and improve this hierarchical theory. Most notable is the feature integration theory, which seeks to explain how our visual system works with novel combinations of elements in a picture (Treisman & Gelade 1980). The theory is framed in the context of which things we can do quickly and effortlessly, and which things will take time and mental effort. Treisman discovered that there are fast and slow processes. The fast ones detect the familiar, previously learned combinations of pixels like for example lines or colors; the slow ones glue these into novel combinations such as a red line. The gluing process requires attention and effort while the fast process is done easily without effort or attention.
The feature integration theory is considered by many the most influential work in psychology of perception and attention. It is the finest that the science of human perception has to offer. And yet, the theory brings us only that far. Our abilities to understand the visual world as illustrated in the above pictures, remains unexplained. Feature binding theory explains that we need attention to combine two circles with a square into one entity, but does not help with the question of how we are able to see a car in that entity, or how we shift the percept into a table.
Much like the puzzle with the working memory discussed in my previous post, a mystery remains with a mere representation of objects in our minds. We don’t know how our brain achieves perception and imagination. We don’t know what our neurons do when the image of our grandmother lights up within our heads. We still do not know how to build an explanation of brain’s physical workings such that the perceptual and intellectual faculties of our mind are explained in the same time.
What will it take to solve this and other unexplained features of the brain such as working memory? Clearly we need to come up with some new, fresh ideas that can solve both together.
Danko Nikolić, is affiliated with savedroid AG, Frankfurt Institute for Advanced Studies and Max Planck Institute for Brain Research
References:
Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology, 160(1), 106-154.
Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005). Invariant visual representation by single neurons in the human brain. Nature, 435(7045), 1102.
Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive psychology, 12(1), 97-136.
This is a very interesting topic! However, it is time to stop even considering the possibility of grandmother cells.
This was a flawed idea that came from recording single units in a brain of 84,000,000,000 units. Quiroga’s paper even proved it is very unlikely that any one object is represented by one neuron’s firing:
“We do not mean to imply the existence of single
neurons coding uniquely for discrete percepts for several reasons:
first, some of these units responded to pictures of more than one
individual or object; second, given the limited duration of our
recording sessions, we can only explore a tiny portion of stimulus
space; and third, the fact that we can discover in this short time some
images—such as photographs of Jennifer Aniston—that drive the
cells suggests that each cell might represent more than one class of
images.”
No more serious consideration of grandmother cells, please! Read Quiroga’s paper, not just its title.
Hi Steve,
I agree. Nobody considers a (single) grandmother cell as a serious hypothesis. But if you read my article above, you will see that all of the critique applies equally well to a single grandmother cell as to a “grandmother population” of million cells. My point is not about the single cell vs. population. My point is that the presumed hierarchy that underlies either of those two hypothesis has the same problems. Be it a grandmother or a population, it cannot explain the mental flexibility that I describe above.
Grandmother cell is only used as a vivid illustration for a non-expert reader.
I read the Quiroga’s article.
Danko