Many theoretical proposals in neuroscience are presented in the form of an engineering solution. But engineering a solution is a different process than creating a fundamental theory and may be where connectionism has floundered.
In previous posts of this series, I discussed where connectionist theories fail in explaining the brain and how also overly relying on modules does not help in explaining the brain-mind relationship. Both these approaches, connectionism and ‘modulism’, rely heavily on methodology that we typically use for engineering—achieving the function at hand by the easiest means possible. However, in order to build a successful theory of brain and mind engineering solutions are not likely to be the best approach.
What is an engineered solution?
Engineering is about solving problems in the simplest possible way to satisfy a given function. The process of creating engineered solutions works something like this:
- Specify the problem: What function or behaviour do I need to achieve from my (neural) machine?
- Specify components: What parts do I have at my disposal?
- Find the simplest way these parts can achieve the needed function.
- Demonstrate that it works.
- Done (submit the paper)
By having completed a process like this it is often claimed that the resulting solution is a theory of how something works in the real brain. Unfortunately, such efforts quickly lead to the superposition problem, which then in turn requires one to pile up modules.
For example, a researcher may notice that connectionists models do not by their very nature show the behaviour that we see in human working memory (or short-term memory). The behaviour typical for working memory does not emerge from the basic assumptions of the connectionist approach. The researcher then naturally attempts to engineer this solution into the model. They list the behaviour that needs be achieved such as e.g., holding information for one second and having limited capacity. Then they ask what do I need to add to a connectionist network in order to achieve this behaviour? At that stage they may pick for example something like fast synapses to carry information for short period of time (e.g., Fiebig et al., 2018), or some sort of recurrent activity of neurons which then maintains information (e.g., Camperi & Wang 1998). After the decision, they engage in an iterative process of improvements until the desired behaviour has been produced. This iterative process requires more changes here and there, until the whole “machine” works.
But as we have seen, this does not help us build the big picture about how the brain creates mind because such activities necessarily lead to knowledge superposition problem. The solution cannot explain how perception works or where the brain’s spontaneous activity come from. This then may force on us the need for modules, but modules do not solve the problem either.
Building a theory: what does it entail?
In my opinion, to solve the brain-mind problem, a fundamentally different approach is needed: Whenever we encounter a superposition problem, we do not resort to modules. Instead, we go back to the fundamental assumptions of the theory and attempt to re-write them such that the superposition problem goes away. This one is by its nature much harder, and it works something like this:
1. Ask a question: Do we see a way for the given function/behaviour to emerge from the fundamental assumptions of the theory, without a need to introduce new contradicting assumptions?
In case of our working memory example, a researcher should maintain a long list of problems other than working memory that also need to be solved and that are very likely to interfere with any easy engineering solutions to the working memory problem. The more difficult these other problems seem, the better the list. Therefore, with every new idea proposed on how to implement working memory one should immediately ask the question: Do I see a potential conflict with anything on that list? Can I with this idea suddenly start easily explaining attention, problem solving, concepts, insights, global workspace? And can I do all this without any conflicts among explanations? No connectionist model does that kind of combing through. And this is exactly where they fail. Rather, such list checking should be done at a very early stage, much before any plans for making computer simulations are being drafted.
2. If the question in (1) is satisfactorily answered, provide a detail explanation of how this function follows (emerges) from the existing theory. And you can declare success.
3. If the function does not follow from the theory, see whether you can alter the assumptions of the theory such that the new function does emerge but in the same time, that the theory continues to explain everything else that it has been shown to explain so far. In other words, ensure that there is no conflict between assumptions. If you can do that, declare success. This is a big deal!
4. If you fail in step 3), this means that you cannot find a solution that works for everything—for both old and new phenomena. This is the more likely outcome. But this is not a failure. This is also a valuable discovery. You simply need to acknowledge and explain the problem that you have just encountered. This can help the field move more quickly and effectively towards a fundamental theory.
Figure: A proposed activity flow for creating a successful theory on brain-mind phenomena.
The practical challenges of fundamental theory over engineering
No doubt, this process is harder than engineering. By definition, there cannot be many success-claiming papers. In most cases they would describe failures but in doing they would reduce the state space in which one searches for solutions. This is why we have step iv).
Another challenge is that for many such a ‘failure’ based approach it may appear difficult to maintain a career in science. This results in researchers feeling compelled to do at least a “little something” successful and end up creating and publishing an engineered solution. This is not helping in the long run. Consider how much connectionism has flooded us with engineering-like solutions over the decades and how little progress has it lead to regarding big questions in neuroscience.
But how does one solve the short-term challenge of publishing and career progress?
Most likely, such diligent work will help you. The contradictions that you observe in number iv) may be jewels for your future work or even for other researchers. Likely, a problem-exposing paper will help much more for someone else to come up with a much better idea in the future than what a ‘successful’ engineered solution could do. Perhaps, we as a research discipline should have ten or so problem-exposing papers for each solution-claiming paper. Today, this is not the case. From my own experience, writing a problem-exposing paper (Nikolić, 2009) can be rewarding.
I believe that only the intensive work of thinking about how to re-write fundamental assumptions can be productive in the long run for the difficult problem of the brain-mind relationship. Engineered solutions give a false sense of a small step being achieved, when in fact the offered solutions are misleading. We need to find a way to re-write connectionism such that it actually works.
References:
Camperi, M., & Wang, X. J. (1998). A model of visuospatial working memory in prefrontal cortex: recurrent network and cellular bistability. Journal of computational neuroscience, 5(4), 383-405.
Fiebig, F., Herman, P., & Lansner, A. (2018). An Indexing Theory for Working Memory based on Fast Hebbian Plasticity.
Nikolić, D. (2009, June). Model this! Seven empirical phenomena missing in the models of cortical oscillatory dynamics. In Neural Networks, 2009. IJCNN 2009. International Joint Conference on (pp. 2272-2279). IEEE.
Danko Nikolić, is affiliated with savedroid AG, Frankfurt Institute for Advanced Studies and Max Planck Institute for Brain Research
Read the entire blog series here.
