No doubt this seems to be a heady topic so soon after my initial “Hello World” post. (pun intended; what’s 3 months in the world of blogging?) Regardless, in one of those “can’t stop thinking moments” while in the midst of travel, I started mulling over some of the research I’ve been reading (and trying to ignore) regarding neural networks. As luck would have it, I’m sitting here at nVidia’s GTC conference and going to be taking in quite a lot of information on the subject. To that end, I wanted to throw some straw men out there to be burned, evaluated, sifted regarding ideals behind neural networks. Who knows? Maybe we’ll all learn something new.
Towards more “Biological” Systems
The big push in neural networks (and a reason why I’m labeling this post as polymorphic) is to emulate biological systems. Companies like BrainChip and others are making use of biological models to provide insights and functions as a type of biomorphism that, until recently, were impossible to achieve. Looking at the human “machine” and deriving values and probability from it is nothing new. Peering through the annals of time, mankind has always sought to understand the body and its functions, sometimes uncovering wisdom in unconventional ways.
For all we’ve been able to accomplish in our enlightened state of being, our replication of biological and neural states has been academic. Sure, we’ve replicated simple neural pairings; we’ve written exhaustively on how these mechanisms should work and operate. We have copious amounts of research on convolution, spiking, and simplistic neural networks and we can demonstrate workable solutions with these models. We’ve built silicon, established algorithms, examined and emulated systems over and over in software and hardware and yet…we still haven’t been able to reach that pinnacle that the man-machine exhibits on a daily basis. In a way, we are looking at stateful solutions for stateless being.
In essence, the human neurological componentry exists as both stateful exchanges and stateless processing. Our statefulness is taken from the action potentials, the neural paths carved in sodium ion channels that elicit actions and reactions, and consequently etch memories correspondingly in a process we determine is learning. We all can remember this process through the simplest of examples: “Don’t touch the stove; it is hot.” Stove being touched, neurons firing and delivering signals that denote pain, etc. We don’t touch the stove ever again on purpose. We learn, we develop, we grow.
Our statelessness, as near as I can determine, is derived from the non-binary values that we process and understand. We have a complexity of emotions (themselves “states” though I’d argue they are inherently mutable and alterable even without holding fast to basic neurology) that are secondary functions or processes attached to our being. In the example of touching the stove above, pain is transmitted, tears are shed and an emotion is attached to said interaction. The event is quite binary or neurologically simple: a threshold is exceeded, the neuron fires, transmission, resting, and reset. What isn’t simple are the ancillary emotions tied to this event. How does one derive “fear” from a binary operation? Or then proceed from fear to casual dismissal of the same? How do these secondary functions apply back to the source data or action?
Yes, there is research into how emotions are set within neurological systems. But the curious part about neural networks as we’ve approached them in technology is that I’ve not seen these secondary states accounted for. The primacy of what we’re aiming for is data processing and we either cascade these neural networks one after the other in a type of information sieving or, we attempt to chain neural networks based on some sort of action potentials (triggered events/thresholds/etc.) and as such, each is responsible for one “thing” that is either the start or end of a data chain. This secondary “emotive” function that we see expressed biologically, then, is negated. However, isn’t that data of importance as well? This isn’t cast-off data that doesn’t meet the neural network criteria (e.g. a picture of a cat instead of a human face in facial recognition routines) and gets binned as “not valid.” Rather, this is trace data that has value against the original action or is dimensional to it.
As I said in the beginning, there are some straw men in here and perhaps some wrong ideologies. However you wish to approach it, I’d be interested in learning and your (constructive) thoughts.