|
1.2 Artificial neurons
Again for pedagogical purposes, we will separate the workings of an
artificial neuron into its input and its output. Note how the sections
correspond to those of the biological neuron.
1.2.1 Input
In modelling a biological neuron we will ignore all temporal effects
and simply consider the inputs and outputs to be signals that we can
represent by numbers which are constant in time. A simple sum of all
inputs is a reasonable model for the input process, so long as each
input is multiplied by a number representing its relative connection
strength, and assuming that inhibitory inputs have negative sign so
that subtracted from the total rather than added. In short, the net
input to the artificial neuron is a weighted sum of its inputs, where
the weights may be either negative or positive.
1.2.2 Output
Note that in a biological neuron there is no significant output from
the neuron when the net input is below some minimum value. As the net
input rises the output also rises until it nears its maximum
value. However after this the output saturates and the rate of
increase drops until the output approaches it limiting value more and
more gradually.
The function that models this process is called an "activation
function" (sometimes called a transfer function). This function has
the shape shown below. The bias term is necessary to model the variability
in the position at which the net input causes the output to rise from its
minimum to its maximum value. In effect, the bias has the effect of shifting
the curve to the right or left.
1.2.3 Summary
It is common to discuss artificial neurons both in terms of a formula
and in terms of a diagram similar to that pictured above. The
meaning of the diagram is that there are several input signals, each
of which is multiplied by the weight as described above. Each of
these values are added together to form a weighted sum, which is then
passed into the activation function. The output from the activation
function is the output from the artificial neuron.
1.3 The formulae of artificial neurons
This section describes the formulae used in the model of a neuron (the
artificial neuron). It may be skipped on a first reading.
A single neuron may be written mathematically as follows:
y = f(bias + w1*x1 + w2*x2 + w3*x3 + ....... + wn*xn)
y is the output, x1, x2, etc., are the inputs (there are n of them)
and f() is the activation function. The most common formula used for
this function is called the "sigmoid":
f(x) = 1/(1+exp(-x))
This function has the shape of the graph shown above.
Note that an artificial neuron with n inputs requires n+1 parameter
values to be specified, namely the bias and the n weights. Other
formula are also sometimes used for f(x) in place of the sigmoid.
1.4 Comparison between artificial and biological neurons
Biological neurons are much more complicated than our artificial one.
Two of the more important ways in which our model differs from the
biological are:
- In a real biological neuron, if the net input is large enough to
cause an output near the maximum, and if this input persists for
more than a few seconds, the output will decline. This is
called the fatigue effect and is not present in our model.
- The response of a real neuron to changes in its net input is
delayed by a few milliseconds because the electrochemical
processes inside it travel with a limited speed. Our model
neuron reacts instantaneously.
There have been computer models that simulate these and other
phenomena, but we will not consider or use them.
Back to top. >>>
Continue with the tutorial. >>>
|
