Herself’s Artificial Intelligence

Kohnonen Neural Nets (Self Organizing Networks)

The Kohonen Self Organizing Map (Network) uses unsupervised, competitive learning. These networks are used for data clustering as in, speech recognition and handwriting recognition. They are also used for sparsely distributed data. Self Organizing Networks consist of two layers, an input layer and a Kohonen layer.The input layer has a node for each dimension of the input vector. The input nodes distribute the input pattern to each node in the the Kohonen layer so the two layers are fully connected. The output layer has at least as many nodes as categories to be recognized. One neurode in the output layer will be activated for each pattern. Each input is connected to each output and there are no connections between the layers.

The network uses lateral inhibition, which is how the vision system works in people. Connections are formed to neighboring neurodes which are inhibitory. The strength of the neurode is inversely proportional to the distance it is away from other nodes. The neurode with the strongest signal dampens the neurodes close to it using a Mexican Hat function. (so called because it looks like a Mexican hat.) The Mexican Hat function is also used in wavelets and image processing..

The neurodes close to the one activated take part in the training, the others do not. To make it computationally efficient a step function is used instead of a true Mexican hat function.

General algorithm
The weights between the nodes are initialized to random values between 0.0 and 1.0.
Then the weight vector is normalized.
The learning rate is set between 1.0 and 0.0 and decreased linearly each iteration.
The neighborhood size is set and decreased linearly each iteration The input vector is normalized and fed into the network.
The input vector is multiplied by the connection weights and the total is accumulated by the Kohonen network nodes.

The winning nodes out put is set to one and all the other nodes are set to zero.
Weights are adjusted Wnew = Wold + training constant ( input Wold) Training continues until a winning node vector meets some minimum error standard.
SelfOrganizing Network example