Plotting weighted mean activations
Plotting graded activations
Training individual maps
Training more individual maps
Hebbian training on non-symmetric maps
References for SOMs

Two data sets

We have a "left" data set and a "right" data set. There are exactly 90 data points in each set. For the left data set, the data points are randomly distributed within three groups. On the x axis, all of the values are between 0 and 10. On the y axis, the first group of values are between 70 and 80, the second group between 50 and 60, and the third group between 30 and 40. As shown:

For the right data set, there are still 90 data points, but with only three values: (50,75), (50,55), and (50,35). As shown (90 data points plotted):

The data sets are arranged so that all of the data points in the top-most group in the left set are associated with the top data point in the right set, middle with middle, and bottom with bottom.

Initializing maps

For each data set, a self-organizing map is created. Some number of neurons (48) are randomly distributed within the range of each data set. Neurons are shown as green plusses:

We could train each map according to its data, but for now we will leave the neurons randomly distributed.

For each map we create a 48x48 matrix, all zeros. These are the initialized weight matrices. During training, we adjust these weights to learn the associations between the left and right data sets.

Training Hebbian connections

The script som_hebbtrain.m takes in both maps (neurons) and both data sets, and adjusts the weights. At each iteration, associated data points are presented to the maps (one data point to each map). The presentation causes some neighborhood of neurons to be activated in that map. For instance, the pink stars in the chart below show how each map is activated by its own data set:

These activations are then compared and used to train the Hebbian weights. Strong weights develop between areas on the two maps that tend to be activated at the same time. Negative or zero weights develop between areas that are not activated at the same time.

Plotting Hebbian responses

Once the weights are defined, we use them to see how presenting data to one map has some effect on the other map. In the next graph, we present one data point to the left map (red circle). This activates some neighborhood of neurons. We multiply that activation by the Hebbian weights associated with each of those neurons in the left map. This translates to some activation on the right map (blue circle).

The placement of the circle corresponds to the weighted mean of the neurons in the left map, where weights correspond to their activations.

Similarly, we can present a data point to the right map and activate some area on the left map.

Here is what happens when we present all of the left data set to the left map. The blue points on the right map actually form small groups, they do not all fall on the same three places on the y axis, although it is difficult to see.

Here is what happens when we present all of the right data set to the right map. Only three points are activated on the left map. Each time we present a point on the right, the exact same Hebbian connections are activated the same amount. The activations on the left are near the center of each grouping of left data. This happens because, for instance, the neuron activated by the top data point on the right has about equal connections to all of the top neurons on the left. The map is activated in the place about equidistant between all of the active neurons. Similarly for middle and low:

Training the Maps, then Hebbian connections

We can first train the self-organizing maps according to its own data set. The neurons move to better reflect the distribution of data.

Then we train the Hebbian connections as before, and plot the responses. Here is the right map's response to the left map's activations:

Here is the left map's response to the right map's data:

There is not a huge difference, with this data set, of starting with trained or random maps underlying the data.