Page 71 - Contributed Paper Session (CPS) - Volume 7

CPS2031 Javier Linkolk L. et al.
The "entry door" activates the entry of information to the "memory cell", and
"the forgetting door" selectively erases certain information in the cell memory
and activates the storage to the next entry [14]. Finally, "the exit door" decides
what information the memory cell will emit [15]. The LSTM network structure
is illustrated in figure 1. At each time step, an LSTM maintains a hidden vector
h and a memory vector m responsible for controlling updates and outputs of
the state. More concretely, Graves et al. [16] define the computation at time
step t as follows:
gu =σ(Wuht1+Iuxt)
gf =σ(Wfht1+Ifxt)
go =σ(Woht1+Ioxt)
gc =tanh(Wcht1+Icxt)
u
mt = gf 0 mt1 + g 0 gc
ht =tanh(g 0mt)
o

where σ is the logistic sigmoid function, ⊙ represents elementwise
multiplication, Wu , Wf , Wo , Wc are recurrent weight matrices and Iu , If , Io
, Ic are projection matrices [17].
In addition, they classify and predict based on time series data, since, there
may be delays of unknown duration between important events in a series of
time. It allows clearly remembering events selected from far away in the past,
which contrasts with basic NRs, for which the memory of an event decays over
time [18].

Self-Organizing Ensemble of LSTM
The proposal is aimed at how to find temporary pollution groups and also
improve the estimation with LSTM networks, that is, a SOM + LSTM assembly
would be developed.
It is important to mention that each neuron is a network. Virtually, it
becomes a neuron properly, when evaluating the membership of the data,
however, when predicting is a whole network.
Each node is a network that specialized in a dataset. The time series are
grouped, and one of the neurons will be the one that best represents the time
series. However, the same neuron becomes an LSTM, and being the one that
best models the series, delivers the prediction. Inclusive, the neighbouring
neurons, also contribute relevant information, by principle of the self‐
organization as neighbourhood of the winning unit.

58 | I S I W S C 2 0 1 9