Draft: this article is a draft, its content will change.
Keras Sequential API (KSA) allows to easily create Deep Learning networks that are made of a sequence of stack of layers. The nice thing about KSA is that we only have to provide the model with the input shape of our data and Keras will take care of managing the shapes of subsequent layers.
The input shape must be passed as an argument to the first layer of our model using the argument
input_length (it depends on the kind of layer being used); the batch dimension must not be specified.
Once a model has been specified we need to compile it to the backend that will be used , e.g. if you want to use Tensorflow as the backend framework Keras will translate the model we just specified into a Tensorflow model so that the training of our network will effectively happen using Tensorflow.
After compiling a model, you have two ways to train it: either you use the
fit method for your model or you use the
fit method you have to pass your dataset and the corresponding labels to your model; this implies that you have loaded all your data in memory, which might be unfeasible if you are training your model on a very large dataset.
To overcome this limitation Keras provide the "fit_generator" model that allows to read the data in batches so that we can avoid to fill the memory with the whole dataset. Instead of passing a dataset to train the model, we pass a generator that returns batches of data and corresponding labels.
Although we can pass a normal Python generator that returns a tuple with the data and labels respectively (and eventually a third element with the associated weights), in Keras documentation they recommend to extend the class
There are two major differences in implementing your own generator depending on whether you want to use a generator or a Sequence-like class:
if you use a generator, the generator has to provide data indefinitely,and you have to tell the fit_generator function after how many batches the epoch can be considered as ended. In some circumstances this means you have to manage yourself making sure the dataset is reshuffled after the appropriate number of batches and so on; also generator are not easily pickable and this can create some issues if you want to run training with multi-processing and some on;
if you extend the
keras.utils.Sequence class, you have to implement three mandatory methods:
__init__ method instantiates your Iterator class and should be used to prepare all the class variables to be returned from your batches.
__getitem__ takes as input the number of the batch to return (numeration starts from 0 like for lists, so to get the 4th batch, you have to consider getitem will get number 3 as input).
__len__ should return the number of batches present in the epoch and it will be used by
fit_generator() to decide when an epoch has ended. Optionally you can implement the
on_epoch_end() method that is called once the epoch training has finished and should be used to prepare the data to be trained in the next epoch (for example, if you want to reshuffle your data after each epoch, this is the right place to do so).