Class AbstractLayer<I extends TensorBase, O extends TensorBase, W extends TensorBase>

java.lang.Object
deepnetts.net.layers.AbstractLayer<I,O,W>
Type Parameters:
O - output tensor class
W - weights tensor class
All Implemented Interfaces:
Backward, Forward, Layer<O>, Serializable
Direct Known Subclasses:
CBOWEmbeddingLayer, ConvolutionalLayer, EmbeddingLayer, FlatEmbeddingLayer, FlattenLayer, FullyConnectedLayer, InputLayer, LayerNorm, MaxPoolingLayer, OutputLayer, SkipGramEmbeddingLayer
public abstract class AbstractLayer<I extends TensorBase, O extends TensorBase, W extends TensorBase> extends Object implements Layer<O>, Serializable
Base class for different types of layers. Provides common functionality for all type of layers: layer dimensions, inputs, outputs, connection to previous and/or next layer, activation function and abstract methods for initialization, forward and backward pass.
See Also:

  • Field Details

    • prevLayer

      protected AbstractLayer prevLayer
      Previous layer in network.

    • nextLayer

      protected AbstractLayer nextLayer
      Next layer in network.

    • networkType

      protected NetworkType networkType

    • weights

      protected W extends TensorBase weights
      Input weight matrix / connectivity matrix for previous layer. Used in FullyConnected and OutputLayer. MaxPooling does not have Weights and ConvolutionalLayer has weights in filters.

    • inputs

      protected I extends TensorBase inputs
      Inputs to this layer. A reference to outputs in previous layer, or external input in input layer).

    • outputs

      protected O extends TensorBase outputs
      Layer outputs.

    • deltas

      protected O extends TensorBase deltas
      Deltas used for learning.

    • deltaWeights

      protected W extends TensorBase deltaWeights
      Weight changes for current and previous iteration.

    • prevDeltaWeights

      protected W extends TensorBase prevDeltaWeights
      Weight changes for current and previous iteration.

    • gradients

      protected W extends TensorBase gradients
      Gradients of a loss function calculates during a backward pass.

    • activation

      protected ActivationFunction activation
      Activation function for this layer.

    • activationType

      protected ActivationType activationType
      Type of activation function for this layer.

    • learningRate

      protected float learningRate
      Learning rate for this layer.

    • momentum

      protected float momentum

    • regL2

      protected float regL2

    • regL1

      protected float regL1

    • optimizerType

      protected OptimizerType optimizerType

    • batchMode

      protected boolean batchMode

    • batchSize

      protected int batchSize

    • width

      protected int width

    • height

      protected int height

    • depth

      protected int depth

    • biases

      protected Tensor1D biases

    • deltaBiases

      protected Tensor1D deltaBiases

    • prevDeltaBiases

      protected Tensor1D prevDeltaBiases

    • trainable

      protected boolean trainable

    • optimizer

      protected Optimizer optimizer

    • randomWeightsType

      protected RandomWeightsType randomWeightsType

    • cudaHandles

      protected transient AcceleratorHandle cudaHandles

    • forwardImpl

      protected transient Forward forwardImpl

    • backwardImpl

      protected transient Backward backwardImpl

    • threadPool

      protected transient DeepNettsThreadPool threadPool

    • numThreads

      protected transient int numThreads

    • mode

      protected transient Mode mode

  • Constructor Details

  • Method Details

    • init

      public abstract void init()
      This method should implement layer initialization in subclasses, when a layer is added to the network (create weights, outputs, deltas, randomization etc.).

    • forward

      public abstract void forward()
      This method should implement forward pass in subclasses
      Specified by:
      forward in interface Forward
      Specified by:
      forward in interface Layer<I extends TensorBase>

    • backward

      public abstract void backward()
      This method should implement backward pass in subclasses
      Specified by:
      backward in interface Backward
      Specified by:
      backward in interface Layer<I extends TensorBase>

    • applyWeightChanges

      public abstract void applyWeightChanges()
      Applies weight changes to current weights Must be diferent for convolutional does nothing for MaxPooling Same for FullyConnected and OutputLayer

    • getWidth

      public int getWidth()

    • getHeight

      public int getHeight()

    • getDepth

      public int getDepth()

    • getPrevlayer

      public AbstractLayer getPrevlayer()

    • setPrevLayer

      public void setPrevLayer(AbstractLayer prevLayer)

    • setNextlayer

      public void setNextlayer(AbstractLayer nextlayer)

    • getNextLayer

      public AbstractLayer getNextLayer()

    • getNetworkType

      public NetworkType getNetworkType()

    • setNetworkType

      public void setNetworkType(NetworkType networkType)

    • getWeights

      public W getWeights()

    • getBiases

      public Tensor1D getBiases()

    • setBiases

      public void setBiases(Tensor1D biases)

    • getOutputs

      public final O getOutputs()
      Description copied from interface: Layer
      Returns output of this layer (as a tensor).
      Specified by:
      getOutputs in interface Layer<I extends TensorBase>
      Returns:
      layer output as a tensor

    • getDeltas

      public final O getDeltas()
      Description copied from interface: Layer
      Returns layer deltas/errors (as a tensor). Deltas are accumulated errors propagated from the next layer.
      Specified by:
      getDeltas in interface Layer<I extends TensorBase>
      Returns:
      layer deltas tensor

    • getGradients

      public final W getGradients()

    • getDeltaWeights

      public W getDeltaWeights()

    • getPrevDeltaWeights

      public W getPrevDeltaWeights()

    • setPrevDeltaWeights

      public void setPrevDeltaWeights(W prevDeltaWeights)

    • getPrevDeltaBiases

      public Tensor1D getPrevDeltaBiases()

    • getDeltaBiases

      public Tensor1D getDeltaBiases()

    • setOutputs

      public final void setOutputs(O outputs)

    • setWeights

      public void setWeights(W weights)

    • setWeights

      public void setWeights(String weightStr)

    • setDeltas

      public final void setDeltas(O deltas)

    • getActivation

      public ActivationFunction getActivation()

    • getOptimizer

      public Optimizer getOptimizer()

    • getLearningRate

      public float getLearningRate()

    • setLearningRate

      public void setLearningRate(float learningRate)

    • isBatchMode

      public boolean isBatchMode()

    • setBatchMode

      public void setBatchMode(boolean batchMode)

    • getBatchSize

      public int getBatchSize()

    • setBatchSize

      public void setBatchSize(int batchSize)

    • setMomentum

      public void setMomentum(float momentum)

    • getMomentum

      public float getMomentum()

    • getOptimizerType

      public OptimizerType getOptimizerType()

    • setOptimizerType

      public void setOptimizerType(OptimizerType optType)

    • getActivationType

      public ActivationType getActivationType()

    • setActivationType

      public final void setActivationType(ActivationType activationType)

    • getL1WeightSum

      public float getL1WeightSum()

    • getL2WeightSum

      public float getL2WeightSum()

    • getL2Regularization

      public float getL2Regularization()

    • setL2Regularization

      public void setL2Regularization(float regL2)

    • getL1Regularization

      public float getL1Regularization()

    • setL1Regularization

      public void setL1Regularization(float regL1)

    • isTrainable

      public boolean isTrainable()

    • setTrainable

      public void setTrainable(boolean trainable)
      Set trainable to false to freeze learned weights.
      Parameters:
      trainable -

    • initTransientFields

      public void initTransientFields()

    • setCudaHandles

      public void setCudaHandles(AcceleratorHandle cudaHandles)

    • getForwardAcc

      public Forward getForwardAcc()

    • getBackwardAcc

      public Backward getBackwardAcc()

    • getMode

      public Mode getMode()

    • setMode

      public void setMode(Mode mode)

    • setThreadPool

      public void setThreadPool(DeepNettsThreadPool threadPool)

    • getNumThreads

      protected int getNumThreads(int numElements)