Package deepnetts.net.layers

Class FullyConnectedLayer

java.lang.Object

deepnetts.net.layers.AbstractLayer<TensorBase,TensorBase,Tensor2D>

deepnetts.net.layers.FullyConnectedLayer

All Implemented Interfaces:

Layer<TensorBase>, Serializable

public class FullyConnectedLayer
extends AbstractLayer<TensorBase,TensorBase,Tensor2D>

Fully connected layer is used as a hidden layer in a neural network, and it has a single row of units/nodes/neurons connected to all neurons in previous and next layer. Previous layer can be input, fully connected, or flattened layer, while next layer can be fully connected or output layer. This layer calculates weighted sum of outputs from the previous layers (as matrix dot product), and applies activation function to all that sum. Mathematical formula is: Y = activation(W . X + B) where Y is output tensor (1D for single input or 2D for batch) W is a 2D weights tensor X is input tensor (1D for single input or 2D for batch) B is a 1D tensor of biases activation is an activation function

See Also:

• ActivationType
• ActivationFunction
• Serialized Form

Constructor Summary

Constructors

Constructor	Description
FullyConnectedLayer(int size)	Creates an instance of fully connected layer with specified number of neurons and ReLU activation function.
FullyConnectedLayer(int size, ActivationType actType)	Creates an instance of fully connected layer with specified width (number of neurons) and activation function type.

Method Summary

Modifier and Type	Method	Description
void	applyWeightChanges()	Applies weight changes to current weights Must be diferent for convolutional does nothing for MaxPooling Same for FullyConnected and OutputLayer
void	backward()	This method should implement backward pass in subclasses
void	forward()	This method should implement forward pass in subclasses
float	getDropout()
void	init()	Creates all internal data structures: inputs, weights, biases, outputs, deltas, deltaWeights, deltaBiases prevDeltaWeights, prevDeltaBiases.
void	initTransientFields()
void	setDropout(float dropout)
String	toString()

Methods inherited from class deepnetts.net.layers.AbstractLayer

getActivation, getActivationType, getBatchSize, getBiases, getDeltaBiases, getDeltas, getDeltaWeights, getDepth, getForwardAccelerator, getGradients, getHeight, getL1Regularization, getL1WeightSum, getL2Regularization, getL2WeightSum, getLearningRate, getMode, getMomentum, getNextLayer, getOptimizer, getOptimizerType, getOutputs, getPrevDeltaBiases, getPrevDeltaWeights, getPrevlayer, getWeights, getWidth, isBatchMode, isTrainable, setBatchMode, setBatchSize, setBiases, setCudaHandles, setDeltas, setL1Regularization, setL2Regularization, setLearningRate, setMode, setMomentum, setNextlayer, setOptimizerType, setOutputs, setPrevDeltaWeights, setPrevLayer, setThreadPool, setTrainable, setWeights, setWeights

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

FullyConnectedLayer

public FullyConnectedLayer(int size)

Creates an instance of fully connected layer with specified number of neurons and ReLU activation function.

Parameters:

size - number of neurons in this layer / layer size

FullyConnectedLayer

public FullyConnectedLayer(int size,
 ActivationType actType)

Creates an instance of fully connected layer with specified width (number of neurons) and activation function type.

Parameters:

size - layer width / number of neurons in this layer

actType - activation function type to use in this layer

See Also:

• invalid reference

  ActivationFunctions

Method Details

init

public void init()

Creates all internal data structures: inputs, weights, biases, outputs, deltas, deltaWeights, deltaBiases prevDeltaWeights, prevDeltaBiases. Init weights and biases. This method is called from network builder during initialization

Specified by:

init in class AbstractLayer<TensorBase,TensorBase,Tensor2D>

initTransientFields

public void initTransientFields()

Overrides:

initTransientFields in class AbstractLayer<TensorBase,TensorBase,Tensor2D>

forward

public void forward()

Description copied from class: AbstractLayer

This method should implement forward pass in subclasses

Specified by:

forward in interface Layer<TensorBase>

Specified by:

forward in class AbstractLayer<TensorBase,TensorBase,Tensor2D>

backward

public void backward()

Description copied from class: AbstractLayer

This method should implement backward pass in subclasses

Specified by:

backward in interface Layer<TensorBase>

Specified by:

backward in class AbstractLayer<TensorBase,TensorBase,Tensor2D>

applyWeightChanges

public void applyWeightChanges()

Description copied from class: AbstractLayer

Applies weight changes to current weights Must be diferent for convolutional does nothing for MaxPooling Same for FullyConnected and OutputLayer

Specified by:

applyWeightChanges in class AbstractLayer<TensorBase,TensorBase,Tensor2D>

getDropout

public float getDropout()

setDropout

public void setDropout(float dropout)

toString

public String toString()

Overrides:

toString in class Object