Class SgdOptimizer

java.lang.Object
deepnetts.net.train.opt.SgdOptimizer
All Implemented Interfaces:
Optimizer, Serializable
public final class SgdOptimizer extends Object implements Optimizer, Serializable
Basic Stochastic Gradient Descent optimization algorithm, which iteratively changes weights in order to find minimum of loss function.
See Also:

  • Constructor Details

  • Method Details

    • calculateDeltaWeight

      public float calculateDeltaWeight(float gradient, int... index)
      Description copied from interface: Optimizer
      Smoothing term to prevent division by zero if sqr grad sum becomes zero 1e-8 should be also tried https://d2l.ai/chapter_optimization/adagrad.html 1e-6 The value to use is 1e-6, 1e-8, Keras uses 1e-7 for adam
      Specified by:
      calculateDeltaWeight in interface Optimizer

    • calculateDeltaBias

      public float calculateDeltaBias(float gradient, int idx)
      Specified by:
      calculateDeltaBias in interface Optimizer

    • setLearningRate

      public void setLearningRate(float learningRate)
      Specified by:
      setLearningRate in interface Optimizer

    • getLearningRate

      public float getLearningRate()

    • setBiasLearningRate

      public void setBiasLearningRate(float biasLearningRate)

    • getBiasLearningRate

      public float getBiasLearningRate()

    • calculateDeltaBias

      public Tensor1D calculateDeltaBias(Tensor1D grad)
      Specified by:
      calculateDeltaBias in interface Optimizer

    • calculateDeltaWeight

      public TensorBase calculateDeltaWeight(TensorBase grad)
      Specified by:
      calculateDeltaWeight in interface Optimizer