Package ai.djl.training.loss

Class Loss

java.lang.Object
ai.djl.training.evaluator.Evaluator
ai.djl.training.loss.Loss
Direct Known Subclasses:
AbstractCompositeLoss, BertMaskedLanguageModelLoss, BertNextSentenceLoss, ElasticNetWeightDecay, HingeLoss, IndexLoss, L1Loss, L1WeightDecay, L2Loss, L2WeightDecay, MaskedSoftmaxCrossEntropyLoss, QuantileL1Loss, SigmoidBinaryCrossEntropyLoss, SoftmaxCrossEntropyLoss, TabNetClassificationLoss, TabNetRegressionLoss, YOLOv3Loss
public abstract class Loss extends Evaluator
Loss functions (or Cost functions) are used to evaluate the model predictions against true labels for optimization.

Although all evaluators can be used to measure the performance of a model, not all of them are suited to being used by an optimizer. Loss functions are usually non-negative where a larger loss represents worse performance. They are also real-valued to accurately compare models.

When creating a loss function, you should avoid having the loss depend on the batch size. For example, if you have a loss per item in a batch and sum those losses, your loss would be numItemsInBatch*avgLoss. Instead, you should take the mean of those losses to reduce out the batchSize factor. Otherwise, it can make it difficult to tune the learning rate since any change in the batch size would throw it off. If you have a variable batch size, it would be even more difficult.

For more details about the class internals, see Evaluator.

  • Constructor Details

    • Loss

      public Loss(String name)
      Base class for metric with abstract update methods.
      Parameters:
      name - The display name of the Loss

  • Method Details

    • l1Loss

      public static L1Loss l1Loss()
      Returns a new instance of L1Loss with default weight and batch axis.
      Returns:
      a new instance of L1Loss

    • l1Loss

      public static L1Loss l1Loss(String name)
      Returns a new instance of L1Loss with default weight and batch axis.
      Parameters:
      name - the name of the loss
      Returns:
      a new instance of L1Loss

    • l1Loss

      public static L1Loss l1Loss(String name, float weight)
      Returns a new instance of L1Loss with given weight.
      Parameters:
      name - the name of the loss
      weight - the weight to apply on loss value, default 1
      Returns:
      a new instance of L1Loss

    • quantileL1Loss

      public static QuantileL1Loss quantileL1Loss(float quantile)
      Returns a new instance of QuantileL1Loss with given quantile.
      Parameters:
      quantile - the quantile position of the data to focus on
      Returns:
      a new instance of QuantileL1Loss

    • quantileL1Loss

      public static QuantileL1Loss quantileL1Loss(String name, float quantile)
      Returns a new instance of QuantileL1Loss with given quantile.
      Parameters:
      name - the name of the loss
      quantile - the quantile position of the data to focus on
      Returns:
      a new instance of QuantileL1Loss

    • l2Loss

      public static L2Loss l2Loss()
      Returns a new instance of L2Loss with default weight and batch axis.
      Returns:
      a new instance of L2Loss

    • l2Loss

      public static L2Loss l2Loss(String name)
      Returns a new instance of L2Loss with default weight and batch axis.
      Parameters:
      name - the name of the loss
      Returns:
      a new instance of L2Loss

    • l2Loss

      public static L2Loss l2Loss(String name, float weight)
      Returns a new instance of L2Loss with given weight and batch axis.
      Parameters:
      name - the name of the loss
      weight - the weight to apply on loss value, default 1
      Returns:
      a new instance of L2Loss

    • sigmoidBinaryCrossEntropyLoss

      public static SigmoidBinaryCrossEntropyLoss sigmoidBinaryCrossEntropyLoss()
      Returns a new instance of SigmoidBinaryCrossEntropyLoss with default arguments.
      Returns:
      a new instance of SigmoidBinaryCrossEntropyLoss

    • sigmoidBinaryCrossEntropyLoss

      public static SigmoidBinaryCrossEntropyLoss sigmoidBinaryCrossEntropyLoss(String name)
      Returns a new instance of SigmoidBinaryCrossEntropyLoss with default arguments.
      Parameters:
      name - the name of the loss
      Returns:
      a new instance of SigmoidBinaryCrossEntropyLoss

    • sigmoidBinaryCrossEntropyLoss

      public static SigmoidBinaryCrossEntropyLoss sigmoidBinaryCrossEntropyLoss(String name, float weight, boolean fromSigmoid)
      Returns a new instance of SigmoidBinaryCrossEntropyLoss with the given arguments.
      Parameters:
      name - the name of the loss
      weight - the weight to apply on the loss value, default 1
      fromSigmoid - whether the input is from the output of sigmoid, default false
      Returns:
      a new instance of SigmoidBinaryCrossEntropyLoss

    • softmaxCrossEntropyLoss

      public static SoftmaxCrossEntropyLoss softmaxCrossEntropyLoss()
      Returns a new instance of SoftmaxCrossEntropyLoss with default arguments.
      Returns:
      a new instance of SoftmaxCrossEntropyLoss

    • softmaxCrossEntropyLoss

      public static SoftmaxCrossEntropyLoss softmaxCrossEntropyLoss(String name)
      Returns a new instance of SoftmaxCrossEntropyLoss with default arguments.
      Parameters:
      name - the name of the loss
      Returns:
      a new instance of SoftmaxCrossEntropyLoss

    • softmaxCrossEntropyLoss

      public static SoftmaxCrossEntropyLoss softmaxCrossEntropyLoss(String name, float weight, int classAxis, boolean sparseLabel, boolean fromLogit)
      Returns a new instance of SoftmaxCrossEntropyLoss with the given arguments.
      Parameters:
      name - the name of the loss
      weight - the weight to apply on the loss value, default 1
      classAxis - the axis that represents the class probabilities, default -1
      sparseLabel - whether labels are integer array or probabilities, default true
      fromLogit - whether labels are log probabilities or un-normalized numbers
      Returns:
      a new instance of SoftmaxCrossEntropyLoss

    • maskedSoftmaxCrossEntropyLoss

      public static MaskedSoftmaxCrossEntropyLoss maskedSoftmaxCrossEntropyLoss()
      Returns a new instance of MaskedSoftmaxCrossEntropyLoss with default arguments.
      Returns:
      a new instance of MaskedSoftmaxCrossEntropyLoss

    • maskedSoftmaxCrossEntropyLoss

      public static MaskedSoftmaxCrossEntropyLoss maskedSoftmaxCrossEntropyLoss(String name)
      Returns a new instance of MaskedSoftmaxCrossEntropyLoss with default arguments.
      Parameters:
      name - the name of the loss
      Returns:
      a new instance of MaskedSoftmaxCrossEntropyLoss

    • maskedSoftmaxCrossEntropyLoss

      public static MaskedSoftmaxCrossEntropyLoss maskedSoftmaxCrossEntropyLoss(String name, float weight, int classAxis, boolean sparseLabel, boolean fromLogit)
      Returns a new instance of MaskedSoftmaxCrossEntropyLoss with the given arguments.
      Parameters:
      name - the name of the loss
      weight - the weight to apply on the loss value, default 1
      classAxis - the axis that represents the class probabilities, default -1
      sparseLabel - whether labels are integer array or probabilities, default true
      fromLogit - whether labels are log probabilities or un-normalized numbers
      Returns:
      a new instance of MaskedSoftmaxCrossEntropyLoss

    • hingeLoss

      public static HingeLoss hingeLoss()
      Returns a new instance of HingeLoss with default arguments.
      Returns:
      a new instance of HingeLoss

    • hingeLoss

      public static HingeLoss hingeLoss(String name)
      Returns a new instance of HingeLoss with default arguments.
      Parameters:
      name - the name of the loss
      Returns:
      a new instance of HingeLoss

    • hingeLoss

      public static HingeLoss hingeLoss(String name, int margin, float weight)
      Returns a new instance of HingeLoss with the given arguments.
      Parameters:
      name - the name of the loss
      margin - the margin in hinge loss. Defaults to 1.0
      weight - the weight to apply on loss value, default 1
      Returns:
      a new instance of HingeLoss

    • l1WeightedDecay

      public static L1WeightDecay l1WeightedDecay(NDList parameters)
      Returns a new instance of L1WeightDecay with default weight and name.
      Parameters:
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of L1WeightDecay

    • l1WeightedDecay

      public static L1WeightDecay l1WeightedDecay(String name, NDList parameters)
      Returns a new instance of L1WeightDecay with default weight.
      Parameters:
      name - the name of the weight decay
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of L1WeightDecay

    • l1WeightedDecay

      public static L1WeightDecay l1WeightedDecay(String name, float weight, NDList parameters)
      Returns a new instance of L1WeightDecay.
      Parameters:
      name - the name of the weight decay
      weight - the weight to apply on weight decay value, default 1
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of L1WeightDecay

    • l2WeightedDecay

      public static L2WeightDecay l2WeightedDecay(NDList parameters)
      Returns a new instance of L2WeightDecay with default weight and name.
      Parameters:
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of L2WeightDecay

    • l2WeightedDecay

      public static L2WeightDecay l2WeightedDecay(String name, NDList parameters)
      Returns a new instance of L2WeightDecay with default weight.
      Parameters:
      name - the name of the weight decay
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of L2WeightDecay

    • l2WeightedDecay

      public static L2WeightDecay l2WeightedDecay(String name, float weight, NDList parameters)
      Returns a new instance of L2WeightDecay.
      Parameters:
      name - the name of the weight decay
      weight - the weight to apply on weight decay value, default 1
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of L2WeightDecay

    • elasticNetWeightedDecay

      public static ElasticNetWeightDecay elasticNetWeightedDecay(NDList parameters)
      Returns a new instance of ElasticNetWeightDecay with default weight and name.
      Parameters:
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of ElasticNetWeightDecay

    • elasticNetWeightedDecay

      public static ElasticNetWeightDecay elasticNetWeightedDecay(String name, NDList parameters)
      Returns a new instance of ElasticNetWeightDecay with default weight.
      Parameters:
      name - the name of the weight decay
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of ElasticNetWeightDecay

    • elasticNetWeightedDecay

      public static ElasticNetWeightDecay elasticNetWeightedDecay(String name, float weight, NDList parameters)
      Returns a new instance of ElasticNetWeightDecay.
      Parameters:
      name - the name of the weight decay
      weight - the weight to apply on weight decay values, default 1
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of ElasticNetWeightDecay

    • elasticNetWeightedDecay

      public static ElasticNetWeightDecay elasticNetWeightedDecay(String name, float weight1, float weight2, NDList parameters)
      Returns a new instance of ElasticNetWeightDecay.
      Parameters:
      name - the name of the weight decay
      weight1 - the weight to apply on weight decay L1 value, default 1
      weight2 - the weight to apply on weight decay L2 value, default 1
      parameters - holds the model weights that will be penalized
      Returns:
      a new instance of ElasticNetWeightDecay

    • addAccumulator

      public void addAccumulator(String key)
      Adds an accumulator for the results of the evaluation with the given key.
      Specified by:
      addAccumulator in class Evaluator
      Parameters:
      key - the key for the new accumulator

    • updateAccumulator

      public void updateAccumulator(String key, NDList labels, NDList predictions)
      Updates the evaluator with the given key based on a NDList of labels and predictions.

      This is a synchronized operation. You should only call it at the end of a batch or epoch.

      Specified by:
      updateAccumulator in class Evaluator
      Parameters:
      key - the key of the accumulator to update
      labels - a NDList of labels
      predictions - a NDList of predictions

    • updateAccumulators

      public void updateAccumulators(String[] keys, NDList labels, NDList predictions)
      Updates the evaluator with the given keys based on a NDList of labels and predictions.

      This is a synchronized operation. You should only call it at the end of a batch or epoch.

      This is an alternative to @{link Evaluator.updateAccumulator(String, NDList, NDList)} that may be more efficient when updating multiple accumulators at once.

      Overrides:
      updateAccumulators in class Evaluator
      Parameters:
      keys - the keys of all the accumulators to update
      labels - a NDList of labels
      predictions - a NDList of predictions

    • resetAccumulator

      public void resetAccumulator(String key)
      Resets the evaluator value with the given key.
      Specified by:
      resetAccumulator in class Evaluator
      Parameters:
      key - the key of the accumulator to reset

    • getAccumulator

      public float getAccumulator(String key)
      Returns the accumulated evaluator value.
      Specified by:
      getAccumulator in class Evaluator
      Parameters:
      key - the key of the accumulator to get
      Returns:
      the accumulated value