Package ai.djl.training.evaluator

Class Evaluator

java.lang.Object
ai.djl.training.evaluator.Evaluator
Direct Known Subclasses:
AbstractAccuracy, BoundingBoxError, IndexEvaluator, Loss
public abstract class Evaluator extends Object
Base class for all Evaluators that can be used to evaluate the performance of a model.

The Evaluators can all be monitored to make an assessment about the performance of the model. However, only ones that further extend Loss are suited to being used to directly optimize a model.

In addition to computing the evaluation, an evaluator can accumulate values to compute a total evaluation. For different purposes, it is possible to have multiple accumulators on a single evaluator. Each accumulator must be added with a String key to identify the accumulator. Before using an accumulator, you must addAccumulator(String). Then, call updateAccumulator(String, NDList, NDList) to add more data to the accumulator. You can use getAccumulator(String) to retrieve the accumulated value and resetAccumulator(String) to reset the accumulator to the same value as when just added.

  • Field Details

  • Constructor Details

    • Evaluator

      public Evaluator(String name)
      Creates an evaluator with abstract update methods.
      Parameters:
      name - the name of the evaluator

  • Method Details

    • getName

      public String getName()
      Returns the name of this Evaluator.
      Returns:
      the name of this Evaluator

    • evaluate

      public abstract NDArray evaluate(NDList labels, NDList predictions)
      Calculates the evaluation between the labels and the predictions.
      Parameters:
      labels - the correct values
      predictions - the predicted values
      Returns:
      the evaluation result

    • addAccumulator

      public abstract void addAccumulator(String key)
      Adds an accumulator for the results of the evaluation with the given key.
      Parameters:
      key - the key for the new accumulator

    • updateAccumulators

      public void updateAccumulators(String[] keys, NDList labels, NDList predictions)
      Updates the evaluator with the given keys based on a NDList of labels and predictions.

      This is a synchronized operation. You should only call it at the end of a batch or epoch.

      This is an alternative to @{link updateAccumulator(String, NDList, NDList)} that may be more efficient when updating multiple accumulators at once.

      Parameters:
      keys - the keys of all the accumulators to update
      labels - a NDList of labels
      predictions - a NDList of predictions

    • updateAccumulator

      public abstract void updateAccumulator(String key, NDList labels, NDList predictions)
      Updates the evaluator with the given key based on a NDList of labels and predictions.

      This is a synchronized operation. You should only call it at the end of a batch or epoch.

      Parameters:
      key - the key of the accumulator to update
      labels - a NDList of labels
      predictions - a NDList of predictions

    • resetAccumulator

      public abstract void resetAccumulator(String key)
      Resets the evaluator value with the given key.
      Parameters:
      key - the key of the accumulator to reset

    • getAccumulator

      public abstract float getAccumulator(String key)
      Returns the accumulated evaluator value.
      Parameters:
      key - the key of the accumulator to get
      Returns:
      the accumulated value
      Throws:
      IllegalArgumentException - if no accumulator was added with the given key

    • checkLabelShapes

      protected void checkLabelShapes(NDArray labels, NDArray predictions, boolean checkDimOnly)
      Checks if the two input NDArray have the same length or shape.
      Parameters:
      labels - a NDArray of labels
      predictions - a NDArray of predictions
      checkDimOnly - whether to check for first dimension only

    • checkLabelShapes

      protected void checkLabelShapes(NDArray labels, NDArray predictions)
      Checks the length of NDArrays.
      Parameters:
      labels - a NDArray of labels
      predictions - a NDArray of predictions