EvaluationBase Class

EvaluationBase is the base class for all evaluation methods in nussl.

class nussl.evaluation.evaluation_base.EvaluationBase(true_sources_list, estimated_sources_list, source_labels=None, **kwargs)

Base class for all Evaluation classes for source separation algorithm in nussl. Contains common functions for all evaluation techniques. This class should not be instantiated directly.

Both true_sources_list and estimated_sources_list get validated using the private method _verify_input_list(). If your evaluation needs to verify that input is set correctly (recommended) overwrite that method to add checking.

Parameters:
  • true_sources_list (list) – List of objects that contain one ground truth source per object. In some instances (such as the BSSEval objects) this list is filled with AudioSignals but in other cases it is populated with MaskBase -derived objects (i.e., either a BinaryMask or SoftMask object).
  • estimated_sources_list (list) – List of objects that contain source estimations from a source separation algorithm. List should be populated with the same type of objects and in the same order as :param:`true_sources_list`.
  • source_labels (list) – List of strings that are labels for each source to be used as keys for the scores. Default value is None and in that case labels are Source 0, Source 1, etc.
  • do_mono (bool) – Whether to make the objects in true_sources_list and estimated_sources_list mono prior to doing the evaluation. This may not work for all EvaluationBase-derived objects.
  • **kwargs – Additional arguments for subclasses.
static _verify_input_list(audio_signal_list)

Base method for verifying a list of input objects for an EvaluationBase-derived object. Override this method when creating new EvaluationBased-derived class.

By default calls nussl.utils.verify_audio_signal_list_strict(), which verifies that all objects in the input list are audio_signal.AudioSignal objects with the same length, sample rate and have identical number of channels.

Parameters:audio_signal_list (list) – List of objects that contain one signal per object. In some instances (such as the BSSEval) this list is filled with AudioSignals but in other cases it is populated with MaskBase -derived objects (i.e., either a BinaryMask or SoftMask object). In the latter case, this method is overridden with a specific function in evaluation.precision_recall_fscore.PrecisionRecallFScore.
Returns:A verified list of objects that are ready for running the evaluation method.
evaluate()

This function runs the evaluation method. Do not call this directly from EvaluationBase

Raises:NotImplementedError
scores

A dictionary that stores all scores from the evaluation method. Gets populated when evaluate() gets run.

exception nussl.evaluation.evaluation_base.AudioSignalListMismatchError

Error class for when true_sources_list and estimated_sources_list are different lengths.