BSSEvalBase Class

Base class for both BSS Eval algorithms (BSSEvalSources and BSSEvalImages). Contains most of the logic for these base classes.

class nussl.evaluation.bss_eval_base.BSSEvalBase(true_sources_list, estimated_sources_list, source_labels=None, algorithm_name=None, do_mono=False, compute_permutation=True)

Base class for mir_eval implementation of the BSS-Eval metrics (SDR, SIR, SAR). Contains logic for loading ground truth AudioSignal`s and estimated :class:`AudioSignal`s to compute BSS-Eval metrics. The ``mir_eval` module contains an implementation of BSS-Eval version 3.

The BSS-Eval metrics attempt to measure perceptual quality by comparing sources estimated from a source separation algorithm to the ground truth, known sources. These metrics evaluate the distortion (SDR) and artifacts (SAR) present in the estimated signals as well as the interference (SIR) from other sources in a given estimated source. Results are returned in units of dB, with higher values indicating better quality.

See also

  • For more information on mir_eval (python implementation of BSS-Eval v3) see

its Github page<https://github.com/craffel/mir_eval>. * For more information on the BSS-Eval metrics, see the webpage for the original MATLAB implementation<http://bass-db.gforge.inria.fr/bss_eval/>. * Implementations of this base class: BSSEvalSources and BSSEvalImages. * BSSEvalV4 for the museval version 4 BSS-Eval implementation.

References

  • Emmanuel Vincent, Rémi Gribonval, Cédric Févotte. Performance measurement in blind

audio source separation. IEEE Transactions on Audio, Speech and Language Processing, Institute of Electrical and Electronics Engineers, 2006, 14 (4), pp.1462–1469. <inria-00544230> * Colin Raffel, Brian McFee, Eric J. Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel P. W. Ellis, “mir_eval: A Transparent Implementation of Common MIR Metrics”, Proceedings of the 15th International Conference on Music Information Retrieval, 2014.

Parameters:
  • true_sources_list (list) – List of AudioSignal objects that contain the ground truth sources for the mixture.
  • estimated_sources_list (list) – List of AudioSignal objects that contain estimate sources, output from source separation algorithms.
  • source_labels (list) – List of strings that are labels for each source to be used as keys for the scores. Default value is None and in that case labels are Source 0, Source 1, etc.
  • algorithm_name (str) – Name of the algorithm if using this object to compute many BSS-Eval metrics. Can be changed later.
  • do_mono (bool) – Should flatten the audio to mono before calculating metrics.
  • compute_permutation (bool) – Should try to find the best permutation for the estimated sources.
SDR = 'SDR'
SIR = 'SIR'
SAR = 'SAR'
ISR = 'ISR'
PERMUTATION = 'permutation'
RAW_VALUES = 'raw_values'
algorithm_name

Name of the algorithm that is being evaluated. :returns: (str) Name of the algorithm being evaluated.

validate()

Checks to make sure the all of the input AudioSignal objects have the same length.

_preprocess_sources()

Prepare the audio_data in the sources for mir_eval. :returns: (np.ndarray, np.ndarray) reference_source_array, estimated_source_array

evaluate()

Actually runs the evaluation algorithm. Will be museval.metrics.bss_eval_images or museval.metrics.bss_eval_sources depending on which subclass is instantiated. :returns: (dict) – Dictionary containing the scores.

_populate_scores_dict(bss_output)

Formats and populates the scores dict from evaluate(). :param bss_output: Direct output from the museval function. :type bss_output: tuple

Returns:(dict) Reformatted dictionary from museval output.
exception nussl.evaluation.bss_eval_base.BssEvalException

Exception class for BSS-Eval