BSSEvalV4Class

Wrapper class for museval implementation of the BSS-Eval metrics (SDR, SIR, SAR). Contains logic for loading ground truth AudioSignals and estimated AudioSignals to compute BSS-Eval metrics. The mir_eval module contains an implementation of BSS-Eval version 4.

class nussl.evaluation.bss_eval_v4.BSSEvalV4(mixture, true_sources, estimated_sources, target_dict={'accompaniment': {'accompaniment': 1}, 'vocals': {'vocals': 1}}, mode='v4', output_dir=None, win=1.0, hop=1.0)

Wrapper class for museval implementation of the BSS-Eval metrics (SDR, SIR, SAR). Contains logic for loading ground truth AudioSignals and estimated AudioSignals to compute BSS-Eval metrics. The mir_eval module contains an implementation of BSS-Eval version 4.

The BSS-Eval metrics attempt to measure perceptual quality by comparing sources estimated from a source separation algorithm to the ground truth, known sources. These metrics evaluate the distortion (SDR) and artifacts (SAR) present in the estimated signals as well as the interference (SIR) from other sources in a given estimated source. Results are returned in units of dB, with higher values indicating better quality.

Examples

BSSEvalV4 can be initialized in two ways, either with a list or a dict. See the example below for a demonstration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
mir1k_dir = 'path/to/MIR-1K'
mix, vox, acc = next(nussl.datasets.mir1k(mir1k_dir))
mix.to_mono(overwrite=True)

r = nussl.RepetSim(mix)
r()
bg, fg = r.make_audio_signals()

# Method 1: Dictionary where sources are explicit
# Note that this dictionary exactly matches nussl.constants.VOX_ACC_DICT
est_dict = {'vocals': fg, 'accompaniment': bg}
gt_dict = {'vocals': vox, 'accompaniment': acc}
bss = nussl.evaluation.BSSEvalV4(mix, gt_dict, est_dict)
scores1 = bss.evaluate()

# Method 2: List
# Note that vocals are always expected to be first, then accompaniment.
bss = nussl.evaluation.BSSEvalV4(mix, [vox, acc], [fg, bg])
scores2 = bss.evaluate()

See also

  • For more information on museval (python implementation of BSS-Eval v4) see

its Github page or ‘its documentation <https://sigsep.github.io/sigsep-mus-eval/>`_.

  • For more information on the BSS-Eval metrics, see the webpage for

the original MATLAB implementation.

  • BSSEvalSources and BSSEvalImages for the mir_eval version 3

BSS-Eval implementations.

References

  • Emmanuel Vincent, Rémi Gribonval, Cédric Févotte. Performance measurement in blind

audio source separation. IEEE Transactions on Audio, Speech and Language Processing, Institute of Electrical and Electronics Engineers, 2006, 14 (4), pp.1462–1469. <inria-00544230>

  • Fabian-Robert Stöter, Antoine Liutkus, and Nobutaka Ito. The 2018 Signal Separation

Evaluation Campaign. In International Conference on Latent Variable Analysis and Signal Separation, pages 293–305. Springer, 2018.

Parameters:
  • true_sources_list (list) – List of AudioSignal objects that contain the ground truth sources for the mixture.
  • estimated_sources_list (list) – List of AudioSignal objects that contain estimate sources, output from source separation algorithms.
  • source_labels (list) – List of strings that are labels for each source to be used as keys for the scores. Default value is None and in that case labels are Source 0, Source 1, etc.
  • algorithm_name (str) – Name of the algorithm if using this object to compute many BSS-Eval metrics. Can be changed later.
  • do_mono (bool) – Should flatten the audio to mono before calculating metrics.
  • compute_permutation (bool) – Should try to find the best permutation for the estimated sources.
SDR_MEANS = 'sdr_means'
_get_scores(scores)
static _parse(str_)
_get_mean_scores(scores)
evaluate()

Actually runs the evaluation algorithm. Will be museval.metrics.bss_eval_images or museval.metrics.bss_eval_sources depending on which subclass is instantiated. :returns: (dict) – Dictionary containing the scores.

_populate_scores_dict(bss_output)

Formats and populates the scores dict from evaluate(). :param bss_output: Direct output from the museval function. :type bss_output: tuple

Returns:(dict) Reformatted dictionary from museval output.