efz_utils.py

The nussl External File Zoo (EFZ) is a server that houses all files that are too large to bundle with nussl when distributing it through pip or Github. These types of files include audio examples, benchmark files for tests, and trained neural network models.

nussl has built-in utilities for accessing the EFZ through its API. Here, it is possible to see what files are available on the EFZ and download desired files. The EFZ utilities allow for such functionality.

nussl.efz_utils.get_available_audio_files()

Returns a list of dicts containing metadata of the available audio files on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).

Each entry in the list is in the following format:

{
    u'file_length_seconds': 5.00390022675737,
    u'visible': True,
    u'file_name': u'K0140.wav',
    u'date_modified': u'2018-06-01',
    u'file_hash': u'f0d8d3c8d199d3790b0e42d1e5df50a6801f928d10f533149ed0babe61b5d7b5',
    u'file_size_bytes': 441388,
    u'file_description': u'Acoustic piano playing middle C.',
    u'audio_attributes': u'piano, middle C',
    u'file_size': u'431.0KiB',
    u'date_added': u'2018-06-01'
}

See also

Returns:(list) – A list of dicts containing metadata of the available audio files on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).
nussl.efz_utils.print_available_audio_files()

Prints a message to the console that shows all of the available audio files that are on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).

See also

Example

>>> import nussl
>>> nussl.efz_utils.print_available_audio_files()
File Name                                Duration (sec)  Size       Description
dev1_female3_inst_mix.wav                10.0            1.7MiB     Instantaneous mixture of three female speakers talking in a stereo field.
dev1_female3_synthconv_130ms_5cm_mix.wav 10.0            1.7MiB     Three female speakers talking in a stereo field, with 130ms of inter-channel delay.
K0140.wav                                5.0             431.0KiB   Acoustic piano playing middle C.
K0149.wav                                5.0             430.0KiB   Acoustic piano playing the A above middle C. (A440)

To download one of these files insert the file name as the first parameter to download_audio_file(), like so:

>>> nussl.efz_utils.download_audio_file('K0140.wav')
nussl.efz_utils.get_available_benchmark_files()

Returns a list of dicts containing metadata of the available benchmark files for tests on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).

Each entry in the list is in the following format:

{
    u'for_class': u'DuetUnitTests',
    u'visible': True, u'file_name':
    u'benchmark_atn_bins.npy',
    u'date_modified': u'2018-06-19',
    u'file_hash': u'cf7fef6f4ea9af3dbde8b9880602eeaf72507b6c78f04097c5e79d34404a8a1f',
    u'file_size_bytes': 488,
    u'file_description': u'Attenuation bins numpy array for DUET benchmark test.',
    u'file_size': u'488.0B',
    u'date_added': u'2018-06-19'
}

Notes

Most of the entries in the dictionary are self-explanatory, but note the for_class entry. The for_class entry specifies which nussl benchmark class will load the corresponding benchmark file. Make sure these match exactly when writing tests!

See also

Returns:(list) – A list of dicts containing metadata of the available audio files on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).
nussl.efz_utils.print_available_benchmark_files()

Prints a message to the console that shows all of the available benchmark files that are on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).

Example

>>> import nussl
>>> nussl.efz_utils.print_available_benchmark_files()
File Name                                For Class            Size       Description
mix3_matlab_repet_foreground.mat         TestRepet            6.4MiB     Foreground matrix for Repet class benchmark test.
benchmark_atn_bins.npy                   DuetUnitTests        488.0B     Attenuation bins numpy array for DUET benchmark test.
benchmark_sym_atn.npy                    DuetUnitTests        3.4MiB     Symmetric attenuation histogram for the DUET benchmark test.
benchmark_wmat.npy                       DuetUnitTests        3.4MiB     Frequency matrix for the DUET benchmark test.

To download one of these files insert the file name as the first parameter to nussl.download_benchmark_file, like so:

>>> nussl.efz_utils.download_benchmark_file('example.npy')

Notes

Most of the entries in the printed list are self-explanatory, but note the for_class entry. The for_class entry specifies which nussl benchmark class will load the corresponding benchmark file. Make sure these match exactly when writing tests!

See also

nussl.efz_utils.get_available_trained_models()

Returns a list of dicts containing metadata of the available trained models on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).

Each entry in the list is in the following format:

{
    u'for_class': u'DeepClustering',
    u'visible': True,
    u'file_name': u'deep_clustering_vocals_44k_long.model',
    u'date_modified': u'2018-06-01',
    u'file_hash': u'e09034c2cb43a293ece0b121f113b8e4e1c5a247331c71f40cb9ca38227ccc2c',
    u'file_size_bytes': 94543355,
    u'file_description': u'Deep clustering for vocal separation trained on augmented DSD100.',
    u'file_size': u'90.2MiB',
    u'date_added': u'2018-06-01'
}

Notes

Most of the entries in the dictionary are self-explanatory, but note the for_class entry. The for_class entry specifies which nussl separation class the given model will work with. Usually, nussl separation classes that require a model will default so retrieving a model on the EFZ server (if not already found on the user’s machine), but sometimes it is desirable to use a model other than the default one provided. In this case, the for_class entry lets the user know which class it is valid for use with. Additionally, trying to load a model into a class that it is not explicitly labeled for that class will raise an exception. Just don’t do it, ok?

See also

Returns:(list) – A list of dicts containing metadata of the available trained models on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).
nussl.efz_utils.print_available_trained_models()

Prints a message to the console that shows all of the available trained models that are on the nussl External File Zoo (EFZ) server (http://nussl.ci.northwestern.edu/).

Notes

Most of the entries in the dictionary are self-explanatory, but note the for_class entry. The for_class entry specifies which nussl separation class the given model will work with. Usually, nussl separation classes that require a model will default so retrieving a model on the EFZ server (if not already found on the user’s machine), but sometimes it is desirable to use a model other than the default one provided. In this case, the for_class entry lets the user know which class it is valid for use with. Additionally, trying to load a model into a class that it is not explicitly labeled for that class will raise an exception. Just don’t do it, ok?

See also

Example

>>> import nussl
>>> nussl.efz_utils.print_available_trained_models()
File Name                                For Class            Size       Description
deep_clustering_model.model              DeepClustering       48.1MiB    example Deep Clustering model
deep_clustering_vocal_44k_long.model     DeepClustering       90.2MiB    trained DC model for vocal extraction

To download one of these files insert the file name as the first parameter to download_trained_model(), like so:

>>> nussl.efz_utils.download_trained_model('deep_clustering_model.h5')
nussl.efz_utils.download_audio_file(audio_file_name, local_folder=None, verbose=True)

Downloads the specified audio file from the nussl External File Zoo (EFZ) server. The downloaded file is stored in :param:`local_folder` if a folder is provided. If a folder is not provided, nussl attempts to save the downloaded file in ~/.nussl/ (expanded) or in tmp/.nussl. If the requested file is already in :param:`local_folder` (or one of the two aforementioned directories) and the calculated hash matches the precomputed hash from the EFZ server metadata, then the file will not be downloaded.

Parameters:
  • audio_file_name – (str) Name of the audio file to attempt to download.
  • local_folder – (str) Path to local folder in which to download the file. If no folder is provided, nussl will store the file in ~/.nussl/ (expanded) or in tmp/.nussl.
  • verbose (bool) – If True prints the status of the download to the console.
Returns:

(String) Full path to the requested file (whether downloaded or not).

Example

>>> import nussl
>>> piano_path = nussl.efz_utils.download_audio_file('K0140.wav')
>>> piano_signal = nussl.AudioSignal(piano_path)
nussl.efz_utils.download_benchmark_file(benchmark_name, local_folder=None, verbose=True)

Downloads the specified benchmark file from the nussl External File Zoo (EFZ) server. The downloaded file is stored in :param:`local_folder` if a folder is provided. If a folder is not provided, nussl attempts to save the downloaded file in ~/.nussl/ (expanded) or in tmp/.nussl. If the requested file is already in :param:`local_folder` (or one of the two aforementioned directories) and the calculated hash matches the precomputed hash from the EFZ server metadata, then the file will not be downloaded.

Parameters:
  • audio_file_name – (str) Name of the trained model to attempt to download.
  • local_folder – (str) Path to local folder in which to download the file. If no folder is provided, nussl will store the file in ~/.nussl/ (expanded) or in tmp/.nussl.
  • verbose (bool) – If True prints the status of the download to the console.
Returns:

(String) Full path to the requested file (whether downloaded or not).

Example

>>> import nussl
>>> import numpy as np
>>> stm_atn_path = nussl.efz_utils.download_benchmark_file('benchmark_sym_atn.npy')
>>> sym_atm = np.load(stm_atn_path)
nussl.efz_utils.download_trained_model(model_name, local_folder=None, verbose=True)

Downloads the specified trained model from the nussl External File Zoo (EFZ) server. The downloaded file is stored in :param:`local_folder` if a folder is provided. If a folder is not provided, nussl attempts to save the downloaded file in ~/.nussl/ (expanded) or in tmp/.nussl. If the requested file is already in :param:`local_folder` (or one of the two aforementioned directories) and the calculated hash matches the precomputed hash from the EFZ server metadata, then the file will not be downloaded.

Parameters:
  • audio_file_name – (str) Name of the trained model to attempt to download.
  • local_folder – (str) Path to local folder in which to download the file. If no folder is provided, nussl will store the file in ~/.nussl/ (expanded) or in tmp/.nussl.
  • verbose (bool) – If True prints the status of the download to the console.
Returns:

(String) Full path to the requested file (whether downloaded or not).

Example

>>> import nussl
>>> model_path = nussl.efz_utils.download_trained_model('deep_clustering_model.h5')
>>> signal = nussl.AudioSignal()
>>> piano_signal = nussl.DeepClustering(signal, model_path=model_path)
exception nussl.efz_utils.FailedDownloadError

Exception class for failed file downloads.

exception nussl.efz_utils.MismatchedHashError

Exception class for when a computed hash function does match a pre-computed hash.

exception nussl.efz_utils.MetadataError

Exception class for errors with metadata.