Comparing and evaluating alignments (mfa compare_alignments)#

As of MFA 3.4, you can compare arbitrary sets of alignments for the same files. The “reference” directory can be hand corrected or manually segmented files from a benchmark corpus such as TIMIT, Buckeye, Seoul Corpus, or the Corpus of Spontaneous Japanese, which can then be interpreted as boundary errors or boundary precision for benchmarking purposes. The alignments do not have to have been generated by MFA for this command (unlike when using the --reference_directory flag for Align with an acoustic model (mfa align)). If the two sets to be compared are both automatically generated, then any boundary errors should be interpreted more as agreement in boundary placement than as a measure of accuracy or evaluation.

If the reference directory specified has audio files or the --audio_directory parameter is specified, then alignment analysis metrics related to the audio file can be calculated, such as Signal-to-Noise Ratio (SNR) and Intensity deviation of phone intervals. Without this, other alignment analysis metrics that just rely on timing information such as the Phone duration deviation will still be calculated. However, since the input is just the alignment outputs and not anything involving an MFA model, metrics involving Alignment log-likelihood from a model cannot be calculated.

See also

Command reference#

mfa compare_alignments#

Align a corpus with a pronunciation dictionary and a pretrained acoustic model.

Usage

mfa compare_alignments [OPTIONS] REFERENCE_DIRECTORY TEST_DIRECTORY
                       OUTPUT_DIRECTORY

Options

--custom_mapping_path <custom_mapping_path>#

YAML file for mapping phones from acoustic model phone set to phone set in golden alignments.

-c, --config_path <config_path>#

Path to config file to use for aligning.

-s, --speaker_characters <speaker_characters>#

Number of characters of file names to use for determining speaker, default is to use directory names.

-a, --audio_directory <audio_directory>#

Audio directory root to use for finding audio files.

--strict_mapping#

Flag for ensuring all phones are in the given mapping file.

--naive#

Flag for skipping interval alignment and using closest boundary for error calculation.

-p, --profile <profile>#

Configuration profile to use, defaults to “global”

-t, --temporary_directory <temporary_directory>#

Set the default temporary directory, default is /home/docs/Documents/MFA

-j, --num_jobs <num_jobs>#

Set the number of processes to use by default, defaults to 3

--clean, --no_clean#

Remove files from previous runs, default is True

--final_clean, --no_final_clean#

Remove temporary files at the end of run, default is False

-v, --verbose, -nv, --no_verbose#

Output debug messages, default is False

-q, --quiet, -nq, --no_quiet#

Suppress all output messages (overrides verbose), default is False

--overwrite, --no_overwrite#

Overwrite output files when they exist, default is False

--use_mp, --no_use_mp#

Turn on/off multiprocessing. Multiprocessing is recommended will allow for faster executions.

--use_threading, --no_use_threading#

Use threading library rather than multiprocessing library. Multiprocessing is recommended will allow for faster executions.

-d, --debug, -nd, --no_debug#

Run extra steps for debugging issues, default is False

--use_postgres, --no_use_postgres#

Use postgres instead of sqlite for extra functionality, default is False

--single_speaker#

Single speaker mode creates multiprocessing splits based on utterances rather than speakers. This mode also disables speaker adaptation equivalent to --uses_speaker_adaptation false.

--textgrid_cleanup, --cleanup_textgrids, --no_textgrid_cleanup, --no_cleanup_textgrids#

Turn on/off post-processing of TextGrids that cleans up silences and recombines compound words and clitics.

-h, --help#

Show this message and exit.

Arguments

REFERENCE_DIRECTORY#

Required argument

TEST_DIRECTORY#

Required argument

OUTPUT_DIRECTORY#

Required argument