pyannote/speaker-diarization-community-1 · Hugging Face

Source: https://huggingface.co/pyannote/speaker-diarization-community-1 This pipeline ingests mono audio sampled at 16kHz and outputs speaker diarization.

• stereo or multi-channel audio files are automatically downmixed to mono by averaging the channels.
• audio files sampled at a different rate are resampled to 16kHz automatically upon loading.

Themain improvements brought byCommunity\-1are:

• improvedspeaker assignment and counting
• simpler reconciliation with transcription timestamps withexclusivespeaker diarization
• easyoffline use(i.e. without internet connection)
• (optionally)hostedon pyannoteAI cloud

https://huggingface.co/pyannote/speaker-diarization-community-1#setupSetup

1. pip install pyannote\.audio
2. Accept user conditions
3. Create access token athf\.co/settings/tokens.

https://huggingface.co/pyannote/speaker-diarization-community-1#quick-startQuick start

# download the pipeline from Huggingface
from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-community-1", 
    token="{huggingface-token}")

# run the pipeline locally on your computer
output = pipeline("audio.wav")

# print the predicted speaker diarization 
for turn, speaker in output.speaker_diarization:
    print(f"{speaker} speaks between t={turn.start:.3f}s and t={turn.end:.3f}s")

https://huggingface.co/pyannote/speaker-diarization-community-1#benchmarkBenchmark

Out of the box,Community\-1is much better thanspeaker\-diarization\-3\.1.

We reportdiarization error rates(in %) on large collection of academic benchmarks (fully automatic processing, no forgiveness collar, nor skipping overlapping speech).

Precision\-2model is even better and can be tested like this:

1. Create an API key onpyannoteAI dashboard(free credits included)
2. Change one line of code

from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained(
-     'pyannote/speaker-diarization-community-1', token="{huggingface-token}")
+     'pyannote/speaker-diarization-precision-2', token="{pyannoteAI-api-key}")
diarization = pipeline("audio.wav")  # runs on pyannoteAI servers

https://huggingface.co/pyannote/speaker-diarization-community-1#processing-on-gpuProcessing on GPU

pyannote\.audiopipelines run on CPU by default. You can send them to GPU with the following lines:

import torch
pipeline.to(torch.device("cuda"))

https://huggingface.co/pyannote/speaker-diarization-community-1#processing-from-memoryProcessing from memory

Pre-loading audio files in memory may result in faster processing:

waveform, sample_rate = torchaudio.load("audio.wav")
output = pipeline({"waveform": waveform, "sample_rate": sample_rate})

https://huggingface.co/pyannote/speaker-diarization-community-1#monitoring-progressMonitoring progress

Hooks are available to monitor the progress of the pipeline:

from pyannote.audio.pipelines.utils.hook import ProgressHook
with ProgressHook() as hook:
    output = pipeline("audio.wav", hook=hook)

https://huggingface.co/pyannote/speaker-diarization-community-1#controlling-the-number-of-speakersControlling the number of speakers

In case the number of speakers is known in advance, one can use thenum\_speakersoption:

output = pipeline("audio.wav", num_speakers=2)

One can also provide lower and/or upper bounds on the number of speakers usingmin\_speakersandmax\_speakersoptions:

output = pipeline("audio.wav", min_speakers=2, max_speakers=5)

https://huggingface.co/pyannote/speaker-diarization-community-1#exclusive-speaker-diarizationExclusive speaker diarization

Community\-1pretrained pipeline returns a newexclusivespeaker diarization, on top of the regular speaker diarization, available asoutput\.exclusive\_speaker\_diarization.

This is a feature which isbackported from our latest commercial modelthat simplifies the reconciliation between fine-grained speaker diarization timestamps and (sometimes not so precise) transcription timestamps.

https://huggingface.co/pyannote/speaker-diarization-community-1#offline-useOffline use

1. In the terminal, copy the pipeline on disk:

# make sure git-lfs is installed (https://git-lfs.com)
git lfs install

# create a directory on disk
mkdir /path/to/directory

# when prompted for a password, use an access token with write permissions.
# generate one from your settings: https://huggingface.co/settings/tokens
git clone https://hf.co/pyannote/speaker-diarization-community-1 /path/to/directory/pyannote-speaker-diarization-community-1

1. In Python, use the pipeline without internet connection:

# load pipeline from disk (works without internet connection)
from pyannote.audio import Pipeline
pipeline = Pipeline.from_pretrained('/path/to/directory/pyannote-speaker-diarization-community-1')

# run the pipeline locally on your computer
output = pipeline("audio.wav")

https://huggingface.co/pyannote/speaker-diarization-community-1#citationsCitations

1. Speaker segmentation model

@inproceedings{Plaquet23,
  author={Alexis Plaquet and Hervé Bredin},
  title={{Powerset multi-class cross entropy loss for neural speaker diarization}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}

1. Speaker embedding model

@inproceedings{Wang2023,
  title={Wespeaker: A research and production oriented speaker embedding learning toolkit},
  author={Wang, Hongji and Liang, Chengdong and Wang, Shuai and Chen, Zhengyang and Zhang, Binbin and Xiang, Xu and Deng, Yanlei and Qian, Yanmin},
  booktitle={ICASSP 2023, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={1--5},
  year={2023},
  organization={IEEE}
}

1. Speaker clustering

@article{Landini2022,
  author={Landini, Federico and Profant, J{\'a}n and Diez, Mireia and Burget, Luk{\'a}{\v{s}}},
  title={{Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks}},
  year={2022},
  journal={Computer Speech \& Language},
}

https://huggingface.co/pyannote/speaker-diarization-community-1#acknowledgmentAcknowledgment

Training and tuning made possible thanks toGENCIon theJean Zaysupercomputer.