- Source value
- 16.0778669871
- Metric
- diarization error rate (DER), corpus-aggregated
- Unit
- %
- Benchmark version
- raven.eval run 2026-07-31-callhome-de; published at results-2026-10-02, commit c3c21c94ec9bd127651643af047fbbe3b42528c6
- Category
- speaker-diarization
- Direction
- Lower is better
- Sample
- 120
- Lower confidence bound
- 14.97
- Upper confidence bound
- 17.27
- Source type
- vendor-reported
- Evaluator
- Raven raven.eval
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. German telephone diarization on talkbank/callhome config deu (CALLHOME-de), 120 files. Exact model pyannote/speaker-diarization-community-1 at revision 3533c8cf8e369892e6b79ff1bf80f7b0286a54ee; dataset revision 17c8a153215aa7c50b805078fd6284ba81c2fc47. Collar 0.25 s, skip_overlap=false, corpus aggregation, automatic VAD, Hungarian speaker mapping, primary scorer pyannote.metrics. The report publishes a 95% percentile-bootstrap CI over files, 10,000 resamples, seed 20260903. This is the NIST/CALLHOME classic metric variant, distinct from the 0.0 s collar row.
- Context
- Raven sells meeting transcription and identifies this pyannote model as built into its product. Treat this as a vendor-reported, reproducible public-data result, not an independent evaluation.