diff --git a/fern/pages/concepts/pre-recorded-evals.mdx b/fern/pages/concepts/pre-recorded-evals.mdx index 2f0c0a69..fdeabba5 100644 --- a/fern/pages/concepts/pre-recorded-evals.mdx +++ b/fern/pages/concepts/pre-recorded-evals.mdx @@ -221,6 +221,16 @@ Qualitative analysis is also useful for tie-breaking when benchmarking metrics d **A/B testing in production**: Serve transcripts from different providers to users and collect feedback. You can ask users to score transcripts directly, or track indirect signals like the number of support ticket complaints about transcription quality. +### Domain-specific evaluation considerations + +WER is not always the right primary metric. Some domains prioritize output qualities that traditional accuracy metrics do not capture: + +- **Medical scribes**: Customers often evaluate based on **user preference rate** and **readability** — whether clinicians prefer the transcript output for generating clinical notes. Formatting quality, medical terminology accuracy, and structured output can matter more than raw WER. See the [Medical Scribe Best Practices](/docs/use-cases/medical-scribe-best-practices) guide for domain-specific evaluation guidance. +- **Legal transcription**: Verbatim accuracy including disfluencies and speaker attribution may be more important than clean, readable output. +- **Media and entertainment**: Proper noun accuracy for names, places, and brands can outweigh overall WER. + +When running evaluations for domain-specific use cases, define your success criteria before choosing metrics. If your end users care about readability and preference, include qualitative evaluation (side-by-side comparisons, user preference scoring) alongside quantitative metrics. + ## Iterating on prompts Finding the optimal prompt for your use case is an iterative process. There are two main approaches: diff --git a/fern/pages/getting-started/models.mdx b/fern/pages/getting-started/models.mdx index 24c03056..86a774e7 100644 --- a/fern/pages/getting-started/models.mdx +++ b/fern/pages/getting-started/models.mdx @@ -120,6 +120,10 @@ Universal-3 Pro is our most advanced transcription model, delivering state-of-th Universal-3 Pro also supports regional dialects and local speech variants out of the box — no special configuration needed. See the full list of [supported dialects](/docs/pre-recorded-audio/supported-languages#regional-dialects-and-variants). + +**Building a medical scribe or clinical documentation app?** Universal-3 Pro delivers exceptional accuracy for medical terminology and clinical documentation. See the [Medical Scribe Best Practices](/docs/use-cases/medical-scribe-best-practices) guide for optimized prompting, speaker diarization configuration, and HIPAA-compliant setup. + + [Try Universal-3 Pro here](/docs/pre-recorded-audio/universal-3-pro) #### Universal-2 diff --git a/fern/pages/getting-started/transcribe-an-audio-file2.mdx b/fern/pages/getting-started/transcribe-an-audio-file2.mdx index be36416e..b1e5dc73 100644 --- a/fern/pages/getting-started/transcribe-an-audio-file2.mdx +++ b/fern/pages/getting-started/transcribe-an-audio-file2.mdx @@ -16,6 +16,10 @@ description: Quickstart guide for transcribing a pre-recorded audio file. This guide walks you through transcribing your first audio file with AssemblyAI. You will learn how to submit an audio file for transcription and retrieve the results using the AssemblyAI API. + +**Building a medical scribe or clinical documentation app?** Check out the [Medical Scribe Best Practices](/docs/use-cases/medical-scribe-best-practices) guide for optimized model selection, medical terminology prompting, and HIPAA-compliant configuration. + + When transcribing an audio file, there are three main things you will want to specify: 1. The speech models you would like to use (required).