Skip to main content

@remotion/whisper-web

warning

Unstable API: This package is experimental for the moment. As we test it, we might make a few changes to the API and switch to a WebGPU-based backend in the future.

Similar to @remotion/install-whisper-cpp but for the browser. Allows you to transcribe audio locally in the browser, with the help of WASM.

npm i --save-exact @remotion/whisper-web@4.0.306
This assumes you are currently using v4.0.306 of Remotion.
Also update remotion and all `@remotion/*` packages to the same version.
Remove all ^ character in front of the version numbers of it as it can lead to a version conflict.

Requirements​

This package requires a page that is cross-origin isolated.

Example usage​

Transcribing with @remotion/whisper-web
tsx
import {transcribe, canUseWhisperWeb, resampleTo16Khz, downloadWhisperModel} from '@remotion/whisper-web';
 
const file = new File([], 'audio.wav');
const modelToUse = 'tiny.en';
 
const {supported, detailedReason} = await canUseWhisperWeb(modelToUse);
if (!supported) {
  throw new Error(`Whisper Web is not supported in this environment: ${detailedReason}`);
}
 
console.log('Downloading model...');
await downloadWhisperModel({
  model: modelToUse,
  onProgress: ({progress}) => console.log(`Downloading model (${Math.round(progress * 100)}%)...`),
});
 
console.log('Resampling audio...');
const channelWaveform = await resampleTo16Khz({
  file,
  onProgress: (p) => console.log(`Resampling audio (${Math.round(p * 100)}%)...`),
});
 
console.log('Transcribing...');
const {transcription} = await transcribe({
  channelWaveform,
  model: modelToUse,
  onProgress: (p) => console.log(`Transcribing (${Math.round(p * 100)}%)...`),
});
 
console.log(transcription.map((t) => t.text).join(' '));

Functions​

License​

MIT