Web transcription: load an fp32 decoder on WASM (fixes MatMulNBits)
The WASM backend couldn't create a session — "TransposeDQWeightsFor
MatMulNBits Missing required scale" — so transcription failed to start on
mobile (which we route to WASM) and on any no-WebGPU device. Root cause:
the default quantized Whisper decoder (q8/q4, decoder_model_merged) uses
MatMulNBits ops that the onnxruntime-web bundled with transformers.js
4.2.0 cannot load on WASM. Reproduced and bisected in a browser harness:
across both Xenova and onnx-community repos, q8/q4/string-fp32 all fail on
WASM with this error, while an EXPLICIT per-file fp32 decoder
({ encoder_model: 'fp32', decoder_model_merged: 'fp32' }) loads and runs
(2/2 chunks, no error) on a no-GPU machine.
So WASM now requests that explicit fp32 decoder; WebGPU keeps fp16. Larger
download on the WASM path, but it actually loads and runs — q8 never did
on WASM. (4.2.0 is the latest transformers.js, so bumping isn't an option.)
tsc clean, 279 tests pass, web export OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -40,7 +40,9 @@ interface AsrOutput {
|
|||||||
type AsrPipeline = (audio: Float32Array, opts: Record<string, unknown>) => Promise<AsrOutput>;
|
type AsrPipeline = (audio: Float32Array, opts: Record<string, unknown>) => Promise<AsrOutput>;
|
||||||
interface PipelineOptions {
|
interface PipelineOptions {
|
||||||
device?: string;
|
device?: string;
|
||||||
dtype?: string;
|
// A single dtype for all sub-models, or a per-file map (encoder_model,
|
||||||
|
// decoder_model_merged, …) — Whisper's decoder needs special handling on WASM.
|
||||||
|
dtype?: string | Record<string, string>;
|
||||||
progress_callback?: (e: { status?: string; progress?: number }) => void;
|
progress_callback?: (e: { status?: string; progress?: number }) => void;
|
||||||
}
|
}
|
||||||
interface TransformersModule {
|
interface TransformersModule {
|
||||||
@@ -111,10 +113,17 @@ export const engine: TranscriptionEngine = {
|
|||||||
const { pipeline } = await lib();
|
const { pipeline } = await lib();
|
||||||
const webgpu = (await resolveBackend()) === 'webgpu';
|
const webgpu = (await resolveBackend()) === 'webgpu';
|
||||||
const asr = await pipeline('automatic-speech-recognition', MODELS[modelId].webRepo, {
|
const asr = await pipeline('automatic-speech-recognition', MODELS[modelId].webRepo, {
|
||||||
// WebGPU + fp16 when available; otherwise 8-bit weights on WASM, which
|
|
||||||
// stays small to download and runs acceptably on a plain CPU.
|
|
||||||
device: webgpu ? 'webgpu' : 'wasm',
|
device: webgpu ? 'webgpu' : 'wasm',
|
||||||
dtype: webgpu ? 'fp16' : 'q8',
|
// WebGPU: fp16 (fast, small). WASM: an explicit fp32 decoder. The default
|
||||||
|
// quantized decoders (q8/q4) use `MatMulNBits` ops that onnxruntime-web
|
||||||
|
// (transformers.js 4.2.0) FAILS to load on the WASM backend with
|
||||||
|
// "Missing required scale" — verified across Xenova + onnx-community
|
||||||
|
// repos. fp32 avoids those ops and actually runs on any CPU (the WASM
|
||||||
|
// path is what mobile and no-WebGPU devices use). Larger download, but it
|
||||||
|
// works; q8 simply does not load on WASM here.
|
||||||
|
dtype: webgpu
|
||||||
|
? 'fp16'
|
||||||
|
: { encoder_model: 'fp32', decoder_model_merged: 'fp32' },
|
||||||
progress_callback: (e) => {
|
progress_callback: (e) => {
|
||||||
if (e.status === 'progress' && e.progress != null) onProgress?.(e.progress / 100);
|
if (e.status === 'progress' && e.progress != null) onProgress?.(e.progress / 100);
|
||||||
},
|
},
|
||||||
|
|||||||
Reference in New Issue
Block a user