Web transcription: load an fp32 decoder on WASM (fixes MatMulNBits)
CI / test (push) Successful in 18s
CI / deploy-web (push) Successful in 35s
CI / build-apk (push) Successful in 6m46s

The WASM backend couldn't create a session — "TransposeDQWeightsFor
MatMulNBits Missing required scale" — so transcription failed to start on
mobile (which we route to WASM) and on any no-WebGPU device. Root cause:
the default quantized Whisper decoder (q8/q4, decoder_model_merged) uses
MatMulNBits ops that the onnxruntime-web bundled with transformers.js
4.2.0 cannot load on WASM. Reproduced and bisected in a browser harness:
across both Xenova and onnx-community repos, q8/q4/string-fp32 all fail on
WASM with this error, while an EXPLICIT per-file fp32 decoder
({ encoder_model: 'fp32', decoder_model_merged: 'fp32' }) loads and runs
(2/2 chunks, no error) on a no-GPU machine.

So WASM now requests that explicit fp32 decoder; WebGPU keeps fp16. Larger
download on the WASM path, but it actually loads and runs — q8 never did
on WASM. (4.2.0 is the latest transformers.js, so bumping isn't an option.)

tsc clean, 279 tests pass, web export OK.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 10:16:42 +02:00
parent fa12817050
commit 4bddc67e1a
+13 -4
View File
@@ -40,7 +40,9 @@ interface AsrOutput {
type AsrPipeline = (audio: Float32Array, opts: Record<string, unknown>) => Promise<AsrOutput>;
interface PipelineOptions {
device?: string;
dtype?: string;
// A single dtype for all sub-models, or a per-file map (encoder_model,
// decoder_model_merged, …) — Whisper's decoder needs special handling on WASM.
dtype?: string | Record<string, string>;
progress_callback?: (e: { status?: string; progress?: number }) => void;
}
interface TransformersModule {
@@ -111,10 +113,17 @@ export const engine: TranscriptionEngine = {
const { pipeline } = await lib();
const webgpu = (await resolveBackend()) === 'webgpu';
const asr = await pipeline('automatic-speech-recognition', MODELS[modelId].webRepo, {
// WebGPU + fp16 when available; otherwise 8-bit weights on WASM, which
// stays small to download and runs acceptably on a plain CPU.
device: webgpu ? 'webgpu' : 'wasm',
dtype: webgpu ? 'fp16' : 'q8',
// WebGPU: fp16 (fast, small). WASM: an explicit fp32 decoder. The default
// quantized decoders (q8/q4) use `MatMulNBits` ops that onnxruntime-web
// (transformers.js 4.2.0) FAILS to load on the WASM backend with
// "Missing required scale" — verified across Xenova + onnx-community
// repos. fp32 avoids those ops and actually runs on any CPU (the WASM
// path is what mobile and no-WebGPU devices use). Larger download, but it
// works; q8 simply does not load on WASM here.
dtype: webgpu
? 'fp16'
: { encoder_model: 'fp32', decoder_model_merged: 'fp32' },
progress_callback: (e) => {
if (e.status === 'progress' && e.progress != null) onProgress?.(e.progress / 100);
},