From 4bddc67e1a781627ccb28aa3b89329afdeced16e Mon Sep 17 00:00:00 2001
From: Nils Briggen <nilsbriggen@gmail.com>
Date: Mon, 15 Jun 2026 10:16:42 +0200
Subject: [PATCH] Web transcription: load an fp32 decoder on WASM (fixes
 MatMulNBits)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The WASM backend couldn't create a session — "TransposeDQWeightsFor
MatMulNBits Missing required scale" — so transcription failed to start on
mobile (which we route to WASM) and on any no-WebGPU device. Root cause:
the default quantized Whisper decoder (q8/q4, decoder_model_merged) uses
MatMulNBits ops that the onnxruntime-web bundled with transformers.js
4.2.0 cannot load on WASM. Reproduced and bisected in a browser harness:
across both Xenova and onnx-community repos, q8/q4/string-fp32 all fail on
WASM with this error, while an EXPLICIT per-file fp32 decoder
({ encoder_model: 'fp32', decoder_model_merged: 'fp32' }) loads and runs
(2/2 chunks, no error) on a no-GPU machine.

So WASM now requests that explicit fp32 decoder; WebGPU keeps fp16. Larger
download on the WASM path, but it actually loads and runs — q8 never did
on WASM. (4.2.0 is the latest transformers.js, so bumping isn't an option.)

tsc clean, 279 tests pass, web export OK.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---
 src/lib/transcription/engineImpl.web.ts | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/src/lib/transcription/engineImpl.web.ts b/src/lib/transcription/engineImpl.web.ts
index 0099af6..7beea30 100644
--- a/src/lib/transcription/engineImpl.web.ts
+++ b/src/lib/transcription/engineImpl.web.ts
@@ -40,7 +40,9 @@ interface AsrOutput {
 type AsrPipeline = (audio: Float32Array, opts: Record<string, unknown>) => Promise<AsrOutput>;
 interface PipelineOptions {
   device?: string;
-  dtype?: string;
+  // A single dtype for all sub-models, or a per-file map (encoder_model,
+  // decoder_model_merged, …) — Whisper's decoder needs special handling on WASM.
+  dtype?: string | Record<string, string>;
   progress_callback?: (e: { status?: string; progress?: number }) => void;
 }
 interface TransformersModule {
@@ -111,10 +113,17 @@ export const engine: TranscriptionEngine = {
     const { pipeline } = await lib();
     const webgpu = (await resolveBackend()) === 'webgpu';
     const asr = await pipeline('automatic-speech-recognition', MODELS[modelId].webRepo, {
-      // WebGPU + fp16 when available; otherwise 8-bit weights on WASM, which
-      // stays small to download and runs acceptably on a plain CPU.
       device: webgpu ? 'webgpu' : 'wasm',
-      dtype: webgpu ? 'fp16' : 'q8',
+      // WebGPU: fp16 (fast, small). WASM: an explicit fp32 decoder. The default
+      // quantized decoders (q8/q4) use `MatMulNBits` ops that onnxruntime-web
+      // (transformers.js 4.2.0) FAILS to load on the WASM backend with
+      // "Missing required scale" — verified across Xenova + onnx-community
+      // repos. fp32 avoids those ops and actually runs on any CPU (the WASM
+      // path is what mobile and no-WebGPU devices use). Larger download, but it
+      // works; q8 simply does not load on WASM here.
+      dtype: webgpu
+        ? 'fp16'
+        : { encoder_model: 'fp32', decoder_model_merged: 'fp32' },
       progress_callback: (e) => {
         if (e.status === 'progress' && e.progress != null) onProgress?.(e.progress / 100);
       },