Two native-only fixes behind the "transcription failed [object Object]"
report on the APK:
1. Model download. engineImpl.native.loadModel assumed the ggml .bin was
already on disk (the downloader was a TODO), so on a real device it
rejected — and since in-app recording is web-only, no model was ever
present. Now loadModel fetches the model from Hugging Face
(ggerganov/whisper.cpp) into <documentDirectory>/models on first use,
reporting 0..1 progress through the existing "Loading model… X%" UI.
Downloads to a .part file and renames on success so an interrupted
download can't leave a truncated model; if initWhisper still rejects,
the file is deleted so the next try re-downloads cleanly. Model URLs
verified (tiny.en 77.7MB / base.en 148MB / small.en 488MB).
2. Error surfacing. transcribeStore did `String(err)`, which renders a
non-Error native rejection (whisper.rn / file system throw plain
objects) as "[object Object]". New shared errorMessage() pulls
message/reason/code (or JSON) out of whatever was thrown, so failures
are actionable instead of opaque.
Progress downloads aren't in the new expo-file-system OO API yet, so the
fetch uses the still-supported expo-file-system/legacy resumable
downloader; native-only, not bundled on web.
Validated: tsc clean, 279 tests pass, web export unaffected (legacy not
in the web bundle), arm64 APK builds and the native JS bundle resolves
the legacy import. On-device download + transcribe still needs a real
phone to confirm end to end.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On native, importing anything but WAV failed with "Only WAV is supported
on native" — ffmpeg-kit-react-native was retired in 2025 and a real
decoder was left as a follow-up. Since in-app recording is web-only, the
native app's only way to add audio is importing a phone recording — which
is m4a/aac — so transcription was effectively broken on the APK.
Add react-native-audio-api (Software Mansion, maintained) and route
non-WAV files through its standalone decodeAudioData(ArrayBuffer), which
uses platform codecs + bundled FFmpeg. WAV keeps its pure-JS fast path.
The decoded channels go through the existing toMono16k, mirroring the web
AudioContext.decodeAudioData path exactly. Bytes are read locally and
passed as an ArrayBuffer, so it doesn't matter whether the picker yields
file:// or content://. WAV is now sniffed by RIFF/WAVE content, not just
the extension.
The lib's config plugin is intentionally NOT enabled: it only sets up
playback (background audio mode, a media-playback foreground service, mic
permission), none of which decoding needs — and adding those would
contradict the app's on-device/no-extra-permissions stance.
Validated locally (arm64 Docker build): react-native-audio-api compiles
and autolinks against RN 0.85 / new arch; the arm64-v8a APK ships
libreact-native-audio-api.so + libav{codec,format,util}/libswresample.so
+ liboboe.so (48MB -> 58MB). tsc clean, 279 tests pass, web export does
NOT bundle the native dep. Runtime decode still needs an on-device check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Android CI job recompiled all native C++ (whisper.cpp, reanimated,
worklets, gesture-handler) from scratch every push — ~41 min on the
shared runner — because nothing survived the `COPY . .` layer
invalidation, and `--load` exported the whole ~15GB image just to copy
one APK out.
Dockerfile (docker/android.Dockerfile):
- Install ccache and patch the NDK CMake toolchain file so every module's
externalNativeBuild routes C/C++ compiles + links through ccache (the
third-party modules don't honor ccache on their own). ccache dir is a
persistent BuildKit cache mount, so objects survive across builds.
- Mount GRADLE_USER_HOME as a cache and pass --build-cache --parallel, so
Kotlin/Java/resource/dex tasks and resolved deps persist too.
- Cache-mount the bun install dir; move the nodejs install into the cached
toolchain layer (was reinstalling on every source change).
- New `apk` stage (FROM scratch) holding just the signed APK, for
--output type=local extraction without loading the 15GB image.
CI (.gitea/workflows/ci.yml):
- build-apk now builds `--target apk --output type=local,dest=./out` and
streams ./out/app-release.apk to /srv/wisp/wisp.apk — no --load, no
docker create/cp, and the build image no longer piles up on the host.
Validated locally (arm64, real sign+assemble path):
cold gradle 5m5s -> warm gradle 57s; 61/61 C++ compiles served from
ccache (100% hit); 226/578 Gradle tasks from cache; export-only build
60s wall, APK valid + v2/v3 signed. The first server build after this
is still cold (~41 min) to populate the caches; pushes after that are
much faster.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Library crammed five text nav-links into a non-wrapping row with no
bottom navigation — unusable on phones. Restructure the primary screens
into a bottom tab bar:
- Add src/app/(tabs)/_layout.tsx: <Tabs> with 5 tabs (Library, Search,
Study, Ask, Settings) with emoji icons and the accent active tint.
- Move index/search/study/ask/settings into the (tabs) route group; the
parens keep URLs unchanged (/, /search, /study, /ask, /settings).
- Root _layout becomes a Stack hosting (tabs) (headerless) plus the
secondary pushed screens (record, transcript/[id], courses, quiz,
bibliography), each with a back-button header.
- Drop the per-screen <Stack.Screen> headers from the moved tabs (titles
now come from Tabs); relocate Study's "Export Anki" action into the body.
- Library: remove the cramped header link-row and duplicate title; add a
"nothing uploaded" subheader and a Courses link.
Verified at 375px: tab bar pinned to the bottom, 5 even tabs, navigation
between tabs works. tsc clean, web export OK, 279 tests pass.
CI: build-apk now runs automatically on every push to master (after
deploy-web), so the APK at /wisp.apk tracks master instead of being
frozen at a stale tag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Strictly opt-in, gated, with the deterministic features as the always-present floor:
- GenerationEngine: WebLLM (Qwen2.5-1.5B, WebGPU, CDN-loaded) + BYO-key cloud
(OpenAI-compatible); native stub. Pure grounding prompt builder (4 tests).
- rag.askLectures: retrieve Phase-1 hits -> grounded prompt -> answer with
citations; refuses when nothing relevant; falls back to search-only when no
engine is available. Never sends raw audio/transcripts — only question + snippets.
- aiStore (BYO key persisted in localStorage on web), Ask screen (answer +
tappable citation chips that jump to the audio + honest "verify" disclaimer),
Settings AI section (engine status + bring-your-own-key form).
279 tests green, 0 tsc errors, web export builds. ROADMAP phases 0-5 complete.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- useRecorder: MediaRecorder mic capture with elapsed timer, live input-level
meter (AnalyserNode), pause/resume, and a screen Wake Lock; returns the
recording as an ArrayBuffer that flows into the existing transcribe pipeline.
- Record screen: big timer + level meter + start/pause/resume/stop; on stop the
pre-capture sheet collects title/course/date, then it transcribes + saves +
indexes (reusing Phase 0/1). Home gets Record + Import buttons.
- Web-only for now; native realtime (whisper.rn), OPFS crash-recovery,
tab-audio capture, and the per-course dashboard are follow-ups.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- android.Dockerfile: build arm64-v8a only (--max-workers=2) → ~48MB APK, ~4x
less compile (the 4-ABI build timed out on the shared prod runner).
- ci.yml: build-apk runs only on version tags (v*), timeout 120m — normal pushes
no longer peg the box; web still auto-deploys on every master push.
- wisp.compose.yml: mount /srv/wisp/wisp.apk read-only so the download survives
web container rebuilds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- docker/android.Dockerfile: install nodejs (image ships Bun; ci-android-sign.sh
patches build.gradle with a Node script).
- buffer polyfill: whisper.rn -> safe-buffer requires Node's `buffer`, absent in
Metro/RN. Add `buffer` dep, alias it in metro.config.js, set the global in
_layout. Web export unaffected (verified).
Validated locally end-to-end: assembleRelease BUILD SUCCESSFUL, 117MB APK across
all 4 ABIs with librnwhisper.so, signed (CN=Wisp, O=briggen.dev).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- .gitea/workflows/ci.yml rewritten for a self-hosted runner co-located with
the host Docker daemon + Traefik: push to master -> test -> deploy-web
(docker compose up --build in place; no SSH, no registry). APK job documented
as the next iteration (socket-safe build+extract; signing secrets already set).
- .gitignore: never commit the release keystore (*.keystore + wisp-release.keystore).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NilsBriggen