wisp/docs/ROADMAP.md

Wisp — Study-System Roadmap

Plan to evolve Wisp from a transcription studio into a private, on-device lecture/study knowledge system. Generated from a 6-lens feature pass (39 features) verified against the live codebase.

North star

"Wisp becomes a private, on-device study system that turns a whole semester of recorded lectures into a searchable, course-organized second brain — without a single byte of your audio or notes ever leaving your device. The single killer workflow is exam-time semantic recall: you sit down the week before an exam, scope to one course, type a question in your own words ('what were the conditions for a Nash equilibrium?'), and Wisp returns the exact lecture moments — definition first, then every later mention across the term — each as a verbatim segment with a one-tap jump that replays the 20 seconds of the professor actually saying it. Everything else (live lecture capture into courses, summaries, flashcards, auto-detected exam dates, related-reading lookup) exists to fill and enrich that searchable corpus and to point you back to the source. Honesty is the product: every derived artifact is labelled, grounded in a timestamp, and verifiable against the professor's real words; cloud and shadow-library options are opt-in, clearly-labelled extras, never the default, never required."

What is missing vs today

"Grouped against today's MVP (one flat Transcript entity, substring search, throwaway audio, import-only): (1) ORGANIZATION — there is no Course concept at all, no second table on either backend, no schedule/instructor/lectureDate/tags/studied fields, and the home screen is one undifferentiated flat list; 'everything sorted by courses' has nowhere to live. (2) DATA-MODEL PLUMBING — neither backend has ANY migration history (Dexie frozen at version(1), native uses bare CREATE TABLE IF NOT EXISTS with no user_version), segments are keyed by array index with no stable id, and source media is a throwaway in-session object URL revoked on the next job ('re-import to scrub' is shipped today) — so jump-to-audio, citations, and vectors have no durable anchor. (3) RETRIEVAL — search is naive case-insensitive substring over a denormalized searchText column returning whole-transcript metas with no score, no per-segment hit, and no timestamp; there are zero embeddings, no vector store, no hybrid ranking — the literal exam-time use case is unbuildable as-is. (4) CAPTURE — there is no recording code whatsoever (no getUserMedia, no expo-audio, no mic permission, no live driver; the web engine self-reports supportsLiveMic:false), so the corpus can only be filled by importing files recorded elsewhere. (5) LEARNING/AI — no embedding engine, no LLM runtime, no summary/glossary/flashcard/SRS/quiz code anywhere. (6) ENRICHMENT/CALENDAR — no date or citation parsing, no .ics/BibTeX export, and the app is 100% offline with no network client for legitimate metadata lookups. What Wisp HAS that makes this tractable: a clean pure pipeline against a swappable TranscriptionEngine interface, a Zod-gatekept StorageRepo, the proven CDN new Function loader with WebGPU/WASM detection that the SAME pipeline() call reuses for 'feature-extraction' embeddings, an already-cross-origin-isolated web app (COOP/COEP, SharedArrayBuffer live — the hardest WebGPU/threaded-WASM prerequisite is already met), a pure jobs reducer, and an extensible export registry."

Roadmap

Phase 0 — Foundation: versioned migrations, courses, capture metadata, persisted media + segment IDs

Goal: Make the data model capable of everything that follows, without losing a single existing transcript, and stop throwing away the two things every later feature needs: the source audio and stable segment identity.

Features:

• Real versioned migration runner on BOTH backends (Dexie .upgrade() chain + native PRAGMA user_version step-runner), idempotent and transaction-wrapped, gated by a seeded-v1 round-trip 'no data loss' test
• Course entity + courseId (nullable, defaults existing rows to 'Unsorted') + lectureDate/instructor/location/tags/lectureNumber on Transcript, all Zod-gated; new repo methods (listCourses/createCourse/listByCourse/reassign)
• Stable per-segment IDs assigned at create()/update() and backfilled for existing transcripts
• Persisted source AUDIO (compressed, not raw video) in IndexedDB/OPFS (web) + expo-file-system (native) keyed by transcriptId, with quota handling and a 'keep media' opt-out; transcript detail loads persisted audio so scrub works after reload
• Small pre-capture sheet: real title + course + lectureDate + language (replacing filename/timestamp default), threaded through transcribeStore.start
• Course-aware library home: course switcher / 'All' chip, grouped sections, sort (by lecture/date/unstudied-first), keeping a clean flat view for zero-course users

Why this order: This is the spine. Migrations, courses, persisted media, and segment IDs are hard prerequisites for retrieval, RAG citations, flashcards, and calendar links — building any of those first would mean reworking the schema under live user data. Capturing course + lectureDate at ingest is free now and expensive to backfill later. Nothing here needs a model download, so it ships fast and de-risks everything downstream.

Phase 1 — Exam-time semantic recall (the killer use case)

Goal: Ask a question in your own words, scoped to a course, and jump straight to the lecture moment that answers it — definition first, then every mention across the term.

Features:

• EmbeddingEngine (web-first) reusing the existing CDN loader + WebGPU/WASM detection, default gte-small/bge-small multilingual; chunk + embed per segment
• Vector store (segvecs table) + brute-force cosine in a Web Worker; embedding-on-save as a background jobs/queue.ts job + a cancellable backfill job for the existing library
• Segment-hit result type {transcriptId, segmentId, start, snippet, score} + hybrid lexical+semantic ranking (RRF) so exact terms/acronyms still win; SQLite FTS5 for the lexical half on native
• Scoped & filtered search (course / date range / instructor) over both retrieval paths
• Cross-lecture concept jump list with a 'likely definition' cue-phrase heuristic, click-to-seek into persisted audio
• Per-course knowledge overview (lecture count, total hours, last recorded) — the 'gain an oversight' ask

Why this order: This is the highest-value request and is buildable the moment Phase 0 lands. It reuses the existing transformers.js loader almost verbatim and the already-enabled cross-origin isolation, so the heaviest infra is essentially free. It ships entirely on-device with NO LLM dependency — pure embeddings + ranking — so it works on far more devices than any generative feature and delivers the core promise before tackling the riskiest pieces.

Phase 2 — Live capture into courses + long-recording reliability

Goal: Make Wisp where lectures actually get recorded, not just imported — reliably, for 90+ minutes, straight into the right course.

Features:

• Live in-app recorder: web getUserMedia + AudioWorklet → Float32 PCM through the existing toMono16k, manually windowed (~25-30s) through transcribeChunk; native whisper.rn realtimeTranscribe; big 'recording, on-device' indicator + level meter; mic permissions per platform
• Long-recording reliability: incremental chunk persistence to OPFS (web) / streaming WAV append via expo-file-system (native), pause/resume with correct absolute-time stitching, 'resume unfinished recording' crash recovery, Wake Lock + 'keep tab in front' UX
• Record/import straight into a course (course picker at capture, 'record into this course' from a course screen)
• System/tab audio capture on web via getDisplayMedia({audio:true}) for online/recorded lectures, feature-detected with graceful degradation
• Per-course dashboard: chronological lecture timeline, coverage bar, manual 'studied'/lastReviewedAt toggle

Why this order: Phase 1 makes the corpus valuable; Phase 2 makes it easy to FILL. It comes after retrieval deliberately — recording into an empty, unsearchable library is far less compelling than recording into one that already pays off at exam time. Capture reuses the existing PCM pipeline; the real work is reliability (incremental persistence, recovery) which is where a whole class's audio is won or lost.

Phase 3 — Deterministic learning helpers (no model required)

Goal: Turn lectures into active-study material that works on every device, with the AI layer as an optional upgrade rather than a gate.

Features:

• Per-lecture extractive summary (TextRank/TF-IDF) + key-term glossary, each item click-to-seek to where the term was defined; stored as a derived artifact
• Flashcards from glossary/definition sentences (cloze) + SM-2 spaced repetition in-app + Anki-compatible CSV export in the EXPORT_META registry
• Quiz/practice mode: MCQ with distractors drawn from sibling course glossary terms, each question linked to its justifying segment
• Exam-prep dashboard: frequency/emphasis-cue topic importance cross-referenced with SRS/quiz coverage to surface 'high-importance topics you haven't reviewed' (labelled 'frequently emphasized', not 'will be on the exam')

Why this order: These are pure src/lib modules in the existing vitest style with zero model download, so they deliver the 'learning helpers' vision broadly and cheaply BEFORE committing to the heavy/risky LLM runtime. They reuse Phase 1's embeddings/keyphrases and Phase 0's course grouping, and they create the SRS/quiz state the exam-prep dashboard needs.

Phase 4 — Enrichment, dates & references (legitimate-first network)

Goal: Surface the deadlines, definitions, and readings buried in lectures — pulling from the wider world by topic only, never by shipping your lectures out.

Features:

• Date/deadline detector (pure relative-date grammar anchored to lectureDate) → reviewable CandidateEvents → .ics export (web) + optional expo-calendar write (native), every event user-confirmed and linked to its source segment
• Citation/reference extractor (DOI/arXiv/ISBN/author-year) → per-course bibliography with BibTeX/RIS/CSL-JSON export
• Metadata lookup via LEGITIMATE APIs (Crossref/OpenAlex/Semantic Scholar/Open Library/Google Books/arXiv) + Wikipedia topic context cards — sending only the topic/keyword/DOI/ISBN string, cached locally
• Book finder leading with legitimate sources (Open Library, Gutenberg, DOAB, Google Books, WorldCat, user-configured university OpenURL); Anna's Archive/LibGen ONLY behind a default-OFF, warned, link-only toggle
• Optional user-owned encrypted whole-library backup/export (WebCrypto, user passphrase, BYO storage)

Why this order: This delivers 'auto add dates to a calendar' and 'auto-search relevant books' — the explicit enrichment asks — and introduces the FIRST network calls, so it's deliberately late and carefully scoped to topic-only, legitimate-first lookups that respect the privacy line. The encrypted backup lands here as the honest no-account safety net for a now-substantial library (ideally pulled earlier if users accumulate data fast).

Phase 5 — Optional generative AI layer (cited RAG, on opt-in)

Goal: Add genuinely-good generated summaries, flashcards, and a cited 'ask your lectures' answer box — strictly as an optional, gated upgrade over everything already shipped.

Features:

• GenerationEngine abstraction (web WebLLM/WebGPU + optional BYO-key cloud; native MLC/llama.cpp as a follow-up), capability-gated with a no-WebGPU fallback to the Phase 3 deterministic helpers
• Cited RAG 'Ask your lectures': retrieve top-K (Phase 1) → grounded answer where every claim carries a lecture+timestamp citation chip that jumps to the persisted audio, with source segments shown verbatim beneath; refuses when unsupported
• Generative upgrades to summaries/flashcards/quiz short-answer grading, always labelled and timestamp-linked
• 'Explain this passage simpler' inline action (gated — no deterministic fallback, so unavailable without a model/key, never faked)

Why this order: Generative AI is the most exciting but most fragile piece — multi-GB downloads, WebGPU-only, native runtime unproven, hallucination risk. By sequencing it LAST, retrieval (Phase 1), persisted audio (Phase 0), and deterministic helpers (Phase 3) already deliver the vision for everyone; the LLM becomes a clearly-optional quality boost for capable devices/BYO-key users, with a real fallback for the rest, instead of a hard dependency that locks out half the user base.

Architecture & on-device AI changes

• Versioned migration runner on BOTH backends (this does not exist today and is the gating risk): Dexie is hardcoded to version(1) in repo.web.ts; native repo.native.ts uses bare CREATE TABLE IF NOT EXISTS with no PRAGMA user_version path. Add a step-by-step, idempotent, transaction-wrapped migration runner keyed on user_version (native) and Dexie .upgrade() chains (web), with a round-trip test that seeds a v1 DB and asserts no data loss before anything ships.
• New Zod-gated entities behind the existing StorageRepo interface, each following the parseDraft() gatekeeper pattern in schema.ts: Course {id,name,code,instructor,term,color,schedule?}; courseId (nullable FK) + lectureDate + studied/lastReviewedAt + tags[] added to Transcript; Attachment {id,transcriptId,kind:'media'|'pdf'|'image'|'link',mime,size,localRef,extractedText?}; Chunk/SegVec {id,transcriptId,segmentId,start,end,vector(Blob)}; Flashcard + Review (SM-2 state); CalendarEvent {id,courseId,startISO,sourceSegmentId,confidence}. Keep every write funneling through Zod — extend the gatekeeper, never bypass it.
• Stable segment IDs: segments are currently keyed by array index (transcript/[id].tsx), which breaks every citation/vector link the moment an edit splits or reorders a segment. Add a persistent per-segment id at create() time and on edit, and migrate existing transcripts to assign ids. This is a hard prerequisite for embeddings, RAG citations, flashcards, and calendar source-links.
• Persisted source media (today the audio is a throwaway in-session object URL revoked on the next job — transcribeStore.makeAudioUrl + transcript/[id].tsx 're-import to scrub' warning). Store the decoded/compressed AUDIO (not raw video) as a Blob in IndexedDB/OPFS (web) or app sandbox via expo-file-system (native), keyed by transcriptId, with quota-aware warnings, a per-item 'keep media' opt-out, and graceful QuotaExceeded handling. This unblocks jump-to-audio for every retrieval feature.
• EmbeddingEngine abstraction mirroring TranscriptionEngine exactly (one interface, engineImpl.web.ts / engineImpl.native.ts split). Web reuses the proven new Function('u','return import(u)') CDN loader, WebGPU/WASM backend detection, and progress_callback plumbing VERBATIM — the same transformers.js pipeline() call runs 'feature-extraction'. Default model gte-small/bge-small (~120MB, 384-dim) multilingual, all-MiniLM fallback. Native embedding runtime is the genuine unknown (whisper.rn is ASR-only) — ship web-first, treat onnxruntime-react-native as a follow-up spike.
• Vector index: brute-force cosine over in-memory Float32 vectors (a few hundred lectures = low tens of thousands of 384-dim chunks fits in RAM and scores in single-digit ms), run in a Web Worker (web) / off the JS thread (native) so it never janks the UI. Add upsertVectors/searchVectors to StorageRepo. Do NOT build an ANN/HNSW index at this scale.
• Hybrid ranking as a pure, unit-tested function in src/lib (reciprocal-rank-fusion or weighted sum) merging the existing lexical searchText score (upgraded toward BM25; SQLite FTS5 can replace the LIKE scan cheaply on native) with cosine similarity, so exact terms/acronyms/symbols never lose to a fuzzy semantic hit. search() must return a segment-hit result type {transcriptId, segmentId, start, snippet, score} — today it only returns transcript-level TranscriptMeta with no score, no timestamp.
• Background indexing: extend the pure jobs/queue.ts reducer with an 'embedding' job type plus a cancellable/resumable backfill job to embed the existing library on upgrade, reusing the existing progress-bar plumbing. Embedding-on-save must run as a background job, never inline in the save path.
• Optional GenerationEngine abstraction (third sibling to Transcription/Embedding engines) with three interchangeable backends: web-local WebLLM/WebGPU (Qwen2.5-1.5B or Llama-3.2-1B/3B q4), native-local MLC/llama.cpp (follow-up), and an OPTIONAL bring-your-own-key cloud client. Hard guardrails: capability-gated behind WebGPU, a no-WebGPU fallback to extractive/deterministic helpers, and a BYO-key path that sends ONLY retrieved text snippets (never raw audio, never whole-library dumps) with explicit pre-send disclosure. Verify WebLLM model-shard fetches work under the existing COEP credentialless headers.
• New pure, vitest-style modules in src/lib (no platform imports), matching the existing pure-engine pattern: src/lib/learn/ (TextRank/TF-IDF summary, keyphrase/glossary, SM-2 srs.ts, cloze + MCQ distractor generation), src/lib/enrich/dates.ts (relative-date grammar anchored to lectureDate), src/lib/enrich/citations.ts (DOI/arXiv/ISBN/author-year regex), and an ics.ts exporter + bibliography exporters (BibTeX/RIS/CSL-JSON) registered in EXPORT_META.
• A thin, clearly-labelled network client (the app is 100% offline today) for legitimate metadata-only lookups (Crossref/OpenAlex/Semantic Scholar/Open Library/Google Books/arXiv/Wikipedia), sending ONLY topic/keyword/DOI/ISBN strings — never audio or transcript bodies — with local result caching, polite User-Agent/mailto, and graceful CORS degradation to a search link. Plus an optional whole-library encrypted backup (WebCrypto AES-GCM + KDF, user passphrase, file goes where the user chooses, import re-validated through the Zod gatekeepers) as the only honest no-account 'sync' story.

Risks

• On-device LLM feasibility is the single biggest risk. WebLLM needs WebGPU + ~1-4GB download + capable RAM; it excludes Safari, older devices, and most Android browsers, and native has NO LLM runtime in this stack (whisper.rn is ASR-only — adding MLC/llama.cpp is substantial new native work, Android-first). Mitigation: a multi-GB model can NEVER be a hard dependency. Every generative helper must have a deterministic/extractive floor (TextRank summary, cloze cards, MCQ from glossary) that works with zero model, with the LLM as an upgrade and an optional BYO-key path. Ship generation web-first and gated; verify shard fetches under COEP credentialless.
• Anna's Archive (and LibGen) are copyright-infringing shadow libraries. Surfacing them — even as an outbound link — carries real legal/ToS/reputational risk for a public app and likely app-store rejection. Honest handling: legitimate sources ALWAYS primary and on top (Open Library borrow, Project Gutenberg + DOAB full text, Google Books preview, Crossref/OpenAlex/Unpaywall OA PDFs, WorldCat, and a user-configured university OpenURL link-resolver for their entitled access). Anna's Archive/LibGen appear ONLY behind a default-OFF settings toggle with a plain-language copyright/legality warning, de-emphasized, link-only — never proxy, host, or download. Acknowledge the user's request but be explicit this is at their own risk.
• Migrations are the classic footgun and the library is the user's only copy. Neither backend has a migration history today (Dexie frozen at version(1); native CREATE TABLE IF NOT EXISTS, no user_version). A half-applied SQLite migration or failed Dexie .upgrade() can corrupt a semester of irreplaceable lectures. Mitigation: idempotent step-wise migrations, native steps wrapped in a transaction, additive-only changes so existing transcript paths/tests stay green, default existing rows to courseId=null/'Unsorted', and a seeded-v1 round-trip test gating release. Land the encrypted backup/export early so users have a safety net.
• Transcription error propagation + AI hallucination. Everything downstream (search, summaries, glossary, flashcards, dates, citations) is built on possibly-misheard Whisper output, and small local LLMs hallucinate. Mitigation: keep RAG strictly grounded (never answer without retrieved support; 'I couldn't find this in your lectures'), always show source segments verbatim with a click-to-seek timestamp so the user verifies against the professor's actual words, and label every derived artifact honestly ('auto-extracted, verify' / 'likely definition' / 'frequently emphasized — not a prediction'). Never assert generated content as authoritative.
• Storage growth and browser eviction. Persisted lecture audio is hundreds of MB to GB; 384-dim float32 vectors add ~1.5KB/chunk (~tens of MB/semester); IndexedDB/OPFS quotas are finite and browsers can evict storage. Mitigation: persist decoded/compressed AUDIO not raw video, store quantized (int8) vectors, show a storage budget + per-item 'delete media keep transcript', handle QuotaExceeded gracefully, and offer recompute-embeddings-on-import to keep backup bundles small.
• Web real-time/long-capture limitations. transformers.js has no realtime API (live-on-web = manual ~25-30s windowing through the existing transcribeChunk, latency = window length), WASM-CPU may not keep up on low-end laptops (drop to tiny or fall back to record-now/transcribe-after), and mobile browsers suspend backgrounded tabs and revoke the mic. Mitigation: Wake Lock + a clear 'keep this tab in front' UX and incremental chunk persistence to OPFS so a refresh/crash doesn't lose the recording; be honest that reliable true-background capture on web is not possible (native needs a foreground service / background audio mode + permissions).
• Heuristic accuracy with honest framing. Relative-date resolution ('next Friday', ASR mis-hearing 'fourteenth' vs 'fortieth'), schedule-based course auto-classify (breaks on rescheduled/overlapping slots), keyphrase/definition-cue detection, and free-text citation/title matching are all approximate and language-dependent. Mitigation: always PROPOSE and let the user confirm/edit/override — never auto-file, auto-create a calendar event, or auto-link a citation silently; show confidence and the source segment. Note that recording createdAt is the save-time proxy unless lectureDate is captured. Diarization/'by speaker' is explicitly out of scope — neither engine does it, so don't promise it.

Deliberately not building (yet)

• A custom ANN/HNSW vector library — premature at student-corpus scale; brute-force cosine over tens of thousands of vectors in a Worker is single-digit ms. Document the few-hundred-lecture ceiling and revisit only if a real user hits it.
• A hosted sync server or any Wisp-operated account/backend — it would break the no-account, privacy-first identity and make Wisp the custodian of users' private lecture data. The honest cross-device story is the user-owned encrypted backup/export file (BYO storage: their Drive/Dropbox/Syncthing/Files). Build that instead.
• Shipping a multi-GB local LLM as a required dependency — it must always be optional and capability-gated, with deterministic/extractive fallbacks and a BYO-key path. Do not let any core flow hard-depend on WebGPU + a giant download.
• Server-side transcription fallback and any cloud upload of raw audio/notes — this is the identity line. Cloud is OK only as opt-in BYO-key sending retrieved TEXT snippets, never the media.
• On-device training/fine-tuning of any model — out of scope, no benefit for personal study, large complexity/perf cost.
• Speaker diarization / 'sort by speaker' — neither transformers.js Whisper nor whisper.rn does it here; promising it would be dishonest without a separate diarization model. Defer.
• True .apkg Anki generation (bundled-media SQLite-zip) and PPTX parsing in-browser — ship Anki-compatible CSV and born-digital PDF text first; these heavier formats are later stretch items, not Phase work.
• 'Explain this passage simpler' as an early feature — it has no deterministic fallback (pure-LLM only), so it's unavailable on no-WebGPU/no-key devices. Gate it as a late add-on once the GenerationEngine exists; don't fake it.
• Generative RAG narrative answers BEFORE persisted audio + segment IDs land — a citation you can't click back to the audio is half a feature. Sequence retrieval + jump-to-text + persisted media first; layer the LLM answer on top.

---

Full feature list by lens

• Live in-app recorder (record-and-transcribe-as-you-go) (L, Fully local. Mic PCM is processed in-bro…) — A record button on the library/course screen that captures the mic, persists raw audio in chunks, and feeds those chunks into the EXISTING pipeline as they arrive so a partial transcript fills in live
• Long-recording reliability: chunked persistence, pause/resume, recover-after-crash (L, Fully local. Chunks are written to the d…) — Make a 90-minute lecture survive a tab refresh, a backgrounded app, a dying battery, or an OOM. Append raw audio to durable storage incrementally during capture (web: IndexedDB/OPFS-backed chunk store
• Record/import straight into a course (M, Fully local. Pure metadata in the existi…) — Add a lightweight Course concept (id, name, color/term) and let every capture path — live recording AND file import — be tagged with a course at the moment of capture (a course picker in the New/Recor
• System / tab audio capture (web) for online lectures & recorded videos (M, Fully local on web — the captured audio …) — On web, add a 'capture tab/system audio' source using getDisplayMedia({audio:true}) so the user can transcribe a Zoom/Teams/YouTube/Panopto lecture playing in another tab without a separate recording
• Attach slide decks / PDFs / readings to a lecture (with on-device OCR of slide text) (XL, Fully local. pdf.js, Tesseract.js, and M…) — Extend ingest beyond audio: let the user attach the lecture's slide deck (PDF/PPTX-exported PDF), readings, or a photo of the whiteboard to the SAME library item as the transcript. Render/extract text
• Capture-time metadata (title, course, date/time, language) instead of filename-as-title (S, Fully local. Pure metadata captured and …) — A small pre-capture sheet that sets a real title, course, the lecture's actual date/time, and the spoken language before recording/importing — replacing today's behavior of using the raw filename or a
• Course/Semester data model + repo extension (the foundation) (M, Fully local. Pure schema + repo work on …) — Introduce a Course entity (id, name, code e.g. 'CS-101', instructor, term/semester label, color, optional weekly schedule) and link every transcript to at most one course via a new courseId field. Add
• Per-course dashboard (timeline + coverage + studied state) (M, Fully local. Pure aggregation over rows …) — A course detail screen: header (code, instructor, term), a chronological timeline of its lectures (lecture #, date, duration, segment count), aggregate stats (total lectures, total hours captured, las
• Auto-classify a new recording to a course (schedule-first, opt-in topic/keyword assist) (M, Fully local. Schedule matching is pure d…) — On save, suggest the most likely course instead of dumping into a flat list. Tier 1 (cheap, reliable): match the recording's wall-clock time against each course's weekly schedule (e.g. 'Mon 10:00 Ling
• Cross-lecture topic overview ('everything on topic X across the term') (L, Fully local in both forms. Keyword/keyph…) — Per course, build a local index that maps recurring topics/keyphrases to the exact lectures + timestamps where they appear, surfaced two ways: (1) a course 'topic cloud / index' you can tap to jump to
• Source-media + attachment storage per lecture (slides, readings, audio) (L, Fully local. IndexedDB stores Blobs nati…) — Let a lecture own attachments beyond the transcript: the original audio/video file (so playback survives a reload — today the object URL is in-session only), plus slides/PDFs/images the user adds. Sto
• Course-aware library home (filter, group, sort) replacing the flat list (M, Fully local. Pure UI + store changes ove…) — Rework app/index.tsx from one undifferentiated list into a course-segmented home: a course switcher / 'All courses' chip row, sections grouped by course, sort options (by lecture number, by date, by u
• On-device semantic search index (segment embeddings + local vector store) (L, Fully local on web: transformers.js embe…) — Compute a small sentence embedding (e.g. transformers.js feature-extraction with Xenova/all-MiniLM-L6-v2 or bge-small, 384-dim, q8) for every transcript segment (or for ~1-3 sentence windows to give e
• Hybrid ranking (semantic + keyword) with exact-term fallback (M, Fully local. Pure computation over alrea…) — Blend the new embedding similarity score with the existing lexical match (the current substring/LIKE path, upgraded to BM25-style term weighting) into one ranked result list. Exact phrase matches and
• Cited RAG: ask-a-question over your lectures with timestamped, jump-to-audio citations (XL, Local-first but with an honest caveat: a…) — A question box ('Ask your lectures') that retrieves the top-K relevant segments (Feature 1), feeds them as context to a local LLM (WebLLM / llama.cpp, e.g. a 1-3B instruct model on web; optional bring
• Scoped & filtered search (course / date / instructor) over the retrieval layer (M, Fully local. Pure metadata filtering ove…) — A filter bar above search/RAG to constrain the corpus before ranking: scope to a single course or all courses, a date range, and instructor. Both semantic and keyword retrieval pre-filter candidates b
• Cross-lecture concept jump list ('Where was X defined / discussed?') (M, Fully local. Retrieval + a small rules-b…) — For a query, return not a single answer but a chronological, deduplicated list of every moment across the whole library where the concept appears — each row showing lecture, timestamp, a one-line snip
• Persisted source media + segment ids (the enabler for jump-to-audio) (L, Fully local — strictly more private than…) — Persist the imported audio/video locally so any cited result can replay the exact moment, and give every segment a stable id. Web: store the media Blob in IndexedDB (a new 'media' store, or OPFS for l
• Per-lecture summary + key-term glossary (extractive-first, generative-optional) (M, Extractive tier is fully local pure JS (…) — On opening a saved transcript, generate (a) a 3-8 bullet TL;DR and (b) a glossary of key terms with the sentence each was first defined in, every item carrying the source segment's start time so tappi
• Flashcard generation with built-in spaced repetition + Anki export (L, Cloze cards, SRS scheduling, in-app revi…) — Turn a lecture (or a selected set) into Q/A flashcards, reviewable in-app with an SM-2 spaced-repetition scheduler, and exportable. Card sourcing is layered: cheap deterministic cloze deletions from g
• Quiz / practice-question mode (MCQ + short-answer) (L, MCQ generation + auto-grading of MCQ + a…) — Generate a practice quiz over one or several lectures: multiple-choice (with distractors drawn from sibling glossary terms in the same course so they are plausible, not random) and open short-answer q
• Local LLM runtime: bundled small model on web + optional BYO-key cloud (shared engine for all generative helpers) (XL, Web on-device path is the CDN dynamic-im…) — A single LearnLLM abstraction (mirroring the existing TranscriptionEngine pattern: one interface, platform impls, plus a cloud impl) that all generative helpers call. Web impl loads a small instructio
• "Explain this passage simpler" inline action (M, On-device via WebLLM, or optional BYO-ke…) — Select a segment (or a range) in the transcript editor and tap 'Explain simpler' to get a plain-language paraphrase, an analogy, and 'what you should already know' for that passage — shown inline bene
• Concept map + cross-lecture topic linking (on-device embeddings) (L, Embeddings run fully locally (MiniLM is …) — Compute a local sentence-embedding per segment (transformers.js feature-extraction, e.g. all-MiniLM-L6-v2, loaded via the same CDN dynamic-import path as Whisper), cluster them into topics per lecture
• Exam-prep mode: predicted-important topics + review-gap tracker (M, Fully local: it is aggregation/scoring o…) — A study dashboard for a course (or selected lecture set) that ranks topics by an importance heuristic (frequency across lectures, professor emphasis cues like 'this is important / will be on the exam'
• Date & deadline detector -> .ics calendar export (M, Fully local. Date parsing is pure determ…) — A pure module in src/lib (e.g. src/lib/enrich/dates.ts) that scans a saved transcript's segment text for dates, deadlines and exam mentions ('the midterm is on March 14', 'submit by next Friday', 'fin
• Citation & reference extractor with title heuristics (M, Fully local for extraction — it is pure …) — A pure src/lib/enrich/citations.ts that mines transcript text for scholarly references: explicit DOIs (regex 10.\d{4,}/...), arXiv ids, ISBNs, 'Author (Year)' patterns, and quoted book/paper titles ('
• Metadata lookup via legitimate scholarly APIs (DOI/Crossref/OpenAlex/Open Library) (L, Needs-network-but-not-private-data. Only…) — An optional 'Resolve references' action that takes the extracted Reference candidates and queries legitimate, free metadata APIs to confirm and enrich them: Crossref / DOI content negotiation for DOIs
• Per-course bibliography (with course grouping as the backbone) (L, Fully local. Courses, assignment, dedup …) — Introduce a first-class Course entity (id, name, term, color) and a courseId on each transcript, then auto-aggregate every confirmed Reference across a course's lectures into one deduplicated, sortabl
• Topic context cards (Wikipedia / Wikidata key-concept glossary) (M, Keyphrase extraction is fully local (pur…) — Detect the salient named entities and technical terms per lecture (capitalized multi-word phrases, glossary-style 'X is defined as...' patterns, optionally TF-IDF over the course corpus to rank import
• Book finder: legitimate sources first, optional user-enabled shadow-library link (M, Needs-network-but-not-private-data, and …) — For each detected book/ISBN, a 'Find this book' panel that LEADS with legitimate sources — Open Library (read/borrow + availability), Google Books preview, Project Gutenberg (public-domain full text),
• On-device semantic search across the library (local embeddings) (L, Fully local. Embedding model runs in-bro…) — Compute sentence embeddings for every segment using a small local model via transformers.js on web (the exact CDN-dynamic-import pattern already used for Whisper, e.g. a MiniLM/all-MiniLM or bge-small
• Versioned data-model expansion behind StorageRepo (Course/Semester, Attachment, Embedding chunk, Flashcard, CalendarEvent) (L, Fully local. IndexedDB (web) and SQLite …) — Extend the single StorageRepo interface with new Zod-validated entities and the migrations to back them. Web: bump Dexie to version(2)/version(3) with .upgrade() transactions adding stores courses
• On-device semantic search engine (transformers.js embeddings + brute-force vector index) (L, Fully local. Embedding model downloads o…) — Add an EmbeddingEngine that mirrors the existing TranscriptionEngine abstraction: a platform-agnostic interface with engineImpl.web.ts (transformers.js feature-extraction pipeline, reusing the SAME
• Optional local generative LLM layer for summaries, flashcards & RAG answers (WebLLM/WebGPU + native MLC, with BYO-key fallback) (XL, Local-first: WebLLM (web) and MLC/llama.…) — Introduce a GenerationEngine abstraction (parallel to TranscriptionEngine/EmbeddingEngine) with three interchangeable backends behind one interface: (a) web local — WebLLM/WebGPU running a small instr
• Course-scoped retrieval, ranking & background indexing (search-quality + the job queue glue) (M, Fully local. Job scheduling, ranking mat…) — The orchestration layer that makes search/RAG actually good and keeps indexing off the UI thread. (1) Extend the existing pure jobs/queue.ts state machine with an 'embedding' job type so chunking+embe
• Auto-extracted dates -> on-device calendar events (CalendarEvent entity, optional native export) (M, Fully local extraction and storage. The …) — A local extraction step that scans a transcript's text for dates/deadlines ('the midterm is on November 14', 'assignment due next Friday') and proposes CalendarEvent rows written to the new `calendarE
• Topic enrichment: legitimate book/paper lookup (search-or-link), with Anna's Archive flagged (M, Needs-network-but-not-private-data: only…) — For a lecture or a selected topic, derive key terms (locally, from the transcript / embeddings / optional LLM) and offer 'find related reading' that queries LEGITIMATE, CORS-friendly metadata APIs and
• Optional user-owned encrypted backup/export (no server, no account) (M, Fully local and user-owned. Encryption h…) — A privacy-preserving way to move the whole library (transcripts, courses, embeddings, flashcards, calendar events) between devices and to back it up — WITHOUT any Wisp-operated server. Export everythi