Skip to content

Add life-story books with an AI ghostwriter, and voice dictation - #7

Merged
JeremySNR merged 16 commits into
ccr-c6ab75dd-8lzqn8from
claude/ai-autobiography-voice-transcription-iuh1ax
Oct 11, 2026
Merged

JeremySNR merged 16 commits into
ccr-c6ab75dd-8lzqn8from
claude/ai-autobiography-voice-transcription-iuh1ax

Conversation

@JeremySNR

@JeremySNR JeremySNR commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

Quillgon can now write true stories (a life story or a memoir) as well as novels. In this mode the AI works like a professional ghostwriter: it interviews the author, drafts the chapters itself, and goes back and forth with them on revisions. Dictation works across the whole app, novels included. The research and design behind this is in docs/life-stories-and-dictation.md.

Dictation

  • There is a mic in every prose editor, margin note and answer box. Hold Alt D to talk.
  • Speech is recorded in the app at 16 kHz and split at pauses.
  • It is transcribed by OpenAI gpt-4o-mini-transcribe, local whisper.cpp (QUILLGON_WHISPER_CMD / QUILLGON_WHISPER_MODEL), or an offline placeholder.
  • The book's names (people, places) are sent to the recogniser so it spells them right.
  • A small scribe agent tidies the transcript. If the tidy-up adds a name or rewrites a sentence, the raw words are kept instead.
  • New Settings → Voice tab. On macOS: a microphone entitlement and usage string.

Life-story books

  • Book type: Book.kind is now novel, life-story or memoir, with life settings (subject, born, audience, who holds the pen).
  • New data: stories, sessions, a question queue, a ghostwriter's notebook, and margin notes anchored in the text.
  • Navigation: the sidebar changes for these books. Novel-only tools (twists, plot points) are hidden.

Interview sessions (engine.interviews, ghostwriter agent)

  • Each session starts with an agenda that gives a reason for each item.
  • The ghostwriter asks one open question at a time and labels the technique it uses.
  • Each story moves up a ladder: what happened, one particular time, who was there, senses, what was said, how it felt then, what it means now.
  • What the author says is filed into people, stories, the timeline and questions as changes that can be undone.
  • Another speaker's account is kept as their own version.

Chapters (engine.life)

  • The ghostwriter proposes the book's shape, and each chapter has its own stage: Gathering, Ready, Draft, Your notes, Revising, Approved.
  • Chapter 1 is the voice sample.
  • Drafts are written from the stories only. Margin notes cover gaps, two tellings that disagree, thin passages, voice choices and sentences with no source.

Revision rounds

  • The author gives free-form feedback. The ghostwriter replies with a numbered plan, then per-passage changes to accept or reject.
  • It may push back once per point, and the author can overrule it.
  • Feedback about the whole book becomes a style rule.
  • Each round is one commit. Approving a chapter locks it.

Other screens

  • Timeline: shows ages, shades the gaps, and offers "Ask me about these years". Moving a date carries it into the stories and scenes that use it.
  • Ghostwriter's notebook: an editable page of what the ghostwriter knows about the author.
  • Before you share: claims with no source, open gaps, people who have not agreed to appear, sensitive stories and unapproved chapters. It also writes an author's note that opens the exported book.

CLI and evals

  • quillgon init --life | --memoir [--about Name] [--born 1948]
  • quillgon session (interviews from the terminal)
  • quillgon eval-life uses simulated interviewees with hidden facts (evals/lives.yaml, interviewee agent). It scores facts drawn out, depth, follow-up on surprising answers, refused / leading / off-limits questions, and unsourced sentences.
  • pnpm e2e:life

Safeguards

  • Questions: code checks every question before the author sees it. Questions that suggest a detail the author never gave (a false-memory risk) are refused, and so are off-limits topics. A refused question gets one retry, then falls back to a safe open question.
  • Off the record: these answers are never sent to the model, and nothing from them is filed.
  • Drafts: where the material runs out, drafts leave [TK: …] gaps instead of guessing. Every sentence is checked against what the author actually said.
  • Quotes: revision edits that would change a quote are refused.
  • Microphone: permission is granted only for audio from the app's own origin. Every other permission is still denied.
  • Recordings: session audio is kept under audio/ in the book and git-ignored. Transcripts are versioned.

Testing

All of these were run locally on 5ad88d7:

  • pnpm lint: 0 errors, the existing 23 warnings
  • pnpm typecheck
  • pnpm test: 284 passed
  • pnpm build
  • pnpm scale-check --max-prompt 8000
  • pnpm site:build
  • pnpm site:check
  • scripts/e2e-ui.ts
  • scripts/ui-refresh.ts
  • scripts/e2e-covers.ts
  • scripts/e2e-life.ts: the whole life-story flow in Chromium with a fake microphone
  • quillgon eval-life --offline

Everything above uses offline models. The ghostwriter has not been run against a live model yet, so eval-life scores mean nothing until it is.

Not built yet

  • Parakeet and other local models beyond whisper.cpp; streaming transcription
  • Telling speakers apart automatically
  • Family and published editions as branches
  • Phone or companion-page interviews; photos as prompts

🤖 Generated with Claude Code

https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre


Generated by Claude Code


Note

Medium Risk
Large new product surface (interviews, drafts, revisions) with model calls and memory-safety checks; dictation and mic permissions widen the desktop security footprint but are scoped to same-origin audio.

Overview
Adds life-story and memoir books alongside novels: interview sessions with a ghostwriter agent, chapter shaping and staged drafting, margin notes and revision rounds, plus timeline, notebook, and before you share screens. The desktop app gets a life-specific sidebar and routes; novel-only flows (twists, ledger in threads, etc.) step aside when project.life is set.

Adds voice dictation end-to-end: browser recording (segmented long takes), /dictate transcription and tidy-up, mic UI and Alt+D hold-to-talk in editors and session/chat boxes, Settings → Voice, and Electron/macOS microphone permission plus CSP media-src for blobs.

Docs and tooling: large README section, quillgon init --life|--memoir, quillgon session, quillgon eval-life, and pnpm e2e:life. Agent prompts gain life-aware intentBlock and shared-life prompts.

Reviewed by Cursor Bugbot for commit 6f0f289. Bugbot is set up for automated code reviews on this repo. Configure here.

claude added 10 commits October 11, 2026 12:30
Covers whether Quillgon could write an autobiography or memoir, how
life stories are structured and gathered, what Wispr Flow does and what
exists in open source for dictation, and how both would map onto the
current code, with a phased plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
Adds how professional memoir ghostwriters interview, draft and revise,
the interviewing research that constrains an AI doing the same, and a
design: interview, draft and revise loops, per-chapter stages, margin
notes, an editable notebook, questioning rules, the existing code each
piece reuses, and an eval with simulated interviewees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
Hold Alt+D or press Dictate in the scene editor and the words land at the
cursor; the thinking partner takes long takes cut at pauses. Audio is
recorded as 16 kHz WAV and transcribed by a local whisper.cpp, OpenAI,
or the offline placeholder. A new scribe agent tidies fillers and
self-corrections, but its text is used only if it adds no names and
keeps the speaker's words; otherwise the deterministic clean-up goes in
and the author is told why. Names come from the book's people and
places plus a per-book word list. Settings gains a Voice tab.

The Electron shell now allows the microphone (audio only, app origin
only); macOS gets the usage string and audio-input entitlement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
A book now has a kind: a novel, a life story or a memoir. A life story
starts with a narrator character, an opening question bank after
McAdams' Life Story Interview, and one act for its chapters. New files:
stories/ (remembered episodes with a ladder of how far each is drawn
out, verbatim quotes, sourced details, other versions), sessions/
(interviews, turn by turn), ghostwriter/questions.yaml and notebook.yaml,
and scenes/<id>/notes.yaml for margin notes. People, timeline events,
chapters and cards gain optional life fields, so a novel's files are
unchanged. Recordings go to audio/, which life books keep out of git.

Agents get a life-story shared prompt (truth first, real people, the
author's voice), and the welcome screen and quillgon init can start a
life story or memoir, for the author or someone else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
A life story is gathered in sessions. The ghostwriter plans an agenda
from the gaps in the book (thin stories, the question queue, empty
years), says why, and asks one question at a time. Each answer is filed
into a story (the ladder: what happened, one particular time, who was
there, senses, words said, felt then, means now), with quotes kept word
for word, details traced to the turn they came from, new people, dated
events, questions for later and notes on how the author talks. Every
exchange can be undone.

Every question is checked in code before it is asked: leading questions
that suggest a detail the author never gave are refused (a known route
to false memories), as is anything off limits; vague, surprising and
emotional answers are flagged for follow-up. Answers given off the
record are never sent to the model or filed. Someone else's account is
kept as their version, never merged into the narrator's memory.

The Sessions view runs it in the app, with spoken answers kept as
recordings; quillgon session runs it in the terminal. The offline
ghostwriter makes every step usable without a model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
Shaping the book: the author says what the book is really about, who
holds the pen (interviewer, co-writer or ghostwriter) and the shape
(chronological, thematic, braided, frame, vignettes, one season); the
ghostwriter proposes chapters from the stories, leaving out (but
keeping) what does not serve the book. Each chapter then moves through
its own cycle on a board: gathering, ready, draft, your notes,
revising, approved.

A chapter is drafted from its stories only, in the author's voice
(their notebook, their spoken words, the chapter that set the voice).
Gaps are left as [TK] markers with a question rather than guessed,
disagreeing tellings and thin passages get notes, and a code check
adds a note for every sentence nothing the author said supports and
every quote that is not word for word. Notes are answered in the
editor by typing or talking; answers become material, and "later"
becomes a question for the next session. Sensitive stories stay out.
Approving a chapter locks it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
The author says what is wrong in their own words, beside the draft:
"it's too posh", "Dad wasn't like that", or nothing beyond the
answers to the notes. The ghostwriter reads back a numbered plan
(edits, rules for the whole book, questions for the next session, and
at most one honest pushback) and proposes span edits. The code refuses
any edit that would change words the author said word for word, or
that does not match the text, and flags new claims nothing the author
said supports.

Each change is reviewed passage by passage (j/k, a/r, shift-enter, as
in ripple review). Applying the round is one commit: accepted edits go
into the prose, gaps answered in notes are filled, rules join the style
sheet (checked on every word from then on), questions join the queue,
and an overruled pushback is recorded in the notebook so it is not
reopened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
…e you share

A life story's sidebar now leads to:
- Timeline: the life by year with ages beneath, key moments marked,
  unsure dates drawn as bars, empty years shaded and turned into
  questions for the next session. Correcting a date carries into the
  stories and scenes told from it, each with a note to check the text.
- People: how each person relates to the narrator, living or not, real
  name, pseudonym or composite, whether they have agreed, and the
  stories they are in, in place of want, need, wound and lie.
- Themes, in place of threads and the setups-and-payoffs ledger.
- The ghostwriter's notebook, editable: how you talk, what is off
  limits, promises, settled arguments, and questions for other people.
- Before you share: claims nothing the author said supports, gaps and
  open notes, living people who have not agreed (stronger for a
  published book), sensitive stories in the text, unsigned chapters,
  and an author's note written from the choices made, which opens the
  exported book.

The writing desk counts stories and sessions and suggests the next
step by whose turn it is. Fiction-only tools (read back, revise to
card, predictability, the ledger, twist verdicts) are hidden, and a
life story's prose stays in sync when the author edits it. Settings
and the welcome screen take a year of birth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
- interviewee agent: a simulated person who reveals hidden facts only to
  questions that earn them (model and offline versions)
- evals/lives.yaml and runLifeEval: interview, shape and draft each life,
  then score facts drawn out, depth, follow-up on surprises, refused,
  leading and off-limits questions, and unsourced sentences
- quillgon eval-life, writing evals/runs/life-<id>/report.md
- pnpm e2e:life
- README: life stories and memoirs, dictation and speech models, the new
  CLI commands and the eval; the research doc gains "What was built"

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
@vercel

vercel Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
quillgon Skipped Skipped Oct 11, 2026 3:42pm UTC

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread apps/desktop/src/components/dictate.tsx
Comment thread apps/desktop/src/views/sessions.tsx
Comment thread apps/desktop/src/views/sessions.tsx Outdated
Comment thread apps/desktop/src/components/dictate.tsx
… field context

Stop leaves listening immediately and waits for the transcription chain so
trailing silence with pending segments cannot reopen the mic. Session send
awaits that flush and snapshots text/heard afterward. Each pause-cut audio
file gets its own player, and fieldTarget commits inserts with flushSync so
chained segments see up-to-date before/after context.
- A long take's next piece read the field before React had rendered the
  last one, so it joined to stale text; fieldTarget now reads back what
  it just wrote until React catches up
- Stopping waits for every piece still being transcribed, and the mic
  shows "tidying" rather than "listening" while they land
- Sending a session answer or a thinking-partner message waits for the
  last spoken words, so they go with the message instead of landing in
  the emptied box
- Every recorded segment of a long answer can be played back, not just
  the first

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
Both fixed the same four findings. Keeps the read-back field target
(tested, no forced synchronous renders) and the ref-based session send,
and takes from the autofix the immediate "tidying" state on stop and the
guard against starting a new take while one is still being tidied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Answers can be sent twice
    • Added a sending ref lock before awaiting dictation stop, and made stop() share one flush promise so nested callers wait for the last WAV.

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit a78183d. Configure here.

Comment thread apps/desktop/src/views/sessions.tsx Outdated
Lock session send before awaiting stop, and coalesce concurrent
dictation stop() callers onto one flush so the last WAV is not missed.
- A second Enter while the last spoken words were still landing could
  send the answer twice; session and thinking-partner sends now lock
  before waiting
- A stop called while another was still handing over the recorder's
  last piece returned early, so a send right after releasing Alt+D could
  miss those words; it now waits for that piece to be queued
- Test for the concurrent stop (fails on the previous version)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
Same fix as the commit before it (lock the send before waiting, and a
second stop waits for the first); keeps that version, which is tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VesxdmVnfJM1sL5RyuQhre
@JeremySNR
JeremySNR merged commit 726c88e into ccr-c6ab75dd-8lzqn8 Oct 11, 2026
6 of 7 checks passed

This branch was previously deployed

1 inactive deployment
Preview — 6f0f2893 Deployed Oct 11, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants