Let’s listen together
I built the parsing and evaluation systems behind Cofheard’s listening and rehearsal experience.
My work spans AI, backend, and product engineering.
Cofheard lets people listen to a script with a cast of voices, perform a character in Run Lines, and share work through script profiles and a community. My central challenge was preserving what the writer put on the page while interpreting enough of its structure to give each line to the right voice.
That makes parsing part of the performance. Assign a sentence to the wrong person and the listener hears a different exchange. Mistake a direction for dialogue and the scene gets interrupted. Run Lines depends on the same interpretation: Cofheard plays the other roles and waits while the actor speaks their part.
The product also connects a script to its writer and the people discovering it. VERSSE profiles hold a description and supporting material; Community provides a place to find and follow work. A shareable profile link is distinct from an invitation into a private script. Community posting and public listing remain limited during private testing.
See the community interface
Keeping the source intact
Scripts communicate through layout as much as words. A name above a paragraph assigns a speaker. Indentation separates dialogue from directions. A speech can continue onto another page without repeating the character’s name. Scans add the earlier problem of recovering the printed text accurately.
I separated that source material from the model’s interpretation. Extraction produces a structured document with stable line identities, coordinates, text, and provenance. The model labels spans within that document, assigning roles and speakers to existing source ranges. The application reconstructs the output from those references, so a model cannot quietly rewrite the script while classifying it.
Every interpreted line can be traced back to the page. That constraint limits accidental rewriting and makes errors easier to locate, but it leaves real problems to solve: extraction can still be wrong, a boundary can split a thought, and a preserved sentence can still end up with the wrong speaker.
- Original PDFExtract text and layout
- Structured sourcePreserve text and line identities
- Span labels & rulesAssign roles, then validate
- Cast & playbackListen and rehearse
Making corrections reusable
I wanted a bad line to lead to a correction the parser could use elsewhere. The improvement loop prepares a worksheet and the relevant page evidence. Specialist judges review speech, directions, and structure, with lyric review for musical scripts. Each finding records the source, its current classification, the proposed interpretation, and the reasoning behind it.
That record feeds rule authoring. A candidate needs a condition that can recur across scripts, such as dialogue continuing at the same margin or a speaker carried across a page break. Adjudication decides whether to accept, merge, replace, or refuse it. Accepted candidates enter staging, where the parser runs again with earlier corrections carried forward as regression checks.
- Page evidence Source document and worksheet
- Specialist judges Classify, explain, propose corrections
- Candidate rules Turn findings into reusable conditions
- Adjudication Accept, merge, replace, or refuse
- Staging Keep candidate changes out of production
- Reparse and review Carry earlier regression checks forward
↳ Findings return to the next review
Staging → human release decision → production
I kept production release separate from that cycle. Here, “self-improving” means proposing and checking corrections. It does not mean retraining the model or letting an agent promote its own rules into the live reader. A person still decides what ships.
Following the evidence
In a saved parse of Petronia Paley’s SWEETIE PIE, the continuation “this Jekyll and Hyde thing comes rolling out” was classified as action without a speaker. One thought had been split between Oona and narration. The judge pointed to the same column, no new character cue, and no block gap, then proposed keeping Oona as the speaker. Later saved parses do. Several changes happened between those records, so they establish the resulting behavior without isolating a single cause.
The full reviewed page below shows a different continuation problem. At the top, “Opening her door.” was labeled action. The judge’s orange marking connects it to a proposed correction in the margin: carry Janus’s speech across from the preceding page. The original classification, expected role, explanation, and suggested fix appear together.
Read the marked correction
- Original classification
- Action: “Opening her door.”
- Proposed correction
- Dialogue for JANUS CHANDLER.
- The judge’s explanation
- The first body line continues Janus’s speech from the previous page. The next line, “Opening—”, sits at the same margin and is already assigned to Janus.
- Suggested rule
- Carry the governing speaker across a page break when the first body line continues at the dialogue margin.
Plain-text summary of the saved AI-authored review.
This example remained unresolved in later saved output. An articulate diagnosis had not become a reliable fix. Keeping that case visible helped me separate a proposed correction from a demonstrated change in behavior.
I also found that the benchmark and production paths could use different prompts. Before interpreting any result, I needed to know which source, prompt, rules, and execution path produced it. The surrounding engineering became as important as the model: source tracking, constrained changes, and records I could inspect directly.
That is the part of Cofheard I’m most interested in carrying forward. A correction needs to survive the next parse and the next script. The purpose is concrete: an actor hears the right cue, and a writer hears the scene they actually wrote.
Product views use demonstration content. Parser evidence comes from saved worksheets and judge reports; no new parser run is presented as a historical result. Script excerpt: SWEETIE PIE, A LOVE STORY, Petronia Paley.