Modalation

Starting with a story

I built Modalation with Avery Espiritu for Brown’s EDUC1485 course, exploring how a personal story could become an entry point into musical modes. A learner might recognize that music feels unsettled, hopeful, or bittersweet while having little language for the notes creating that impression. We narrowed a broader music-theory idea to modes so that one interaction could receive more attention.

My work connected the prototype to a functioning composition and playback system. After the story, three fixed questions ask how it begins, how its intensity changes, and what the ending should evoke. The answers make the learner’s emotional intention explicit. I connected a two-octave piano to browser audio so they could hear notes, select three, and replay a sequence. Those choices become anchors the composer is instructed to feature: a musical contribution made before the learner needs to name a mode.

Giving the composer more context

The piece follows an opening, development, climax, and resolution, each with a mode and playable notes. During development, the model tended toward familiar pairings, such as Ionian for triumphant endings and Aeolian for sadness. That could turn an exploration of expressive possibilities into another set of fixed associations.

I expanded the composition instructions to make the full modal palette explicit and added retrieval from a Supabase collection of modal knowledge. For each narrative section, the system looks for emotional keywords. When that search returns too few candidates, it broadens the search using tension and brightness. The retrieved descriptions give the composer context for an interpretation the listener can hear and question.

I connected Claude’s response to a structured composition format and added a Zod check for the data the player needs, with one correction attempt when the response fails validation. Each section arrives with its mode, notes, durations, and bass part, ready for the same playback system.