“Read To Me”

As a Resident Artist at CultureHub, prototyped a space that visually responds to the live reading of poetry and prose.

Thanks to the teaching of Matt Romein and help of Oren Shoham, was able to get a basic version working in four days in June 2018, which was then tested by an amazing assortment of poets, writers, and editors.

As text is read aloud, the words appear on screen, changing the color of the entire 50 foot space based on what word was read (for example, if you read "volcano" the space turns red). While very basic aesthetically, the project opens up new possibilities for how new media artists and writers could collaborate to perform literary work live.

We used Google Cloud's speech-to-text API, Python, Max/MSP, and the NRC Word-Emotion Association Lexicon created by Saif Mohammad and Peter Turney at the National Research Council Canada.

During the second week in November, we ported the project into Unity with the help of Sam Von Ehren, and then game developers Flan Falacci, Dennis Carr, and Milan Koerner-Safratta spent three days developing prototypes for several types of interactive graphics.

This culminated in a public event titled “Read To Me,” a reading with poets Meghann Plunkett, Rhiannon McGavin, Angel Nafis, and playwright Jeremy O. Harris.

You can watch video of Saeed Jones and Tommy Pico trying out early versions. Video of the second version is available here and here and here.

GitHub available here.

Deirdre Coyle wrote about her experience as a reader for Electric Literature here. Was interviewed by the Google Cloud dev team here. And featured on Google’s Instagram here