“Read To Me”
As a Resident Artist at CultureHub, prototyped a space that visually responds to the live reading of poetry and prose.
Thanks to the teaching of Matt Romein and help of Oren Shoham, was able to get a basic version working in four days in June 2018, which was then tested by an amazing assortment of poets, writers, and editors.
As text is read aloud, the words appear on screen, changing the color of the entire 50 foot space based on what word was read (for example, if you read "volcano" the space turns red). While very basic aesthetically, the project opens up new possibilities for how new media artists and writers could collaborate to perform literary work live.
We used Google Cloud's speech-to-text API, Python, Max/MSP, and the NRC Word-Emotion Association Lexicon created by Saif Mohammad and Peter Turney at the National Research Council Canada.
During the second week in November, we ported the project into Unity with the help of Sam Von Ehren, and then game developers Flan Falacci, Dennis Carr, and Milan Koerner-Safratta spent three days developing prototypes for several types of interactive graphics.
GitHub available here.