For Death and the Powers, I have primarily been working on the interactive systems, including the wearable sensors and vocal analysis system that we use to transform Jim Maddalena’s live performance into the performance of the entire theatrical set. It’s a terrific theatrical challenge: what do you do when your main performer is not actually visible to the audience, but has to be present in a completely non-anthropomorphic form? I’m measuring all kinds of elements of Jim’s performance, from his singing voice to his gestures to his breath, and abstracting all of these elements into complex parameters that convey the character’s emotional expressivity, like the qualities of his movement or the timbre of his voice. These parameters can then be mapped to parameters of the graphical system that drives the visual content on the three “bookshelves” of the set. These mappings are not naïve or one-to-one, with just one particular sensor linked to just one parameter of the visual output. Instead we’re creating sophisticated and rich connections that make the walls seem alive and able to convey the presence of Simon Powers.
One important part of this work was to create a mapping system that would be very flexible and react appropriately to the fast-paced theatrical rehearsal process. The visual system is capable of creating an enormous variety of representations of Simon Powers; as those representations change and develop from scene to scene and during the course of rehearsals, the way that they’re controlled by the live performance has to change as well. I have to be able to adjust the mappings between Jim’s performance and the visuals immediately when given directions from Diane Paulus or visual notes from Alex McDowell, without having to stop the program to make changes. The rehearsal process so far has been testing that system, but it seems to be working successfully!
Opera of the Future 