You are here

Virtual World Music Performance


Pianist Chryssie Nanou rehearses for a networked concert using JackTrip, a system for sending high-quality audio signals around the globe.
Photo Credit: 
David Kerr

Stanford Musicians on Two Continents Meet in New Virtual World for Live Shows

Stanford music and humanities scholars will entertain European and on-line audiences with a unique hybrid of music and technology this Fall.

Researchers from the Stanford Center for Computer Research in Music and Acoustics [CCRMA] and the Stanford Humanities Lab [SHL] will perform two mixed reality shows at the Torino Milano International Music Festival, often referred to as MiTo. Performers on both acoustic and electronic instruments located in Stanford, CA and simultaneously in Italy, will meet online in a custom virtual environment before a live audience in Milan.

The program titled, Due serate in Sirikata, spans multiple experimental genres, including musicians operating within a virtual landscape and performances streamed in live from distant locations. Due serate in Sirikata will be presented in two evening shows on September 12th and 13th. Each musical composition was specifically designed for the Carlo de Carli concert hall at the Milan Politechnic, and will be broadcast live on in Second Life on Idearium Island at 10 pm (1 pm pacific time.)

Due Serate in Sirikata explores the concept of being “live” within a mixed media constellation. Stanford Humanities Lab Director, Jeffrey Schnapp noted that it also celebrates a hyper-networked world where it is possible to experiment with radical new forms of collaboration. “This is where architectures can become instruments and bodies can become magic wands, and where the boundary lines once separating the acoustical from the visual, the gestural, or the verbal can be productively collapsed.”

Debut of 3D Multi-User Online Environment

The MiTo festival performances mark the inaugural public presentation of Sirikata, an open source virtual world platform for deploying 3D multi-user online environments, developed at Stanford University over the past three years. Schnapp said that the name was inspired by two meanings of the word; it designates populations of meerkats who choose to work together in underground burrows and it also translates from the Japanese term for ways of knowing. Both meanings underscore the developers’ ideas that collaboration and open information sharing and the triangulation of research, pedagogy, and experimental art practice.

Innovative Software Amplifies Listening Experience

A number of other technological components also play important roles in the performances. A surround-sound 16-channel audio system made up of 8 speakers, is powered by an algorithim technology called Ambisonics.  The software encodes sound with specific coordinate positions and decodes that sound to a speaker array. This allows for an exceedingly high degree of accuracy in the placement of sound.

CCRMA researcher, Juan-Pablo Caceres explained that Ambisonics allows the performers to bring nuanced audio elements to listeners. “Ambisonics is one of the technologies that allow us to represent a virtual space into its physical corresponding room, in the case, the concert hall and the virtual Sirikata world. The audience can this way not only experience Sirikata visually but also aurally; sound generated in certain places in the vitual world are heard at the same physical location on the concert hall.”

JackTrip software, developed at CCRMA by the SoundWIRE research group, is another component of the audio technology being employed for each show. JackTrip allows high quality audio to be streamed bi-directionally across the internet, making use of ultra-fast research internet connections for the sending of multiple streams of audio with a single audio channel often assigned to every single instrument. The SoundWIRE group is exploring ways not only of minimizing network delays so as to allow distant players to perform together live, but also to employ delay as a structural musical element that may, for example, give rise to works that are heard differentially on either side of the internet divide.

Stanford Laptop Orchestra

The performers will also incorporate, Q3osc, a virtual musical environment built on the Quake III open-source game engine used in concert performances by the Stanford Laptop Orchestra. The program begins with a laptop orchestra performance of one of the seminal works of the second half of the 20th century: Terry Reilly's 1964 composition In C. Premiered in 1964, In C introduced the musical style now known as minimalism to mainstream audiences.

Written as an "open score," it can be played by any combination of instruments and, over the decades, has been performed by percussion ensembles, guitar groups, a traditional Chinese orchestra, and a microtonal band, among many others. In C will be performed by members of the Stanford Laptop Orchestra ("SLOrk"), a novel orchestra of laptop computers in which each performer generates sound from their own computer through a custom-designed hemispherical speaker array.

Live, Intercontinental Musical Dialogue with Performing Avatars

The second segment of the program is Canned Bits Mechanics by Juan-Pablo Caceres: a work in which a live pianist interacts with complex musical patterns generated by two disklaviers (mechanical pianos) physically located at CCRMA in California but played in real-time from Milan by the composer himself. The co-presence of multiple pianos within a single concert hall is made possible by the use of JackTrip. In the case of Canned Bit Mechanics, the interplay between remote controlled disklaviers and a live piano is further shaped by a pair of avatar-performers, present only within the virtual world of Sirikata.  T

he third program element is an experimental work entitled Dei due mondi by CCRMA researcher Robert Hamilton, which will be performed live within a Sirikata virtual environment by eight performers, four at CCRMA and four in Milan. Dei due mondi explores the relationship between separate yet interconnected worlds. It bridges the gap between the physical and the virtual; the Italian concert arena and the CCRMA performance space.

The composition begins with virtual reconstructions of the Aula Carlo di Carli arena and CCRMA linked by means of a video feed with virtual performers each performing in Sirikata versions of their respective physical locations. As the realistically rendered walls of the two sites fade away, they gradually transform themselves into a single ghostly landscape in which the performers’ motions and actions are sonified and spatialized by means of Ambisonic processing and the 8 channel speaker array that surrounds the live audience.

"Dei due mondi overlays the virtual world over the physical world, initially in realistic fashion --if a perfomer runs within Sirikata from stage right to stage left generating sound, the audience will hear that sound move across the stage— but ever increasingly in abstract form." Hamilton said. The concluding piece in the program is a networked improvisation entitled dialoghi: an improvisatory work featuring virtual and live performers, located in physically distant physical sites, using computer networks as the medium through which they communicate. The same network that enables intercontinental musical dialogue inevitably creates temporal distortions that enhance and contort the final internal musical structure.

Thus an evening built around the diminution of latencies transforms latencies into an expressive element.