✘ Multimodal hyperdrive: from the vision of a release, to releasing a vision
And: Artificial emotional intelligence; a listening bar in Milan; the new director of the Venice Music Biennale; an acoustic playground in Shanghai; Electronic folklore
My whole career I have been seeking compelling combinations of instrumental performance, traditional forms of composition, electronic music and computer music. And also figuring out ways for these experiments to find their listeners, whether through my music or the soundtracks and records I contribute to. I have the good fortune of having an amazing global network of musicians, artists, labels, scientist and technologists to collaborate with, which has made it possible to bring my idea of “Classical Computer Music” to fruition from clubs and raves all the way to the gilded halls of opera houses and the United Nations. In these diverse scenarios I keep seeing paradigm shifts big and small in the way artists and audiences relate to each other, moving beyond some tried and true but decreasingly applicable practices of the music industry. Today I’d like to share with you one story of my own, a project called ‘Modes of Vibration’, to explain how ongoing efforts in amplifying human musical gestures by melding the real world and the virtual world have led me to rethink the idea of a release.
Exploring the simulation: algorithmic adventures
A decade ago the trajectory of my work was deeply influenced by a pivotal encounter with a team of scientists in Edinburgh, modern sound synthesis pioneers who, led by the visionary Dr. Stefan Bilbao, created a new class of algorithms that simulate real and impossible instruments. I made my first two solo records ‘Multiverse’ and ‘Chaos and Order’ with them, exploring extreme sound simulation techniques reminiscent of raytracing in video games (running, in fact, on a supercomputing GPU cluster), layering them with synths and live strings. I loved it because I could dial in different degrees of ‘unreality’, bridging the gap between organic and synthetic, creating impossible instruments that still sounded like material objects with otherworldly properties. And through the ambient label “A Strangely Isolated Place”, these works found appreciative ears. However, when performing live, I came up against an interesting problem: these simulated instruments sounded amazing in immersive venues, but on a regular stage their power was diminished by the lack of a worldly presence to match the sounds. When the audience wasn’t immersed inside the simulation, it lost emotional impact, and I felt like I was playing violin against a backing track.
Escaping the simulation: transforming algorithms into sound sculptures
So about a year ago, when I was commissioned a piece for EMPAC, an arts & tech theatre in Troy, NY, I decided to turn the paradigm on its head: instead of creating an algorithm to represent a physical object, I wanted to create a tangible embodiment of an algorithm. I wanted to create something visually striking, but playable: a sculpture-based synth. I floated the idea to mathematician Dr. Michele Ducceschi, who runs the NEMUS lab at University of Bologna and specialises in the rather sci-fi mission of bringing ancient instruments back to life through algorithmic restoration, and he got enthusiastically on board.
I wanted a simple object, something rudimentary, so we designed four suspended metal plates, each with a different slash in the middle that changes how they resonate to various frequencies. This way, each plate embodies one of four ‘modes of vibration’ algorithms, which later came to be the project’s title. Calculations in hand, I enlisted a designer in upstate New York and a local blacksmith in Milan, added four transducers (small contact speakers), and we created this mechanically simple, yet acoustically complex device, ready to be played with a computer or electric instrument.
Audience resonance
Frankly, I thought this was going to be a one-off experiment, but much to my surprise, when I showed these in an art gallery and a theatre, controlling them with Ableton and my electric violin, people reacted really positively, like I was performing alongside them, rather than through them. The reaction went something like: it isn’t a speaker, but makes sound, looks like a sculpture, but it moves, sounds like an instrument, but isn’t human. People really clicked with this idea of vibration and resonance between human and machine. Suddenly, some notably paid a lot more attention to my gestures on the violin, and how they affected the sounds and motion of the sculptures. Others closed their eyes and blissed out, visibly emoting to the sound. Something powerful was happening.
Unexpected journeys
And so I came back from the States with these four plates stashed in a rifle case, went back to NEMUS and told Michele, we have to do something with these. We each set out to do what we do best: the scientists, further analysing the embodied plates in the lab and refining the algorithms, and me, writing music for this weird hybrid instrument and figuring out how to expand its emerging language.
This was starting to look like a legitimate research project, so we were able to apply for a bit of funding from the European Research Council to support its initial development: this allowed us to turn the installation into a fine-tuned instrument, record the first half of the studio album, develop and document the performance and kickstart software development. Institutional support is a luxury sometimes afforded to art-science collaborations, and here it made a big difference.
Creative hyperdrive: the feedback loop between reality and simulation
As I made the music, I kept coming up with new ways to control this sculpture-based instrument using computers, violins, pianos, electric guitars, organs and modular synths. And here an unexpected development really put us into hyperdrive: we started creating playable simulations of the things I was doing, feeding those back into the plates, then figuring out how to model the result into yet another algorithm. Rinse and repeat, it was the movie ‘Inception’ gone peculiar. I would play the plates like a bizzarro quadraphonic surround metallic AM synth, then look for lateral strategies to simulate that in a stereo field. We let our imaginations run totally wild. The same plates that were the embodiment of a simulation kept being fed simulations of themselves, in a constant feedback loop between real and virtual, spawning some wild ideas for software development.
Tetrad, the digital twin
We looped in Craig Webb, a computer scientist and synthesis researcher who runs a plug-in company called Physical Audio, to turn some of those ideas into real-time plug-ins. They were originally only meant as tools for our production, but they proliferated into such a broad and compelling expansion of the vision that they evolved into a soon to be released commercial software instrument called ‘Tetrad’ (the Greek word for a 4 part chord in music theory). In many ways, ‘Tetrad’ is both a digital twin of the sculpture-based synth, a disembodiment of the compositions of the album, and something of an interactive manifesto for the whole project.
Soul vibrations: exploring human and emotional resonance
While this endless cycle of embodiment and disembodiment of vibration was taking place, it made me think about the concept of resonance on a conceptual, philosophical and emotional level, beyond the merely mechanical or algorithmic. So for the second half of the album I dove into the human and psychological side of this idea. I sought that in duets with members of my family: my brother, a punk hardcore singer, my mum, a scholar of Indian devotional singing, and my dad, a jazz guitarist. I also resonated with a couple of my very own bespoke synthetic friends: a custom built AI sonic agent trained on 10 hours of my music, and my personal digital twin ‘555n’, a synthetic machine artist modelled on my 20 year old musical and literary self, responding with ‘Tetrad’ to my human piano improvisation. For me, this creatively and emotionally brought my reflection on vibration full circle.
Releasing the Vision
The last few months have been a whirl of late night studio sessions, software-design brainstorming, UI prototypes, burning amplifiers (literally) and emerging storytelling. After all this, we have three things on our hands: my third studio album ‘Modes of Vibration’, released April 18th, a complex audiovisual performance system about to go on tour, and the ‘Tetrad’ plug-in which will be released by Physical Audio on June 18th. They are all part of a bigger whole.
As these elements coalesced, one thing became clear to me: this isn’t just a record release. The compositions are the core of a process, the starting point of a journey. Enter Anne McKinnon of Ristband, who fosters new paradigms, in touch with what bubbles up. She and her company have been amazing in encouraging and championing this idea of a abandoning the classic PR cycle of a release in favour of expanding and disseminating a vision. And so here we are.
Good vibrations
The remarkable way people have been reacting to this relatively simple piece suggests to me that stumbling upon the idea of embodiment has been timely: it has encouraged me down a prolific path, but also seems to tap into something special… something that not only engages the audience, but that has led me to engage my creative community in many different ways and even to include my family in the creative process. Perhaps in this era where everything is competing on a marketplace of attention that is crowded and weird, where AI and social media are undermining our ability to feel present and grounded, there’s great value in just making things vibrate directly around us and invite us to resonate with them.
LINKS
🫦 Emotion AI: transforming human-machine interaction (Noor Al Mazrouei)
“As a result, machines can simulate a human-like ability to recognize and express affection, which is a cornerstone of natural interactions. By doing so, emotion recognition not only boosts the machine’s decision-making capabilities in line with human emotional states but also allows for more empathetic and engaging user experiences.”
✘ I’m delivering a speech at the UN about this topic in July: the increasing capability of AI to simulate emotional intelligence is in my opinion extremely significant, disruptive and potentially dangerous - and nobody is talking about it.
👂Triennale Milano opens 'Voce', a sound-first space (Thomai Tsimpou)
“Voce unfolds like a modern-day chapel, its spatial logic echoing the layout of a church nave. A grid of rectangular columns divides the hall into three asymmetrical aisles, directing movement and framing the experience. Where an altar might traditionally sit, a sculptural wall of modular acoustic panels rises, a shrine to sound.”
✘ Very excitingly, Milan is becoming a hotspot for listening spaces where people meet around music and sound art. Tokyo-style listening bars are the latest trend, even in institutional spaces: this article celebrates the one called “Voce” built inside Triennale design museum, on the ashes of the club Jimi Hendrix used to play in my hometown.
💈 Caterina Barbieri appointed artistic director of Venice Music Biennale (Meena Sears)
"A vote of confidence in the intelligence and genius of the new generations.”
✘ Caterina is a champion of the new movement of young composers and artists bringing listening and community to the centre of their practice with an eye for the continuity between tradition and innovation that is dear to my heart.
🪁 Yuri Suzuki creates acoustic playground for Shanghai park (Alyn Griffiths)
“My main goal was to design a space that is open and welcoming, without barriers or intimidation. The sculpture invites people to gather, explore and engage with sound naturally and intuitively.”
✘ I just love everything about this, it’s so sweet.
🧝♀️ How today’s electronic music is bringing age-old folk traditions back to life (Philip Sherburne)
“It’s not just folk music: Zoom out, and it becomes clear that a number of centuries-old styles are seeping into experimental electronic music as of late.”
✘ One of the most interesting music movements for me right now is this highly creative resurgence of folk music mixed with electronics. I saw Tarta Relena, one of the bands in this article, at CTM festival and they blew me away.
MUSIC
Here’s the Bandcamp page of Modes of Vibration, where you can also find the white label vinyl and some discounted licenses of Tetrad on pre-order if you feel like supporting this project. Thanks!