HOW TO STAY HUMAN (without much effort)

⌈ Machine learning is entering audio in earnest, for now in professional audio. Special algorithms "bred" to distinguish the sound of individual instruments and voices allow the reverse operation of mixing - "un-mixing." The following text is about the challenges of it. ⌋

HEN WE READ: "VACATION FROM WORK", it's hard to tell which segment the emphasis falls on. Until recently, it would have been clear - it would have been about the conditions that need to be met in order to have a vacation from work, or - this is the cynical version - how to work, and not to strain yourself. In modern times, we understand it even differently. Since the massive shift to remote work, the two, that is, 'vacation' and 'work,' are no longer mutually exclusive. As Bartek Chacinski mentions in his micro-fiction Holidays from Work, it's all about so-called workation (work and vacation combined) ("Polityka" No. 30, 19.07-25.07.2023, p. 93).

However, I have this feeling that no matter how we read these two words, we will have in mind a break from work and a rest. And this one is easiest when someone will do all our work for us. For example - a robot. In the Polish cultural sphere, the word, and the concept itself, was popularized by Stanislaw Lem, whose The Tales of the Robots, part of the monumental Cyberiada, has been part of the school reading canon for years.

⸜ Stanisław Lem’s Cyberiada with wonderful graphics by Daniel Frost, and a communist-era pirate translation of Isaac Asimov's I, Robot novel, published on a duplicator

In the Anglo-Saxon world, on the other hand, the word, and the term 'robotics', was popularized by Isaac Asimov, first using it in the short story The Game of Tag (Runaround, 1942). Many of us who have been interested in SF novels and films of this type are familiar with the three laws of robotics formulated by the writer:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Two things draw attention. The first is the absolute obedience of man. Because, after all, that's what robots were created for - first in the mind of the writer, and then in the minds of capitalists. The figure of the writer was used here for a simple reason: the word robot comes from the Slavic word robota, and this means hard work, effort. Its author was Josef Čapek, a writer, brother of another writer we probably know better - Karel Čapek. In fact, he was the one who popularized it, writing the play R.U.R. (Rossumovi Univerzální Roboti) (1920). And it's one of the few examples of how powerfully the Slavic language has influenced languages around the world.

But to the point. While obedience is an important component of the laws of robotics, the other component, arguably more important, is our, or people's, safety. It is on these fears that a whole galaxy of SF movies, books and games have grown, including the iconic Terminator. To some extent, this fear of a rebellious machine has been contained, or at least tamed. And what about algorithms, the modern robots?

Man has it in his nature that if he doesn't have to work, he won't work. The robot invented by Čapek was thus a perfect sublimation of this aspiration. It didn't come from nowhere, that's clear; its predecessors were machines, primarily steam engines, and the ideal was the Industrial Revolution of the 19th century. But while robots were supposed to liberate us, the Luddites, rightly so, pointed to the dark side of progress and the lack of knowledge of what it would bring. In a word, it was about responsibility for inventions and how one uses them. The first two laws of robotics speak to this.

⸜ The first such successful intervention of artificial intelligence into material that is part of our cultural legacy, ELLA FITZGERALD's album The Lost Berlin Tapes - photo press mat. Verve

Benjamin Labaut, a Chilean Booker and National Book Award finalist and author of The Terrible Green (Black, 2023), selected by The New York Times as one of the ten best books of 2021 - it was also on Barak Obama's short list - said in an interview with Newsweek:

We are all responsible for the world we create. With our actions, thoughts, desires and words. (...) The problem is not technology itself. It is the nature of human beings. We carry demons within us. The only thing that has changed is that now some of them are made out of code, out of algorithm, they have more reach. But demonicity is what it was. (No. 30/2023, p. 95).

Indeed, the modern robot, or machine meant to work for us and thus make us happier, is algorithms. For example, ChatGPT and other programs based on machine learning. Although they have nothing to do with the machine as such, there remains something of the "robot" in that name and the belief that the machine is fundamentally something other than a human. And if that's the case, then its goals are also different. And this is what raises most doubts, also in the audio world.

⸜ Deluxe Hardback Edition of EVA CASSIDY’S I Can Only Be Me • photo mat. Press, Blix Street Records

Thomas Conrad, reviewing EVA CASSIDY's I Can Only Be Me album for “Stereophile” magazine, points out that although the singer died at the age of 33 and was almost completely unknown during her lifetime, after her death she became one of the world's most popular singers. During her lifetime, she released, self-released, just three albums, while the album Songbird, compiled in 1998, sold in 12 million copies. The thing is, her latest album was prepared using artificial intelligence.

This was not the first time this has happened. The technology that was used comes in a straight line from Peter Jackson. It was he who used AI to prepare the soundtrack for the The Beatles: Get Back (2021). The idea was to separate the various instruments and vocals as best as possible and remix them. A new version of the album Revolver (2022) was similarly created. Recorded in 1968 on a four-track tape recorder, the material was used by Giles Martin to "teach" the computer program the sound characteristic of each musician. He used Jackson-owned WingNut Films Productions and the engineer in charge of machine learning there, Emile de la Rey, for it.

Martin was very impressed with what de la Rey did with the material he gave him:

I sent him Taxman and he sent me, literally, the guitar, bass and drums separately - you could even hear the squeak of Ringo's pedal on his bass drum. It's alchemy... we refined it and worked on it together until we ended up with more than just four tracks to work with, and so we were able to do a stereo mix of Revolver. That opened the door for us.

⸜ What Giles Martin discovered when remixing Revolver, → www.YOUTUBE.com, accessed: 27.07.2023.

The analogy Martin reached for in this interview spoke of "unbaking" the dough and separating the original ingredients from it, such as flour, eggs, sugar, etc. This allowed Revolver to be "put together" anew.

⸜ Gil Martin remixed THE BEATLES’ Revolver album by teaching an artificial intelligence program to sound individual vocals and instruments

The results of these activities surpassed my wildest imaginings. Like what I heard when I reviewed ELLA FITZGERALD's The Lost Berlin Tapes (Verve Records | UMe - B0032589-01, 1962/2020; reviewed → HERE ⸜ PL). Originally stereophonic recording was remastered with tracks abstracted by Kevin Reeves, using iZotope RX 8 Music Rebalance software. With it, he split the original stereo mix into four tracks: drums, bass, piano and vocals, which he then remixed. In his words:

"This technology did not exist a year ago and they called me by chance (just as I was looking into the tapes). On the original tape, Ella’s voice was a little thin in the mid-range and the piano and drums were panned hard left and hard right, which is very old school. I was able to bring her more forward and brought up the bottom so you can even hear fingers on the strings. The result is that Ella's much more in the room with you. When I sent it to Ken, he said, 'This is the best live recording of Ella I've ever heard.'"

⸜ ROB LEDONNE, Unearthing A Lost Ella Fitzgerald Recording, 60 Years Later, → www.GRAMMY.com, Oct. 1st. 2020, accessed: 25.07.2022.

This was probably my first encounter with material created from a combination of a person's knowledge and his ignorance of what the software he is using actually does. Because using the help of learning programs is a great unknown. We are at a place where so many new things can be done with musical material that it will not resemble the original work in any way. Is Revolver taken apart and put back together the same Revolver as before? Certainly it is a new artistic work, created from the combination of human creativity and the tool that is AI.

The ethical implications of such actions are manifold. Can we even alter someone's work, especially if that person is no longer alive? Or maybe Cassidy wouldn't have wished to sing with an orchestra; after all, she was a humble girl performing neofolk covers, not an opera diva. We don't know that. What's more, we're adding new elements to the world, and yet there are enough older ones, we're generally overstimulated.

Conrad, in “Stereophile”, rightly notes that the singer's voice sounds wonderful and that probably never before has it been so emotionally moving. And after all, that's what we're all about in audio, the truth, the transfer of emotion from "that" side of the recording to "ours." It is standard to say that the better the audio system, the better it reconstructs these emotions. Thus, one can put an equal sign between the degree of intensification of emotions and the quality of sound.

It is extremely tempting, after all, I myself urge you to listen to more and better products and CD releases. Somewhere in a corner, a little dark and dusty one, however, lurks this thought: what if artificial intelligence gets to know us well enough that it, or rather the humans using it, will bend the original recordings in such a way as to squeeze out of us as many emotions as possible, and thus makes us spend more money? And, after all, these do not have to be the emotions originally contained in the recording, but a completely new, stronger one, only that it has a different vector - it will be emotions derived from manipulation.

But artists also influence us, even if they do it unconsciously, right? - Of course they do, that's what art is all about. However, it is not manipulation, as far as art is concerned, but creation. Is a piece newly constructed from small original bricks still original? It seemed that we had put these considerations behind us, because this was, after all, what all postmodern art criticism dealt with. The answer was clear: yes, it's new art, only that it's undeveloped. Nothing new comes from it, so to speak. What is authentic and innovative also draws on the achievements of predecessors, but in the role of inspiration, and not by arranging the components into new, static configurations.

⸜ Screenshot from iZotope RX 8 Music Rebalance with the "Isolate a Vocal with Music Rebalance in RX" window, a view of an isolated vocal track from a stereo mix • photo mat. press. iZotope

Joanna Brach-Czaina, a Polish philosopher, wrote in one of her articles:

Modern man is afraid of the lack of events and prefers strong stimuli, even unpleasant ones, over weak ones. He fears a diluted existence. He is a child of the big city, accustomed to an excess of sensations. He notices silence and feels anxious. He cannot hear noise. He constantly needs a source of sound and an amplifier.

⸜ JOANNA BRACH-CZAINA, Gęstość, tłok, miasto w: tejże, Rzeczywistość komponowana, Warszawa 2023, p. 171.

And algorithms remixing old recordings and creating new ones will provide us with these impressions in abundance. However, the author of The Cracks of Existence is not a pessimist. She points out that we are "working through" this - physical and mental - abundance thanks to this extraordinary liveliness, alternation and dynamism. Admittedly, we are talking about the city and its problems, but this can, I think, be extended to the subject of this editorial as well.

Because also in the case of artificial intelligence vs. audio matters there is nothing to be afraid of. While the use of AI to automatically master material for various streaming services is a reality, and that means lowering the average sound quality, the upper end, so to speak, of this type of action can produce interesting results. For, the aforementioned Cassidy does indeed sound great.

Most important in all this, however, is that the recording engineers' system is not perfect. When one listens to her voice, especially in recordings from concert albums, one can hear remnants of the low reverberation and imperfections of the stage microphone she sang to. The orchestra, on the other hand, recorded the material in the former St. Luke's Church, an 18th-century building with a large volume and lots of glass, which in itself produced a long sound extinction. The long reverb imposed on the whole does not always cover the original one.

Expect, soon enough, a rash of stereo and even surround Atmos recordings originally available only in mono. The deep archives of Blue Note, Columbia, as well as many record labels specializing in classical music will take on new value - literally. It will be possible to sell the same material once more, but at a much higher price. Is this a good thing or a bad thing? - We don't know that yet. What is known is that instead of an old-style technological breakthrough, i.e. a change in format, we will get a 21st century breakthrough, i.e. a change in the philosophy behind the recording.

I am not clear on how to treat such recordings. It will take some more time to look at this new branch of music, which is being born before our eyes, with a critical eye and assess it as such. Here and now, it is necessary to accept it on its own merits and hope that in all this the authentic spirit of the work in question will be preserved. And we can hope based on the fact that these are not omnipotent and omniscient programs, and above all that they are not perfect, they make mistakes, as we all do. ●

WOJCIECH PACUŁA
Chief editor