Evolutionary Computer Music

This paper presents a system that implements a society of agent performers that evolve expressive music performances (EMP) through their interactions. Each agent performer evaluates a performance using a fitness function derived from the structure of the performing piece. Agents are born with different fitness functions, representing preferences for different types of performance. A new-born agent firstly applies a Genetic Algorithm to generate a refined performance with its fitness function. Through an agent's life, it listens to the performances of others' that it connects to in the social network. Given specific conditions, it may try imitate some performances and modify its own preference. An agent performer dies of age or inactivity. New agent performers are occasionally created. With this model, we aim to generate expressive performances through simulating the effect of social pressure and culture transmission.

A multi-agent system is presented which generates melody pitch sequences with a hierarchical structure. The system has no explicit melodic intelligence and generates the pitches as a result of emotional influence and communication between agents, and the hierarchical structure is a result of the emerging agent social structure. Another key element is that the system is not a mapping from multi-agent interaction onto musical features, but actually utilizes music for the agents to communicate emotions. Each agent in the society learns its own growing tune during the interaction process.

Digital video offers an interesting source of control information for musical applications. A novel synthesis technique is introduced where digital video controls sound spectra in real time. Light intensity modulates the amplitudes of 32 harmonics in each of several synthesized "voices." Problems addressed include how to map from video to sound, dealing with global variations in light level, dealing with low frame rates of video relative to high sample rates of audio, and overall system implementation. In one application, images of light reflected from a shallow pool of water are used to control sound, offering a rich tactile interface to sound synthesis.

Recombining two or more music pieces to create a new composition is a captivating yet underexplored research challenge. This paper seeks to advance the field of music recombination by enhancing the architecture of the genetic algorithm proposed by Majumder and Smith [1]. Building on this foundation, essential genetic algorithm concepts are incorporated from the technical paper by Hermawanto [2], emphasizing musical aesthetics and computational efficiency. The core challenges addressed in this work include extracting features from original music pieces, designing an effective algorithm for generating new compositions, ensuring musical correctness and variation, and preserving the unique style of the resulting piece. To achieve these objectives, the music21 toolkit [3] is utilized to extract features from MIDI (Musical Instrument Digital Interface) files. This research evaluates the computational efficiency, pitch accuracy, and musical aesthetics of compositions generated by the improved system, comparing them with those created using the method outlined by Majumder and Smith [1]. This study bridges critical gaps in music recombination research and provides a solid foundation for future advancements in this innovative field.

tais pessoas. Nossa fonte de pesquisa é a literatura e através dela buscamos entender como uma determinada época, e as tramas que nela foram tecidas são dadas a ler. Trata-se, portanto, conforme afirma Chartier de um, {...} mundo de práticas que podemos unicamente ver de uma maneira particularmente parcial, limitada, obscura, e que este mundo de experiências, de crenças, de representações, de emoções, para nós, qualquer que seja a maneira de nos aproximarmos dele, é um mundo de opacidade, um mundo de distanciamento e, desta maneira, nos sugere uma prudência. Ao mesmo tempo resiste por outra razão: é que se é um mundo de práticas, para nós esse mundo de práticas se transforma em um mundo de cartas, em um mundo de papel, em um mundo de escrita, e há uma antinomia insuperável (CHARTIER, 1999, p. 213).

This paper presents a system that uses Genetic Algorithm (GA) to evolve hierarchical pulse sets (i.e., hierarchical duration vs. amplitude matrices) for expressive music performance by machines. The performance profile for a piece of music is represented using pulse sets and the fitness (for the GA) is derived from the structure of the piece to be performed; hence the term "structural fitness". Randomly initiated pulse sets are selected and evolved using GA. The fitness value is calculated by measuring the pulse set's ability of highlighting musical structures. This measurement is based upon generative rules for expressive music performance. This is the first stage of a project, which is aimed at the design of a dynamic model for the evolution of expressive performance profiles by interacting agents in an artificial society of musicians and listeners.

This paper presents the design of representing the performance profile with hierarchical pulse sets (i.e., hierarchical duration vs. amplitude matrices), and then applying Genetic Algorithm (GA) to evolve the hierarchical pulse sets for music interpretation, where the fitness of GA is derived from the structure of the music to be performed. In previous work , we have shown that GA can evolve suitable pulse sets for musical performance. Also, commonality and diversity are found among the performance profiles decided by those evolved pulse sets. This paper reports the experiment results from an improved system where a new version of fitness rules has been devised. On basis of this system, we are proposing the next steps for the research, that is, to build a dynamic model that evolves expressive music performance through agent performers' interactions.

As early as the 1950s and early 1960s, pioneers such as Lejaren Hiller, Gottfried Michael Koenig, Iannis Xenakis, and Pietro Grossi, among a few others, started to gain access to computers to make music. It soon became clear that to render music with a socalled "human feel," computers needed to process information about performance (e.g., deviations in tempo and loudness), in addition to the symbols that are normally found in a traditional musical score (e.g., pitch and rhythm). This was especially relevant for those interested in using the computer to play back scores. Indeed, the first ever attempt at creating a computer-music programming language, by Max Mathews at Bell Telephone Laboratories in 1957, was motivated by his wish to "write a program to perform music on the computer" (Park 2009 p. 10). It appears that this development began after Mathews and John Pierce went to a piano concert together. During the intermission, Pierce suggested that perhaps a computer could perform as well as the pianist. Mathews took up the challenge, which resulted in Music I, the ancestor of music programming languages such as Csound . Research into computational models of expressive performance of music (Widmer and Goebl 2004) is still an active area of study-particularly, research into devising increasingly more sophisticated automated and semi-automated computer systems for expressive music performance, hereinafter referred to as CSEMP. A CSEMP is able to generate expressive performances of music. For example, software for music typesetting is often used to write a piece of music, but most packages play back the music in a

This paper presents a project whereby the Sony AIBO robot (ERS-220 model) was programmed to compose original music in real-time through direct interaction with its environment and people. It begins with a brief introduction to robotic applications in the arts and music. Then it introduces the technical issues encountered in realising the project (namely, AIBO's functionality and musical intelligence) followed by the proposed solutions. It presents AIBO-Max, a new tool that we implemented for programming AIBO combining Max/MSP and R-Code, and the musical composition system that is used in association with AIBO behaviours to generate the music.

We propose a model of expressive music performance (EMP), focusing on the emergence of EMP under social pressure, including social interaction and generational inheritance. Previously, we have reported a system to evolve EMP using Genetic Algorithm, exploring the effect of generational inheritance. This paper presents a system that evolves expressive performance profiles through social interaction, with a society of artificial agent performers. Each performer owns a hierarchical pulse set (i.e., hierarchical duration vs. amplitude matrices), representing a performance profile for a given piece. An agent performer evaluates a performance profile with a set of rules derived from the structure of the piece in question, and imitates others' performances if appropriate. Then it modifies its pulse set accordingly. We demonstrate that suitable performance profiles emerge from social interactions where the diversity and the commonality of evolved performances are observed in the society of agents.

his paper presents a framework being developed to clarify discussion of algorithmic musical processes. We will firstly present some of the current terminological problems then discuss related research and background concepts. Following this, two principal attributes that enable intuitive comparison of algorithmic compositional processes will be defined in detail. The framework will then be demonstrated by applying it to some representative examples of differing algorithmic music systems.

Exploring the sounds available from a synthesis algorithm is a complicated process, requiring the user either to spend much time gaining heuristic experience with the algorithm or requiring them to have a deep knowledge of the underlying synthesis algorithms. In this paper we describe a computer system which facilitates a more exploratory approach to sound design, allowing the user to work at the level of the sounds themselves. In this system the synthesis parameters are managed by a genetic algorithm, which is directed by the ...

Artificial life (alife) is of interest to computer musicians due to its generative potential and the potential for producing lifelike behaviours for musical interaction. In this paper we consider how future developments in alife music could have equal bearing on the major themes in alife as on the music it produced. We focus on a discussion of the socio-cultural dimensions of making music with technology and argue that modern popular music making practice outside of individualist academic research projects is an important context for the ...

In this paper we introduce a system in which a cellular automaton is employed to control the evolution of synthesised complex sounds. One of the most interesting sound synthesis techniques available to musicians today is the granular synthesis technique. This technique involves the production of thousands of tiny sound granules (e.g. 35 milliseconds long) that combine to form complex and dynamic sounds. The main difficulty with granular synthesis is the specification of the nature of each sonic granule and how it will affect the overall result. Traditionally, granular synthesis systems have used probabilistic methods to control this. In this paper we describe an alternative method based on cellular automata. Our method has proved to produce sounds of a noticeably better quality than probabilistic methods in terms of the dynamics of unfolding and the naturalness of the overall form.

Transformational creativity requires a change of the search space. As such, Evolutionary Computation (EC) approaches are incapable of transformational creativity. In this paper, we discuss how canonical EC techniques can be extended in order to yield the potential of transformational creativity. We present a formalized description of how this can be attained, and the experimental results achieved with a metaevolutionary scheme.

BeatBender is a computer music project that explores a new method for generating emergent rhythmic drum patterns using the subsumption architecture. Rather than explicitly coding symbolic intelligence into the system using procedural algorithms, BeatBender uses a behavior-based model to elicit emergent rhythmic output from six autonomous agents. From an artistic perspective, the rules used to define the agent behavior provide a simple but original composition language. This language allows the composer to express simple and meaningful constraints that direct the behavior of the agent-percussionists. From these simple rules emerge unexpected behavioral interactions that direct the formation of complex rhythmic output. What is striking is that these rhythmic patterns, whose complexity is beyond human grasp, are both musically interesting and aesthetically pleasing. The output from the system is evaluated using both subjective and objective criteria to assess degrees of complexity, convergence, and aesthetic interest.

This chapter examines a range of approaches to algorithmic music making inspired by biological systems, and considers topics at the intersection of contemporary music, computer science and computational creativity. A summary of core precursor movements both within and beyond musical practice (ALife, cybernetics, systems art etc.) sets the scene, before core models and algorithms are introduced and illustrated. These include evolutionary algorithms, agent-based modelling and self-organising systems, adaptive behaviour and interactive performance systems, and ecosystemic approaches to composition and computational creative discovery. We close by reviewing themes for future work in this area: autonomy and agency, and the poetics of biologically-inspired algorithms.

A new model for automatic generation of Evolutionary Algorithms (EAs) by evolutionary means is proposed in this paper. The model is based on a simple Genetic Algorithm (GA). Every GA chromosome encodes an EA, which is used for solving a particular problem. Several Evolutionary Algorithms for function optimization are evolved by using the considered model. Numerical experiments show that the evolved Evolutionary Algorithms perform similarly and sometimes even better than standard approaches for several wellknown benchmarking problems.

A new model for automatic generation of Evolutionary Algorithms (EAs) by evolutionary means is proposed in this paper. The model is based on a simple Genetic Algorithm (GA). Every GA chromosome encodes an EA, which is used for solving a particular problem. Several Evolutionary Algorithms for function optimization are evolved by using the considered model. Numerical experiments show that the evolved Evolutionary Algorithms perform similarly and sometimes even better than standard approaches for several wellknown benchmarking problems.

In this paper we report the current status of our research work involving the application of dynamical systems to music composition. We introduce the fundamentals of our work and present CAMUS, a system for music composition which uses cellular automata. We then suggest a new engine for CAMUS, which uses fractal techniques and genetic algorithms to extend the capabilities of the system.

Numerical simulations used in computer animation provide computational environments for algorithmic sound production. We discuss the parallel production of images and sounds from a common model, and the data generated in such a model. Two cases are presented, one based upon classical differential equations, the other based upon parameterized gestures.

Tone Transfer is a novel deep-learning technique for interfacing a sound source with a synthesizer, transforming the timbre of audio excerpts while keeping their musical form content. Due to its good audio quality results and continuous controllability, it has been recently applied in several audio processing tools. Nevertheless, it still presents several shortcomings related to poor sound diversity, and limited transient and dynamic rendering, which we believe hinder its possibilities of articulation and phrasing in a real-time performance context. In this work, we present a discussion on current Tone Transfer architectures for the task of controlling synthetic audio with musical instruments and discuss their challenges in allowing expressive performances. Next, we introduce Envelope Learning, a novel method for designing Tone Transfer architectures that map musical events using a training objective at the synthesis parameter level. Our technique can render note beginnings and endings accurately and for a variety of sounds; these are essential steps for improving musical articulation, phrasing, and sound diversity with Tone Transfer. Finally, we implement a VST plugin for real-time live use and discuss possibilities for improvement. CCS CONCEPTS • Applied computing → Sound and music computing; • Computing methodologies → Neural networks; • Human-centered computing → Interaction techniques.

FM Synthesis is a well-known algorithm used to generate complex timbre from a compact set of design primitives. Typically featuring a MIDI interface, it is usually impractical to control it from an audio source. On the other hand, Differentiable Digital Signal Processing (DDSP) has enabled nuanced audio rendering by Deep Neural Networks (DNNs) that learn to control differentiable synthesis layers from arbitrary sound inputs. The training process involves a corpus of audio for supervision, and spectral reconstruction loss functions. Such functions, while being great to match spectral amplitudes, present a lack of pitch direction which can hinder the joint optimization of the parameters of FM synthesizers. In this paper, we take steps towards enabling continuous control of a well-established FM synthesis architecture from an audio input. Firstly, we discuss a set of design constraints that ease spectral optimization of a differentiable FM synthesizer via a standard reconstruction loss. Next, we present Differentiable DX7 (DDX7), a lightweight architecture for neural FM resynthesis of musical instrument sounds in terms of a compact set of parameters. We train the model on instrument samples extracted from the URMP dataset, and quantitatively demonstrate its comparable audio quality against selected benchmarks.

pela participação na banca examinadora e pelos comentários para a melhoria do trabalho. Ao Prof. Dr. Florivaldo Menezes Filho, pela participação na banca examinadora e pelos comentários para a melhoria do trabalho. Ao Prof. Dr. Raul Thomaz Oliveira do Valle, pela participação na banca examinadora e apoio e apoio durante a preparação deste trabalho. Ao Prof. Dr. Fernando Von Zuben, pelos comentários e sugestões para a melhorias deste trabalho. Ao NICS-Núcleo Interdisciplinar de Comunicações Sonoras, pelo apoio institucional ao longo do desenvolvimento deste trabalho. Ao CNPq, como órgão financiador da parte inicial da pesquisa. Bolsa Sanduíche durante o período de 05/02/1996 a 30/06/1997. V Índice Geral Capitulo 1: Introdução 1.1 A criação de uma síntese evolutiva de segmentos sonoros 1.2 A evolução histórica das sínteses sonoras tradicionais 1.3 A síntese evolutiva de segmentos sonoros 1.4 Organização da tese Capitulo 2: Descrição do método da síntese evolutiva 2.1 Introdução 2.2 A manipulação evolutiva dos segmentos sonoros 2.3 Os operadores genéticos 2.3.1 Operador Crossover 2.3.2 Operador Mutação 2.4 A medida da adequação do som 2.4.1 Distância vetorial 2.4.2 Distância de Hausdorff Capitulo 3: Utilização de curvas psicoacústicas como critério de adequação na síntese evolutiva 3.1 Introdução 3.2 Do segmento sonoro às curvas psicoacústicas 3.3 A extração das curvas psicoacústicas do indivíduo 3.4 A medida da adequação do som através das curvas psicoacústicas 3.4.1 Distância de Hausdorff 3.5 As operações genéticas sobre as curvas psicoacústicas 3.5.1 Operador Crossover 3.5.2 Operador Mutação 3.6 Construção do novo indivíduo a partir do genótipo modificado 3.7 A síntese evolutiva baseada na manipulação de curvas psicoacústicas VI Capítulo 4: Simulação dos modelos de síntese evolutiva 4.1 Cálculo do indivíduo como segmento sonoro 4.2 Cálculo das curvas psicoacústicas do indivíduo 4.3 A operação genética sobre o indivíduo 4.4 A operação genética sobre as curvas psicoacústicas do indivíduo 4.5 A medida da distância entre o indivíduo e o conjunto alvo 4.6 A medida da distância entre as curvas psicoacústicas do indivíduo 4.7 A construção do novo indivíduo a partir da variação das curvas psicoacústicas 4.8 Simulação do método da síntese evolutiva sobre o indivíduo 4.9 Simulação da síntese evolutiva utilizando as curvas psicoacústicas como genótipo do indivíduo Capitulo 5: Conclusões e comentários finais 5.1 Resultados da síntese evolutiva

In recent years, there have been numerous implementations of algorithmic systems that model the phenomena of emergent behaviour. In an attempt to reproduce the necessary conditions for emergent behaviour to occur, these systems employ bottom-up strategies through the ...

We are interested in developing intelligent systems for music composition. In this paper we focus on our research into generative rhythms. We have adopted an Artificial Life (A-Life) approach to intelligent systems design in order to develop generative algorithms inspired by the notion of music as social phenomena that emerge from the overall behaviour of interacting autonomous software agents. Whereas most A-Life approaches to implementing computer music systems are chiefly based on algorithms inspired by biological evolution (for example, Genetic Algorithms [2]), this work is based on cultural development (for example, Imitation Games [12, 13]). We are developing a number of such "cultural" algorithms, one of which is introduced in this paper: the popularity algorithm. We also are developing a number of analysis methods to study the behaviour of the agents. In our experiments with the popularity algorithm we observed the emergence of coherent repertoires of rhythms across the agents in the society.

COVID-19 is transmitted primarily via respiratory droplets and aerosols. Early in the pandemic, concerns were raised that wind instruments could increase the risk of infection. Therefore, several studies investigated the droplet and aerosol emissions from wind instruments. While wind instruments don't emit droplets, they still produce aerosols. As a countermeasure, aerosol filters for brass instruments were proposed. This study investigates whether such filters affect the timbre and radiation patterns of the filtered instrument. Four brass and one woodwind instrument (flute, French horn, trombone, trumpet, tuba) were recorded in three registers (low/mid/high), at two dynamic levels (p/f), and in three filter conditions (0/1/2 layers). The instruments were played by musicians and recorded on stage to approximate concert conditions. The radiation patterns were captured with an acoustic camera. Subjects rated the timbre, intonation, and articulation of the recorded sounds. The ratings were compared using a paired t-test. Low-level audio features were extracted and analyzed to identify the acoustic properties affected by the filter. One-layered filters had no influence on the subjective ratings. With two layers, the difference from the zerofilter condition was significant. Overall, the impact on intonation was considerably greater than on timbre. Differences in the radiation patterns were not found.

John von Neumann first presented his theory of machine self-reproduction in the late 40's [3], in which he described a machine capable of performing the logical steps necessary to accommodate selfreproduction and provided an explanation in principle for how arbitrarily complex machines can construct other ("offspring") machine of equal or even greater complexity. This project implements the von Neumann architecture for self-reproduction within the pre-existing evolutionary platform of Tierra, which implements a (mutable) genotype-phenotype mapping during reproduction. Initially, the memory image of the automaton's genotype and phenotype are physically identical, and each symbol in memory may be interpreted as either as passive numerical data (g-symbol), or a functional instruction (p-symbol) depending on how the symbol is interpreted. If redundancy is introduced to a mutable genotypephenotype mapping, the mapping system becomes non-invertible, rendering it impossible to compute an automaton's exact genotypic memory image by analysis of the phenotype alone. However, this non-invertible mapping may allow for a more robust genotype, increasing its robustness to fatal mutations and therefore increasing its ability to preserve its phenotypic form under perturbations.

In this paper we propose the use of an interactive multi-agent system for the study of rhythm evolution. The aim of the model proposed here is to show to what extent new rhythms emerge from both the interaction between autonomous agents, and self-organisation of internal rhythmic representations. The agents' architecture includes connectionist models to process rhythmic information, by extracting, representing and classifying their compositional patterns. The internal models of the agents are then explained and tested. This architecture was developed to explore the evolution of rhythms in a society of virtual agents based upon imitation games, inspired by research on Language evolution.

This paper reports the first results of an innovative approach to modelling music cognition based on the emergent behaviour of interacting autonomous systems. A group of interactive autonomous singing robots were programmed to develop a shared repertoire of songs from scratch, after a period of spontaneous creations, adjustments and memory reinforcements. The robots interact with each other by means of vocal-like sounds. They use real sounds as opposed to software simulation. They are furnished with a physical model of the vocal tract, which synthesises vocal singing-like intonations, and a listening mechanism, which extracts pitch sequences from audio signals. The robots learn to imitate each other by babbling heard intonation patterns in order to evolve vectors of motor control parameters to synthesise the imitations. Models of the basic mechanisms underlying the emergence of songs are of great interest for musicians looking for hitherto unexplored ways to create music with interactive machines.

Abstract. Our research is aimed at investigating the genesis and development of musical styles in artificial worlds. Focusing on the analysis of piano improvisation, we have designed and implemented a computer system (Interactive Musical Environments - iMe) with which we analyse ...

Celerina is the software core of a realtime system for dynamic music generation. Several one-dimensional binary cellular automata generate melodic patterns that are subsequently reduced and processed to form musical motifs and gestures. The music generated by Celerina is set to conform with such musical styles as jazz, classical or ambient music.

This article addresses aspects of an ongoing project in the generation of artificial Persian (-like) music. Liquid Persian Music software (LPM) is a cellular automata based audio generator. In this paper LPM is discussed from the view point of future potentials of algorithmic composition and creativity. Liquid Persian Music is a creative tool, enabling exploration of emergent audio through new dimensions of music composition. Various configurations of the system produce different voices which resemble musical motives in many respects. Aesthetical measurements are determined by Zipf’s law in an evolutionary environment. Arranging these voices together for producing a musical corpus can be considered as a search problem in the LPM outputs space of musical possibilities. On this account, the issues toward defining the search space for LPM is studied throughout this paper.

The research described here reflects our initial experiments in producing synthetic traditional Persian Music. Liquid Persian Music (LPM) is an assisting software capable of producing audio output. Equipped with cellular automata (CA) as a creative computational intelligence model and pattern recognition rules, it can produce novel musical material. This enables the exploration of new dimensions of music. Navigating in the LPM musical search space can be considered as a problem suitable for evolutionary algorithms. To help constraining the search space to desired musical classes, a fitness function based on a machine learning tool has been introduced to the problem. The fitness function is a support vector machine trained with features extracted from Persian music and LPM random sequences. The features are based on Zipf’s law proportions as one of the measurable universals for characterizing the aesthetical aspects occurring in natural phenomena. The search is guided towards followi...

This paper describes a compositional, extensible framework for music composition and a user study to systematically evaluate its core components. These components include a graph traversal-based chord sequence generator, a search-based melody generator and a pattern-based accompaniment generator. An important contribution of this paper is the melody generator which uses a novel evolutionary technique combining FI-2POP and multi-objective optimization. A participant-based evaluation overwhelmingly confirms that all current components of the framework combine effectively to create harmonious, pleasant and interesting compositions.

This paper introduces an interactive system called Graph-Cuisine that lets users steer an Evolutionary Algorithm (EA) to create random graphs that match user-specified measures. Generating random graphs with particular characteristics is crucial for evaluating graph algorithms, layouts and visualization techniques. Current random graph generators provide limited control of the final characteristics of the graphs they generate. The situation is even harder when one wants to generate random graphs similar to a given one, all-in-all leading to a long iterative process that involves several steps of random graph generation, parameter changes, and visual inspection. Our system follows an approach based on interactive evolutionary computation. Fitting generator parameters to create graphs with pre-defined measures is an optimization problem, while assessing the quality of the resulting graphs often involves human subjective judgment. In this paper we describe the graph generation process from a user's perspective, provide details about our evolutionary algorithm, and demonstrate how GraphCuisine is employed to generate graphs that mimic a given real-world network. An interactive demo of GraphCuisine can be found on our website http://www.aviz.fr/Research/Graphcuisine.

This paper describes the design, implementation, and application of an audio-driven computer-based system for live and interactive performance with an orchestra. We begin by describing the compositional and musical goals, emphasizing the electronics aspect. We then discuss the challenges of working with an orchestra and integrating the electronics in a concert hall. We describe the hardware setup and then detail the software implementation, from audio analysis and "perceptual parameter" estimation to the generative algorithm.

Genesynth is an analysis/synthesis framework that uses a genetic algorithm to search for a noise band sound model. The framework is written as an open source C++ tool, which allows for both the modification and resynthesis of found sound models and also genealogical exploration of the generations of sound variations created as a side effect of the genetic algorithm. The genetic algorithm used is specialized for the problem of audio analysis by using variable chromosome length as well as hierarchical chromosome structure. The algorithm's fitness function compares cached and compressed FFT data against estimated noise band models represented in the chromosomes. The noise band model is synthesized using sinusoids that are stochastically modulated in frequency to achieve a flexible bandwidth.

Artificial Intelligence (AI) algorithms have been extensively employed for generating art in many forms. From image generation to sound design, the self-emergence of structures in such algorithms make them suitable for exploring forms of computational creativity in automated generation of art. This work explores sound synthesis by combining swarm intelligence, user interaction and a novel sonic communication protocol between socially-capable artificial agents. These are termed "Sonoids" and socially behave following the well-known boids algorithm but perceive their environment (positions, velocities and identities of other agents) through sound. For the purposes of the work, the overall sound-synthesis environment is demonstrated as an iOS application that handles the Sonoids movement and sonifies their social interaction. User interaction is additionally allowed, which can modify specific parameters of the sonic communication protocol, leading to rich sonifications that present self-emergent structure.

This paper introduces the performance-based research led by the author in the area of collective decision-making and musical metacreative performances with audience participation. A series of six experiments (or interactive music performances) has been created in order to test alternatives to the implementation of features in metacreative systems aiming to foster engagement and improve interaction between performers and audience. This paper describes the evolution of the solutions sequentially adopted in these experiments with regard to control data, sound generation and user experience.

This paper describes a project to create a software instrument using a biological model of neuron behavior for audio synthesis. The translation of the model to a usable audio synthesis process is described, and a piece for laptop orchestra created using the instrument is discussed.

A multi-agent system is presented which generates melody pitch sequences with a hierarchical structure. The system has no explicit melodic intelligence and generates the pitches as a result of emotional influence and communication between agents, and the hierarchical structure is a result of the emerging agent social structure. Another key element is that the system is not a mapping from multi-agent interaction onto musical features, but actually utilizes music for the agents to communicate emotions. Each agent in the society learns its own growing tune during the interaction process.

FM Synthesis is a well-known algorithm used to generate complex timbre from a compact set of design primitives. Typically featuring a MIDI interface, it is usually impractical to control it from an audio source. On the other hand, Differentiable Digital Signal Processing (DDSP) has enabled nuanced audio rendering by Deep Neural Networks (DNNs) that learn to control differentiable synthesis layers from arbitrary sound inputs. The training process involves a corpus of audio for supervision, and spectral reconstruction loss functions. Such functions, while being great to match spectral amplitudes, present a lack of pitch direction which can hinder the joint optimization of the parameters of FM synthesizers. In this paper, we take steps towards enabling continuous control of a well-established FM synthesis architecture from an audio input. Firstly, we discuss a set of design constraints that ease spectral optimization of a differentiable FM synthesizer via a standard reconstruction loss. Next, we present Differentiable DX7 (DDX7), a lightweight architecture for neural FM resynthesis of musical instrument sounds in terms of a compact set of parameters. We train the model on instrument samples extracted from the URMP dataset, and quantitatively demonstrate its comparable audio quality against selected benchmarks.

Tone Transfer is a novel deep-learning technique for interfacing a sound source with a synthesizer, transforming the timbre of audio excerpts while keeping their musical form content. Due to its good audio quality results and continuous controllability, it has been recently applied in several audio processing tools. Nevertheless, it still presents several shortcomings related to poor sound diversity, and limited transient and dynamic rendering, which we believe hinder its possibilities of articulation and phrasing in a real-time performance context. In this work, we present a discussion on current Tone Transfer architectures for the task of controlling synthetic audio with musical instruments and discuss their challenges in allowing expressive performances. Next, we introduce Envelope Learning, a novel method for designing Tone Transfer architectures that map musical events using a training objective at the synthesis parameter level. Our technique can render note beginnings and endings accurately and for a variety of sounds; these are essential steps for improving musical articulation, phrasing, and sound diversity with Tone Transfer. Finally, we implement a VST plugin for real-time live use and discuss possibilities for improvement. CCS CONCEPTS • Applied computing → Sound and music computing; • Computing methodologies → Neural networks; • Human-centered computing → Interaction techniques.