Audio Matching via Chroma-Based Statistical Features

2010

We investigate the problem of matching symbolic representations directly to audio based representations for applications that use data from both domains. One such application is score alignment, which aligns a sequence of frames based on features such as chroma vectors and distance functions such as Euclidean distance. Good representations are critical, yet current systems use ad hoc constructions such as the chromagram that have been shown to work quite well. We investigate ways to learn chromagram-like representations that optimize the classification of “matching” vs. “non-matching” frame pairs of audio and MIDI. New representations learned automatically from examples not only perform better than the chromagram representation but they also reveal interesting projection structures that differ distinctly from the traditional chromagram.

Proceedings of the International …, 2007

The analysis of electroacoustic music may benefit from the design and development of tools for automatic audio matching, allowing musicologists to compare different versions of the same work and to draw the artistic path from the synthesis of fragments to the creation of a complete work. Music alignment has been explored as a tool in many areas, from score following to music information retrieval. This paper presents a tool for the analysis of electroacoustic music based on the comparison of two audio sources. The tool has been applied to a relevant case study: Stria by John Chowning.

IEEE/ACM transactions on audio, speech, and language processing, 2021

This paper deals with a score-audio music retrieval task where the aim is to find relevant audio recordings of Western classical music, given a short monophonic musical theme in symbolic notation as a query. Strategies for comparing score and audio data are often based on a common mid-level representation, such as chroma features, which capture melodic and harmonic properties. Recent studies demonstrated the effectiveness of neural networks that learn task-specific mid-level representations. Usually, such supervised learning approaches require score-audio pairs where the score's individual note events are aligned to the corresponding time positions of the audio excerpt. However, in practice, it is tedious to generate such strongly aligned training pairs. As one contribution, we show how to apply the Connectionist Temporal Classification (CTC) loss in the training procedure, which only uses weakly aligned training pairs. In such a pair, only the time positions of the beginning and end of a theme occurrence are annotated in an audio recording, rather than requiring local alignment annotations. We evaluate the resulting features in our theme retrieval scenario and show that they improve the state of the art for this task. As a main result, we demonstrate that with the CTC-based training procedure using weakly annotated data, we can achieve results almost as good as with strongly annotated data. Furthermore, we assess our chroma features in depth by inspecting their temporal smoothness or granularity as an important property and by analyzing the impact of different degrees of musical complexity on the features.

Applications of Signal …, 2003

We describe a method that aligns polyphonic audio recordings of music to symbolic score information in standard MIDI files without the difficult process of polyphonic transcription. By using this method, we can search through a MIDI database to find the MIDI file corresponding to a polyphonic audio recording. 0-7803-7850-4/03/$17.00

While recent developments in MIR have proven automatic analysis of content-based similarity to be reliable, it remains unclear whether these algorithms can be meaningfully applied to cross-cultural analyses of more diverse samples. There is thus a need to evaluate automatic methods with perceptual ground truth data. Here we conducted multiple perceptual ratings tests on a subset of the Cantometrics recordings across 62 participants. We compared evaluating perceptual similarity by pairwise comparison, an odd-one-out method and a simplified feature evaluation based on 6 Cantometric features (namely, Ornamentation, Vocal Range, Tempo, Rhythmic Regularity, Vocal Tension, and Vocal Texture). Preliminary analysis shows that while the same person may disagree with themselves depending on the evaluation method, groups in general tend to agree with each other for the same method. We then compare the perceptual similarity ratings against existing automated algorithms and find minimal correlation, suggesting that there is still much room for improvement in automated cross-cultural content-based similarity.

One major goal of structural analysis of an audio recording is to automatically extract the repetitive structure or, more generally, the musical form of the underlying piece of music. Recent approaches to this problem work well for music where the repetitions largely agree with respect to instrumen-tation and tempo, as is typically the case for popular music. For other classes of music such as Western classical music, however, musically similar audio segments may exhibit significant variations in parameters such as dynamics, timbre, execution of note groups, modulation, articulation, and tempo progression. In this paper, we propose a robust and efficient algorithm for audio structure analysis, which allows to identify musically similar segments even in the presence of large variations in these parameters. To account for such variations, our main idea is to incorporate invariance at various levels simultaneously: we design a new type of statistical features to absorb micro-variations, introduce an enhanced local distance measure to account for local variations, and describe a new strategy for structure extraction that can cope with the global variations. Our experimental results with classical and popular music show that our algorithm performs successfully even in the presence of significant musical variations.

ACM Multimedia Conference, 2000

With the growth in digital representations of music, and of music stored in these representations, it is increasingly attractive to search collections of music. One mode of search is by similarity, but, for music, similarity search presents several difficulties: in particular, deciding what part of the music is likely to be perceived as the theme by a listener, and deciding whether two pieces of music with different sequences of notes represent the same theme. In this paper we propose a three-stage framework for matching pieces of music. We use the framework to compare a range of techniques for determining whether two pieces of music are similar, by experimentally testing their ability to retrieve different transcriptions of the same piece of music from a large collection of MIDI files. These experiments show that different comparison techniques differ widely in their effectiveness; and that, by instantiating the framework with appropriate music manipulation and comparison techniques, pieces of music that match a query can be identified in a large collection.

This paper addresses the design of feature functions for the matching of a musical recording to the symbolic representation of the piece (the score). These feature functions are defined as dissimilarity measures between the audio observations and template vectors corresponding to the score. By expressing the template construction as a linear mapping from the symbolic to the audio representation, one can learn the feature functions by optimizing the linear transformation. In this paper, we explore two different learning strategies. The first one uses a best-fit criterion (minimum divergence), while the second one exploits a discriminative framework based on a Conditional Random Fields model (maximum likelihood criterion). We evaluate the influence of the feature functions in an audio-to-score alignment task, on a large database of popular and classical polyphonic music. The results show that with several types of models, using different temporal constraints, the learned mappings have the potential to outperform the classic heuristic mappings. Several representations of the audio observations, along with several distance functions are compared in this alignment task. Our experiments elect the symmetric Kullback-Leibler divergence. Moreover, both the spectrogram and a CQT-based representation turn out to provide very accurate alignments, detecting more than 97% of the onsets with a precision of 100 ms with our most complex system. Index Terms-Music information retrieval, audio-to-score alignment, conditional random fields, discriminative learning.

2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2011

A key processing step in music-to-score alignment systems is the estimation of the intantaneous match between an audio observation and the score. We here propose a general formulation of this matching measure, using a linear transformation from the symbolic domain to any time-frequency representation of the audio. We investigate the learning of the mapping for several common audio representations, based on a best-fit criterion.

Proc. ISMIR, 2006

Identifying versions of the same song by means of automatically extracted audio features is a complex task for a music information retrieval system, even though it may seem very simple for a human listener. The design of a system to perform this task gives the opportunity to analyze which features are relevant for music similarity. This paper focuses on the analysis of tonal similarity and its application to the identification of different versions of the same piece. This work formulates the situations where a song is versioned and several musical aspects are transformed with respect to the canonical version. A quantitative evaluation is made using tonal descriptors, including chroma representations and tonality. A simple similarity measure, based on Dynamic Time Warping over transposed chroma features, yields around 55% accuracy, which exceeds by far the expected random baseline rate.