Visualizing Music

How does a musician's system of coding influence their sonic perception? As part of a greater research project involving Greek musicians (Athanasopoulos and Kitsios, 2015) we asked participants to depict a short excerpt of Western 'art' music and a piece of traditional Greek music as images in a free-drawn design, adopting the method used successfully by , in which participants heard brief musical stimuli and were asked to represent them graphically on paper. The participants shared the same store of symbols (literate in Greek and musically literate in western standard notation and/or tablatures) and had comparable sociocultural norms due to their upbringing. They were selected based on these similarities in order to ensure that any cases of Codal Incompetence or Codal Interference would be primarily the result of their variable musical performance background, which was European classical, or mixed traditional and classical. Without establishing a hypothesis as to the results, our purpose was to examine whether this difference would influence the method in which they would code their responses, with a particular interest in noting whether the resulting visualisations could be classified as scores of either prescriptive or descriptive nature .

P. Muñoz Lago, N. Usula, E. Parada-Cabaleiro, A. Torrente, “Visualising the Structure of 18th Century Operas: A multidisciplinary Data Science Approach,” Proceedings of the 24th International Conference on Information Visualisation (2020-IV) 508-514., 2020

Data visualisation is an effective strategy to communicate information. From a multidisciplinary perspective, applying this methodology (typical of computer science) to humanistic purposes (such as visualising musical, narrative and dramaturgical structure) has shown very promising results. Opera is a complex form of art, whose structure is strongly influenced by dramaturgical and musical aspects, both present in all operatic librettos. Although visualisation methods have been successfully applied in the understanding of music and narrative individually, a combined approach, aimed to jointly visualise both aspects together-by this enhancing opera comprehension-has not yet been developed. With this in mind, through a cooperative methodology of musicology and computer science disciplines, we carry out a data science project aimed to graphically represent the structure of 18 th century operas' libretto structure. The presented approach, based on XML librettos from the Progetto Metastasio, automatically generates a comprehensive graphical representation of the opera structure, based upon musical and dramaturgical information. The schemes developed in this work show all the elements relevant to the 18 th century opera structure, and graphically synthesise its complex shape, in order to encourage opera understanding across a large set of users, from general listeners to musicians and finally to musicologists.

2017

Standard western notation supports the understanding and performance of music, but has limited provisions for revealing overall musical characteristics and structure. This paper presents several visualisers for highlighting and providing insights into musical structures, including rhythm, pitch, and interval transitions, also noting how these elements modulate over time. The visualisations are presented in the context of Shneiderman’s Visual Information-Seeking Mantra, and terminology from the Cognitive Dimensions of Music Notations usability framework. Such techniques are designed to make understanding musical structure quicker, easier, less error prone, and take better advantage of the intrinsic pattern recognition abilities of humans.

2022

Medieval methods to visualize music developed in dialogue with geometrical, arithmetical, cosmological, grammatical, philosophical, logical, and theological knowledge. Located between the sensory and the imaginative, the quadrivium of musica, geometry, arithmetic, and cosmology was nonetheless grounded in the realm of the visual. Visualizations of musica appear in manuscripts as diagrams, graphs, and line drawings using parchment, ink, and pigments. While Art History as a scholarly discipline only evolved in Modernity, music theory – rooted in Antiquity – took a long turn from being part of the quadrivial disciplines in the Middle Ages to becoming associated with musical practice, in which notational systems started to overshadow diagrammatic strategies of visualizations. Going beyond Lessing’s Laokoon, my talk draws from recent art historical approaches to medieval and especially cosmological diagrams and aims to question what facilitated the neglect of musical material within the field of Art History, how this history is entangled with the history of musica – and what that means for us today.

2020 24th International Conference Information Visualisation (IV), 2020

Some complex musical parameters might be especially difficult to understand for someone with no theoretical expertise in music. Musicians and music scholars alike normally evaluate such parameters visually by departing from scores, which present the musical events at once. Yet for the understanding of such symbolic representations, musical training is essential, making scores mostly incomprehensible for amateurs. Data visualisation has been applied to meaningfully represent complex musical parameters, thus enabling music amateurs to grasp concepts such as texture or structure. Although scores are one of the "primary" sources to understand music, previous work shows a strong bias towards the visualisation of acoustic data, in detriment of the visualisation of symbolic information. To bridge the gap, we present SymPlot, a web-based open source tool to automatically visualise textural density, scoring, and structure from MusicXML files. Due to the multidisciplinary nature of the topic, in this project we have applied the Scrum's agile methodology, an iterative incremental approach specifically tailored for interdisciplinary projects. The tool, aimed at enhancing musical understanding in amateurs and students, as well as in scholars of other disciplines who need to incorporate music into their discourses, i.e., historians, philologists, etc., enables visualisation of local features at various hierarchical levels, highlighting similarities both within and across scores. Our evaluation of SymPlot-based on a five-level rating-scale test performed by 50 participants-suggests that colours increase users' understanding of complex musical parameters.

Presence: Teleoperators and Virtual Environments

The EvoluSon project proposes an immersive experience where the spectator explores an interactive visual and musical representation of the main periods of the history of Western music. The musical content is constituted of original musical compositions based on the theme of Bach’s Art of Fugue to illustrate the eight main musical eras from Antiquity to the contemporary epoch. The EvoluSon project contributes at the same time to the usage of VR for intangible culture representation and to interactive digital art that puts the user at the center of the experience. The EvoluSon project focuses on music through a presentation of the history of Western music, and uses virtual reality to valorize the different pieces through the ages. The user is immersed in a coherent visual and sound environment and can interact with both modalities. This project is the result of collaboration between a computer science research laboratory and a research laboratory on art and music. It was first present...

Isochords is a visualization of music that aids in the classification of musical structure. The Isochords visualization highlights the consonant intervals between notes and common chords in music. It conveys information about interval quality, chord quality, and the chord progression synchronously during playback of digital music. Isochords offers listeners a means to grasp the underlying structure of music that, without extensive training, would otherwise remain unobserved or unnoticed. In this paper we present the theory of the Isochords structure, the visualization, and comments from novice and experienced users.

• Illo Humphrey, PhD-HDR | CNU-18 | Iconography as a source for Music History • RMA Study Days-8 & 9-XI-2019 | SOAS – University of London (SOAS) | under the direction of Patrick Huang (SOAS) & Susan Bagust (RMA) | Report of Day 2: November 9, 2019 | Academia.edu | 22-XI-2020 • • ¶1 Introduction: The Royal Music Association Research Study Days – 2019, entitled Iconography as a source for Music History, were organised by Patrick Huang (SOAS) and Susan Bagust (RMA) at the School of Oriental and African Studies [SOAS], University of London. These Study Days explored what one may legitimately call iconographic proto-philology – that is to say, a composite approach and methodology, allowing one to determine and to identify by critical deduction the invisible ties between the intangible and tangible elements of a given research, and thereby to formulate well-founded hypotheses, and to arrive at sound conclusions concerning studies in Iconographia. Within the context of the RMA Study Days – 2019, iconographic proto-philology is by definition a research method which takes into account the interplay of various aspects of general culture inherent in an iconographic study, namely: music, musicology, organology (the history and study of the classification and manufacture of musical instruments), mathematics, geometry, astronomy, liturgy, chromatology, chemistry, terracotta, ceramics, ceramic painting, black-figure pottery, mural painting, frescoes, metallurgy, sculpture, archæology, architecture, palæography, semiology, codicology (the study of manuscript making), secular and biblical history, art history, literature, poetry, philosophy, political science, sociology, etc., and, at the same time, reconstitutes the missing links in a given chain of events (linguistic, literary, poetic, artistic, musical, etc.), making this composite approach of iconographic proto-philology a most reliable and valuable research tool in studying the sources of music history presented during the RMA Study Days – 2019 organised by the University of London (Institute of Musical Research [IMR]) at the School of Oriental and African Studies. • ¶2 This report will limit itself to the presentations of Panels 4, 5, and 6 of the RMA Study Days – 2019, held on Saturday, the 9th of November at the School of Oriental and African Studies (SOAS) of the University of London, and will feature 8 lectures scheduled on the final program, namely numbers 11, 13 through 19, presented respectively by: (11) Konstantinos Karagounis & Zoe Naoum (Volos Academy for theological Studies, Thessalias, Volos, Greece): “Musical aspects on the works of Greek folk painter Theofilos Hatzimichael ” • (13) Richard J. Dumbrill (Royal Hollaway, University of London | ICONEA): “Does seal TH 95-35 suggest orchestral performance? ” • (14) James Lloyd (University of Reading, UK): “Greek Black-Figure Pottery (590 – 490 BCE): Images of regional music” • (15) Claudina Romero Mayorga (University of Reading, UK): “The Sound of Silence: Harpocrates and the role of music in Greco-Roman cults” • (16) Niroshini Senevirachne (University of Peradeniya, Peradeniya, Sri Lanka): “Descriptions of Women Musicians in Ancient Sri Lankan Temple Frescoes (with special reference to Mulkirigala Temple)” • (17) Manoj Alawathukotuwa (University of Peradeniya, Peradeniya, Sri Lanka): “Depiction of Musical Instruments, Social Status Gender of Musicians through Temple Paintings of Sri Lanka” • (18) Fueanglada “Organ” Prawang (Bangor University, UK): “The reverence of Giants and the challenge it creates for performing Thai opera” • (19) Illo Humphrey (University of Bordeaux Montaigne, 33607 Pessac [Bordeaux], France): “Observations on the elements of music and philosophy in the Carolingian illumination David rex et prop[heta], frontispiece of the Book of Psalms in the monumental Bible in-folio known as the 1st Bible of Charles the Bald (*823-†877), conserved in the codex Paris, BnF, Fonds latin 1, f. 215v, written in the 9thcentury between 844-851at the scriptorium of Saint-Martin of Tours” • • ¶3 The present report passes in review the 8 presentations, indicating respectively: the name and position of the lecturer, the title of the lecture with pertinent iconographic examples, when possible, a short synthesis of the lecture with key words, key names, key concepts, and for each lecture a brief bibliography. • Nota bene: Lecture Nr. 12 “You can take the Rat out of the Ghetto…Urban Art and its Journey from Street to Gallery”, scheduled to be given by Dr. Debra Pring on 9-X-2019, was not presented. • ¶4 Conclusion: The RMA 2019 Study Days, Iconography as a source for Music History, revealed the emergence of what one may legitimately call iconographic proto-philology – that is to say, a new approach and a new methodology of image analysis that was spontaneously applied in the majority of the lectures presented. The main theme of the RMA Study Days – 2019, suggested implicitly the fusion of multiple disciplines, such as: the philosophy of image, the science of colours, dyes, and pigments, the philosophy of music, musicology, organology, the philosophy of numbers and proportions, the philosophy of the written word, the philosophy of education, the philosophy of culture, the philosophy of language, the philosophy of politics, the philosophy of ethics, the philosophy gender studies, the protection and safeguard of the tangible and intangible cultural heritage and cultural diversity of humanity, etc. This theme gave rise to interesting scientific, historical, and socio-cultural analyses, hypotheses, and subsequent discussions, based on a combination of fundamental and universal concepts, namely: the genesis of the soul-consciousness, the cognitive process, sense perception, musical harmony based on the regime of the octave, musical and vocal sound, the essence of number, measured structure and form, educational and cultural diversity, the sevenfold canon of the Liberal Arts, politics – that is to say, the governance and the management of the State and its citizens, the Sovereign divine and human Good: including both theology and ethics-morality, and of course, the Carolingian renaissance, Charlemagne’s Capitularium XXII Admonitio generalis, and the concept of the Unity of culture in the building of civilisations, etc. Indeed, this coherent set of fundamental concepts, most of which were discussed in the lectures of Panels 4, 5, and 6, are among the conducting threads in all civilisation management and civilisation development, observable in the Tradition of Knowledge of the East and of the West, of the Southern and the Northern hemispheres, from China to Thailand, Thailand to India, India to Sri Lanka, Sri Lanka to Babylonia, Babylonia to Egypt, Egypt to Timbuktu, Egypt to Greece, Greece to Rome, and from Rome to Great Britain, etc. • ¶5 Keeping this in mind, the RMA Study Days – 2019, organized under the auspices of the University of London (SOAS), show once again that the identification of invisible ties between the intangible and tangible elements of iconographic research by critical deduction is altogether possible, and that this new approach of iconographic proto-philology can engender well-founded hypotheses and sound scientific conclusions. • ¶6 In behalf then of all the participants, we wish to express our full gratitude to the Royal Musical Association, and especially to the outstanding team of organisers, namely: Patrick Huang, Susan Bagust and their colleagues, for this very stimulating 2-day symposium held on the 8th and 9th-X-2019, featuring very diverse aspects of Iconography as a source for Music History. • ¶7 In fine, it is useful to remind ourselves that the cultural and educational reforms of the Carolingian renaissance were impulsed by the cultural legislation of Charlemagne, namely the Capitularium XXII entitled Admonitio generalis, cf. article 72: “Sacerdotibus” (“To the attention of the Clergy”). This important legislation, drafted, so it seems, by Charlemagne’s remarkable minister of Education and Culture, Alcuin of York, was promulgated die lunae, decimo kalendas aprilis, anno Domini, septingentesimo octogesimo nono – that is to say, Monday, the 10th day before the Calends of April, i.e. the 23rd of March, Year of the Lord, 789, and since that historical date, the Admonitio generalis, which stimulated greatly the development of the Fine Arts in the 9th-century– thus making possible the creation of the illumination David rex et prop[heta], has never ceased to be an important source of the European Unity of Culture and Education • Explicit • • © Illo Humphrey, PhD-HDR-CNU-18 | Directeur-Fondateur des Colloquia Aquitana • • Mediævalist – Musicologist – Proto-Philologist – Concert-Baritone – Trilingual simultaneous Interpreter • • scripsi et subscripsi • • Die dominico• decimo kalendas decembres• anno Domini intercalario ED• bis millesimo vicesimo • • IH | ih | PhD-HDR | scripsi et subscripsi •