When you listen to a Beethoven piano concerto, pick a song or channel to listen to on the web, send your kids to music lessons, or donate to your local symphony orchestra, do you think of data science? Unlikely. But data science is increasingly critical in the creative process and commercial activities of music. Using Big Data analytics, online music platforms tailor individualized song recommendations for their users; music theorists search for defining features in different types of music; computer scientists develop algorithms that can compose music in any style; behavioral scientists determine how to best engage concert goers; the list is endless. The Michigan Institute for Data Science (MIDAS) has recently launched the Data Science for Music Research Initiative.  U-M has incredible depth in data science expertise and a world-class School of Music, Theatre and Dance. With this initiative, MIDAS hopes to help U-M scientists lead the nation in research at the intersection of data science and music.

Currently, this initiative funds four projects that form the Data Science for Music Hub:

  • Daniel Forger (Mathematics) and James Kibbie (Organ) lead the project “Understanding how the brain processes music through the Bach trio sonatas”, and will create a library of digitized performances of the Bach Trio Sonatas and analyze common features and errors in these performances.
  • Danai Koutra and Walter Lasecki (Electrical Engineering and Computer Science) lead the project “Understanding and mining patterns of audience engagement and creative collaboration in largescale crowdsourced music performances”, and will use data mining techniques to increase audience participation during live performances.
  • Rada Mihalcea (Electrical Engineering and Computer Science) and Anıl Çamcı (Performing Arts Technology) lead the project “The sound of text”, and will use neural network architectures to learn sequence-to-sequence mappings and to develop computerized, text-based, music composition.
  • Somangshu Mukherji (Music Theory) leads the project “A computational study of patterned melodic structures across musical cultures”, and will examine melodic structures in six different musical corpora using data-science methodologies and put U-M on the map as a pioneer in the emerging research discipline of empirical music theory.

For more on each project, click on the projects below, and see the press release announcing the funding.

Starting from this initial research, the MIDAS Data Science for Music Hub hopes to:

  • Build a collaborative network of Data Science for Music researchers, and help U-M stay at the national forefront in this research discipline.
  • Promote innovative research ideas, disseminate tools and methods and maximize research sustainability.
  • Form academic and industry partnerships and bring research findings to data science and music education and to the market.

If you are interested in finding out more about collaboration, partnership and resources, please contact us: midas-contact@umich.edu

Research Areas

Understanding How the Brain Processes Music through the Bach Trio Sonatas

The composer Varèse defined music as “organized sound,” yet much remains to be learned about how sound is organized to make music. This project takes a “big data” approach to understanding the patterns and principles of music with a focus on the Bach Trio Sonatas for organ.  Bach is the go-to composer for understanding the fundamentals of music.  His Trio Sonatas have been important pedagogical tools from Bach’s time to today in teaching performance, improvisation and composition.  The organ is the ideal instrument for performance digitization since the performer only controls the attack and release of each note: once a note is started, pitch and timbre remain the same.

Learn more…

A Computational Study of Patterned Melodic Structures across Musical Cultures

Musical compositions are generally made up of highly-patterned structures—pitch, rhythm, timbre, dynamics, and so on. The study of such structures using computational models, especially machine-assisted analyses of large musical corpora, is a rapidly growing field. Yet such work has tended both to confine itself to single musical styles and to ignore the rich conceptual legacy of music theory.  Music theorists has developed sophisticated descriptions of pitch and other aspects of patterned musical structure.  However, these conceptualizations have rarely been examined empirically, especially with computation methods.  In other words, there is a gap between theoretical and empirical studies of musical structure.

This research team will help fill this gap by bringing a renewed theoretical attention to empirical studies of musical structure, focusing on the melody along with pitch, voice, and voice leading.

Learn more…

Understanding and Mining Patterns of Audience Engagement and Creative Collaboration in Largescale Crowdsourced Music Performances

Modern mobile and web audio technologies enable large-scale audience participation in concerts in various ways.  For example, audience members can use interactive music applications to generate music from their smartphones as a connected ensemble and shape a live music performance. However, it remains an ongoing challenge to design interactions that encourage and sustain audience participation over time.  This research team will use data mining techniques over the time-evolving networks of audience collaboration to identify audience preferences and interaction patterns.   Such knowledge will be used to improve audience engagement in an audience participatory music performance.

Learn more…

The Sound of Text

If words were music, what would they sound like? What is the sound of the book you just read, the email you just sent, the news article you just received, or the research paper you just wrote?  Music and words come together in the millions of songs that delight us, and yet for most of the words in the world, their music is silent.  This research team will develop algorithms that leverage existing alignments between words and music to produce a musical interpretation for any text.

Learn more…