Research Spotlight: Data Science Fellows Hackathon Uses Natural Language Processing to Analyze the Past 60 Years of Pop Music

Research Spotlight: Data Science Fellows Hackathon Uses Natural Language Processing to Analyze the Past 60 Years of Pop Music

When you listen to popular music have you ever picked up on recurring themes or trends in the lyrics? Have the attitudes or feelings expressed in music changed over time or largely stayed focused on the same subjects? For example, between the 1960’s and today has popular music taken a more negative note? How do popular music and our culture influence each other? A group of postdoctoral researchers, the Michigan Data Science  Fellows, were curious about this, so they came together over a weekend for a “Hackathon” and  analyzed pop music sentiment over time using data science techniques. 


 

 

 

 

 

 

 

The Fellows used natural language processing (NLP) methods to analyze both the lyrical content and positive or negative sentiment of popular music from the 1960’s to the present day to see if the lyrics have become more or less negative over time.

The Fellows retrieved song lyrics from Genius and ran them through a rule based “empath” algorithm and neural network model. The Empath algorithm has pre-existing categories of words that are known to belong to different topics and emotions: for example, the “suffering” list contains words like “sad” and “fear”. The Fellows then matched each song’s lyrics to these word categories and computed the proportion of each category that is represented by the lyrics. If 60% of a song’s lyrics belong to the “suffering” list, for example, the song is classified as a “suffering” song.

The initial results shown in the figures above seem to indicate that popular music has become more negative in sentiment over time. The Fellows plan to take their results from this weekend of work and publish a joint paper with their final analysis.

The MIDAS Data Science Fellows Program is one of the few data science postdoc programs in the country. Fellows come from diverse research fields, with data science and AI as their central methodology, and form extensive collaboration on campus. The program is part of MIDAS effort to strengthen the data science and AI research community on campus, especially to bring together postdocs who carry out data-intensive research.

Efrén Cruz Cortés, one of the postdocs, was motivated to organize the Hackathon in hopes of kickstarting a new tradition for the program. “We decided that we would all pitch projects we thought would be fun to investigate, and decide on one.” says Cruz Cortés, “The five of us who came to the Hackathon brought our skills and creativity together. Ian is an NLP expert, I’m a computer scientist studying algorithmic fairness, Dan develops control theories for AI systems, Ruby studies epidemiology, Maria is a mathematician and astronomer but this project was her idea. We learned so much about each other’s skills through this project and had a super fun weekend.” 

Learn more about MIDAS’ Data Science Fellows Program here.

Read the Story