Robot Playing Music
Photo by Franck V. on Unsplash

Our research aims to discover if successful AI image generative models can also be used to generate music. We use a Generative Adversarial Network, a particular type of Artificial Neural Network, in an attempt to generate Irish music.

Although there are other neural network based Irish music generators, we found no GANs which successfully generated music. These other neural networks use a prediction based approach to generate music. Unlike them, GANs could truly generate music from scratch. GANs have worked well with pictures, but to our knowledge there has not been a successful attempt to generate music using GANs. Also, to our knowledge, no one has presented the idea of organizing music like a picture, which makes our encoding unique.

Irish Music

Man Playing an Irish Harp
Photo by Vicente Veras on Unsplash

Typically, when someone is talking about Irish music, they mean traditional Irish music, or Irish folk music. Defining Irish music can be difficult because of the vague definition of folk music, and determining the music's origin. Some define traditional music as "old songs, with no known to composers" or as a piece which has accumulated cultural value by being passed down through the generations.

We have three main reasons for choosing to use Irish music:

  1. Even with possible cultural influence, Irish music tends to be distinctive with a fairly simple rhythmic structure and a consistent form between pieces.
  2. Newfoundland culture has been heavily influenced by Irish immigration, as can be seen in the language and music.
  3. An abundance of Irish music exists online already in ABC Notation, which eliminates the need to transcribe music.

For our purposes, we effectively define Irish music as something that the typical person, with little to no musical training, would believe originated in Ireland many years ago, or, a tune that one would expect to be included with other traditional Irish tunes, possibly being played in a pub or a relaxed setting among a group of musicians. In this sense, indistinguishability of the generated tune from other genuine Irish pieces is the key element we're striving for in our output.

Generative Adversarial Network

Generative Adversarial Network Process
visualizing a generative adversarial network

An Artificial Neural Network, or ANN, is based on a collection of connected units called nodes, which can be loosely compared to neurons in the human brain. There are various types of ANNs, many of which have been successfully used to classify, predict, and generate different types of data. A Generative Adversarial Network, or a GAN, is an ANN consisting of a Generator that generates data when fed noise, and a Discriminator which classifies data. A common analogy is the Generator is akin to an art forger, while the Discriminator is like a detective. After being trained, the Generator can be used to create original ‘art’.

The GAN has been structured to take advantage of the features that have been observed in the image forms of the Irish tunes in the training dataset. We opted for a Deconvolutional GAN since they are renowned for performing well on images, and making training a lot more stable.

Our Approach

Vectorizing Process to Generate Music
our overall process of generating Irish music

We hope to use the distinctive structure of Irish tunes to make them suitable for GAN-based music generation. Our main idea is to regard an Irish melody as a fixed-size object with cross-references among its parts: “music as a picture” view. The two key components of this project are:

  1. Preprocessing:
    • Creating a well-defined format for music encoding suitable for GANs
    • Representing “vertical” dependencies in the tunes
  2. Modifying GANs for improved performance via:
    • Statistical “Bagging” and “Boosting” to mimic the full input distribution
    • Modifying training rate to avoid “Mode Collapse”


Check back here for more findings soon. Here are some images generated while coming up with our Music as a Picture view. Our code and interesting findings and results will be uploaded here as well as to an online repository for public access here.

Vectorized Images
visualizing music as a structured picture
Song Types
song types from our dataset
Dr. Antonina Kolokolova

Dr. Antonina Kolokolova

Antonina is an Associate Professor at the Department of Computer Science , Memorial University of Newfoundland. Her research interests are in theoretical computer science, in particular complexity theory and logic, including proof complexity

Robert Bishop

Robert Bishop

Robert is a Computer Science student at the Memorial University of Newfoundland.

Mitchell Billard

Mitchell Billard

Mitchell is a Computer Science student at the Memorial University of Newfoundland.

Moustafa Elsisy

Moustafa Elsisy

Moustafa is a Computer Science student at the Memorial University of Newfoundland.

Caleb Graves

Caleb Graves

Caleb is a Computer Science student at the Memorial University of Newfoundland.

Zachary Northcott

Zachary Northcott

Zachary is a Computer Science student at the Memorial University of Newfoundland.

Vineel Nagisetty

Vineel Nagisetty

Vineel is a Computer Science student at the Memorial University of Newfoundland.