GANs Reels

• Learn more
• Listen
• Generate
• Code Repository
• Who We Are

Motivation

Our research aims to discover if successful AI image generative models can also be used to generate music. We use a Generative Adversarial Network, a particular type of Artificial Neural Network, in an attempt to generate Irish music.

Although there are other neural network based Irish music generators, we found no GANs which successfully generated music. These other neural networks use a prediction based approach to generate music. Unlike them, GANs could truly generate music from scratch. GANs have worked well with pictures, but to our knowledge there has not been a successful attempt to generate music using GANs. Also, to our knowledge, no one has presented the idea of organizing music like a picture, which makes our encoding unique.

• Irish Music

Irish Music

Typically, when someone is talking about Irish music, they mean traditional Irish music, or Irish folk music. Defining Irish music can be difficult because of the vague definition of folk music, and determining the music's origin. Some define traditional music as "old songs, with no known to composers" or as a piece which has accumulated cultural value by being passed down through the generations.

We have three main reasons for choosing to use Irish music:

1. Even with possible cultural influence, Irish music tends to be distinctive with a fairly simple rhythmic structure and a consistent form between pieces.
2. Newfoundland culture has been heavily influenced by Irish immigration, as can be seen in the language and music.
3. An abundance of Irish music exists online already in ABC Notation, which eliminates the need to transcribe music.

For our purposes, we effectively define Irish music as something that the typical person, with little to no musical training, would believe originated in Ireland many years ago, or, a tune that one would expect to be included with other traditional Irish tunes, possibly being played in a pub or a relaxed setting among a group of musicians. In this sense, indistinguishability of the generated tune from other genuine Irish pieces is the key element we're striving for in our output.

• Neural Network

Generative Adversarial Network

An Artificial Neural Network, or ANN, is based on a collection of connected units called nodes, which can be loosely compared to neurons in the human brain. There are various types of ANNs, many of which have been successfully used to classify, predict, and generate different types of data. A Generative Adversarial Network, or a GAN, is an ANN consisting of a Generator that generates data when fed noise, and a Discriminator which classifies data. A common analogy is the Generator is akin to an art forger, while the Discriminator is like a detective. After being trained, the Generator can be used to create original ‘art’.

The GAN has been structured to take advantage of the features that have been observed in the image forms of the Irish tunes in the training dataset. We opted for a Deconvolutional GAN since they are renowned for performing well on images, and making training a lot more stable.

---

The discriminator is a 6-tower convolution network. We chose to use dilated filters on 5 of the towers to capture relative information between bars and phrases, with the remaining layer being an undilated 4x4 convolution. A 4x2 filter with a 16x2 dilation captures the $n^{th}$ note of each bar at the first and third or second and fourth phrases, for example. This helps the model learn the structure of music - there are thematic patterns and repetitions at the song level as well as within phrases and within bars. Having multiple towers each learning different aspects gives us a versatility and helps avoid the "tunnel vision" nature of other generative music models.

• Our Approach

Our Approach

We hope to use the distinctive structure of Irish tunes to make them suitable for GAN-based music generation. Our main idea is to regard an Irish melody as a fixed-size object with cross-references among its parts: “music as a picture” view. The two key components of this project are:

1. Encoding temporal information using spatial data.
2. Leveraging knowledge of Irish music to design the model.

• Results

Results

Evaluating the performance of our model is hard since music is subjective. Nevertheless, here are some results. Our code and interesting findings and results will be uploaded here as well as to an online repository for public access here.

---

• Listen
• Generate

GAN Generated Music

These songs were generated using our model. You can generate your own unique songs below, using tensorflow js.