Music Recommendations Based on User's Mood Using Convolutional Neural Networks

  • A Petrescu Department of Computer Science, Babes-Bolyai University, 1, M. Kogalniceanu Street, 400084, Cluj-Napoca, Romania

Abstract

This paper proposes a method for music recommendations using emotions, using deep learning techniques. The method is composed of two modules. The emotion detection module, which utilizes a hybrid architecture involving a Convolutional Neural Network (CNN) and a Reccurent Neural Network using Long-Short Term Memory (LSTM) Cells. We compared individual architectures of CNNs and LSTMs against our hybrid approach, outperforming them during experiments. We evaluated the modules on our own data set, created using Spotify’s API and containing 2028 songs from different genres and linguistic families, labeled with valence and arousal values. The model also outperforms other related approaches, however we did not evaluate them on the same data set. The predictions are used by the second module, for which we proposed a simple method of ordering the results based on the similarity to user’s input.

References

[1] Bhattarai, B., and Lee, J. Automatic music mood detection using transfer learning and multilayer perceptron. International Journal of Fuzzy Logic and Intelligent Systems 19, 2 (2019), 88–96.
[2] Clifton, A., Pappu, A., Reddy, S., Yu, Y., Karlgren, J., Carterette, B., and Jones, R. The spotify podcast dataset. arXiv preprint arXiv:2004.04270 (2020), 1–4.
[3] Delbouys, R., Hennequin, R., Piccoli, F., Royo-Letelier, J., and Moussallam, M. Music mood detection based on audio and lyrics with deep neural net. In Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27, 2018 (2018), pp. 370–375.
[4] Dey, R., and Salem, F. M. Gate-variants of gated recurrent unit (GRU) neural networks. In IEEE 60th International Midwest Symposium on Circuits and Systems, MWSCAS 2017, Boston, MA, USA, August 6-9, 2017 (2017), IEEE, pp. 1597–1600.
[5] Hevner, K. Experimental studies of the elements of expression in music. The American Journal of Psychology 48, 2 (1936), 246–268.
[6] Kamm, T., Hermansky, H., and Andreou, A. G. Learning the mel-scale and optimal vtn mapping. In Center for Language and Speech Processing, Workshop (1997), pp. 1–8.
[7] Li, T., and Ogihara, M. Detecting emotion in music. CiteSeer (2003), 1–3.
[8] Lidy, T., and Schindler, A. Parallel convolutional neural networks for music genre and mood classification. MIREX2016 (2016), 1–4.
[9] Liu, T., Han, L., Ma, L., and Guo, D. Audio-based deep music emotion recognition. AIP Conference Proceedings 1967, 1 (2018), 040021.
[10] Malik, M., Adavanne, S., Drossos, K., Virtanen, T., Ticha, D., and Jarina, R. Stacked convolutional and recurrent neural networks for music emotion recognition. CoRR abs/1706.02292 (2017).
[11] Peeters, G. A generic training and classification system for mirex08 classification tasks: audio music mood, audio genre, audio artist and audio tag. In Proceedings of the International Symposium on Music Information Retrieval (ISMIR’08) (2008), Citeseer.
[12] Petrescu, A. Spotify dataset. https://github.com/AndreiPetrescu99/SpotifyDataset.git/, 2022.
[13] Raju, A., R.S, D., Gurang, D., Kirthika, R., and Rubeena, S. AI based music recommendation system using deep learning algorithms. IOP Conference Series: Earth and Environmental Science 785 (06 2021), 012013.
[14] Russell, J. A. A circumplex model of affect. Journal of personality and social psychology 39, 6 (1980), 1161.
[15] Tan, K., Villarino, M., and Maderazo, C. Automatic music mood recognition using russell’s twodimensional valence-arousal space from audio and lyrical data as classified using svm and naive bayes. IOP Conference Series: Materials Science and Engineering 482 (03 2019), 012019.
[16] Yang, G. Research on music content recognition and recommendation technology based on deep learning. Security and Communication Networks 2022 (03 2022), Article ID 7696840.
Published
2022-07-03
How to Cite
PETRESCU, A. Music Recommendations Based on User's Mood Using Convolutional Neural Networks. Studia Universitatis Babeș-Bolyai Informatica, [S.l.], v. 67, n. 1, p. 45-60, july 2022. ISSN 2065-9601. Available at: <http://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/78>. Date accessed: 27 sep. 2022. doi: https://doi.org/10.24193/subbi.2022.1.04.
Section
Articles