A Personalized Multimodal Approach to Music Recommendation for Music Therapy
Mental health disorders and suicidality are rising among adolescents and young adults (A-YA) while rates of treatment engagement continue to remain low. Though the effectiveness of music therapy to enhance mental health is yet unknown for A-YA, the power of music to evoke emotions, facilitate communication, and promote relaxation suggests its promise. In this research study we discuss a multimodal music recommendation system which is capable of taking inputs in the form of a BFI-10 questionnaire, a series of self-view questions in the form of a survey, and an image of the user’s face in order to determine music preferences based on existing correlations between personality traits and taste in music, as an effort to make advancements in the field of music therapy.
“Adolescent mental health statistics,” UNICEF DATA. https://data.unicef.org/topic/child-health/mental-health/#_edn1
D. S. Pledge, R. T. Lapan, P. P. Heppner, D. Kivlighan, and H. J. Roehlke, “Stability and severity of presenting problems at a university counseling center: A 6-year analysis.,” Professional Psychology Research and Practice, vol. 29, no. 4, pp. 386–389, Aug. 1998, doi: https://doi.org/10.1037/0735-7028.29.4.386.
S. A. Benton, J. M. Robertson, W.-C. Tseng, F. B. Newton, and S. L. Benton, “Changes in counseling center client problems across 13 years.,” Professional Psychology Research and Practice, vol. 34, no. 1, pp. 66–72, Feb. 2003, doi: https://doi.org/10.1037/0735-7028.34.1.66.
D. Nopf, M. Park, and T. Mulye, “The Mental Health of Adolescents: A National Profile, 2008,” 2008. Available: https://nahic.ucsf.edu/wp-content/uploads/2008/02/2008.MentalHealthBrief.pdf
L. Rebecchini, “Music, mental health, and immunity,” Brain, Behavior, & Immunity - Health, vol. 18, no. 100374, p. 100374, Dec. 2021, doi: https://doi.org/10.1016/j.bbih.2021.100374.
J. Huang and X. Li, “Effects and Applications of Music Therapy on Psychological Health: A Review,” Advances in Social Science, Education and Humanities Research, 2022, doi: https://doi.org/10.2991/assehr.k.220110.186.
S. Koelsch, “Music-evoked emotions: principles, brain correlates, and implications for therapy,” Annals of the New York Academy of Sciences, vol. 1337, no. 1, pp. 193–201, Mar. 2015, doi: https://doi.org/10.1111/nyas.12684.
A. C. North, D. J. Hargreaves, and S. A. O’Neill, “The importance of music to adolescents,” British Journal of Educational Psychology, vol. 70, no. 2, pp. 255–272, Jun. 2000, doi: https://doi.org/10.1348/000709900158083.
“Resource-Oriented Music Therapy in Me-
ntal Health Care,” barcelonapublishers.com.
https://barcelonapublishers.com/resource-oriented-music-therapy-mental-health-care(accessed Jun. 29, 2024).
D. Paul and S. Kundu, “A Survey of Music Recommendation Systems with a Proposed Music Recommendation System,” Advances in Intelligent Systems and Computing, pp. 279–285, Jul. 2019, doi:
https://doi.org/10.1007/978-981-13-7403-6_26.
Y. Song, S. Dixon, and M. Pearce, “A Survey of Music Recommendation Systems and Future Perspectives.” Available: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=e008
afae01ad796060abcf602abff6024754
P. J. Rentfrow and S. D. Gosling, “The do re mis of everyday life: The structure and personality correlates of music preferences.,” Journal of Personality and Social Psychology, vol. 84, no. 6, pp. 1236–1256, 2003, doi: https://doi.org/10.1037/0022-3514.84.6.1236.
L. Tian, R. Alaei, and N. O. Rule, “Appearance Reveals Music Preferences,” Personality and Social Psychology Bulletin, vol. 48, no. 12, p. 014616722110482, Sep. 2021, doi: https://doi.org/10.1177/01461672211048291.
A. Mollahosseini, B. Hasani, and M. H. Mahoor, “AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild,” IEEE Transactions on Affective Computing, vol. 10, no. 1, pp. 18–31, Jan. 2019, doi: https://doi.org/10.1109/TAFFC.2017.2740923.
T. Schäfer and P. Sedlmeier, “What makes us like music? Determinants of music preference.,” Psychology of Aesthetics Creativity and the Arts, vol. 4, no. 4, pp. 223–234, Nov. 2010, doi: https://doi.org/10.1037/a0018374.
Pranav Chandane, “Facial Beauty Rating - SCUT-FBP5500 V2,” Kaggle.com, 2024. https://www.kaggle.com/datasets/pranavchandane/scut-fbp5500-v2-facial-beauty-scores (accessed Jun. 29, 2024).
L. Liang, L. Lin, L. Jin, D. Xie, and M. Li, “SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction,” arXiv:1801.06345 [cs], Jan. 2018, Available: https://arxiv.org/abs/1801.06345
“Top 10000 Songs on Spotify 1960-Now,” www.kaggle.com.https://www.kaggle.com/ datasets/joebeachcapital/top-10000-spotify-songs-1960-now/data
A. I. Siam, N. F. Soliman, A. D. Algarni, F. E. Abd El-Samie, and A. Sedik, “Deploying Machine Learning Techniques for Human Emotion Detection,” Computational Intelligence and Neuroscience, vol. 2022, p. e8032673, Feb. 2022, doi: https://doi.org/10.1155/2022/8032673.
A. Mollahosseini, B. Hasani, and M. H. Mahoor, “AffectNet: a database for facial expression, valence, and arousal computing in the wild,” IEEE Transactions on Affective Computing, vol. 10, no. 1, pp. 18–31, Jan. 2019, doi: https://doi.org/10.1109/taffc.2017.2740923.
D. Rawlings and V. Ciancarelli, “Music Preference and the Five-Factor Model of the NEO Personality Inventory,” Psychology of Music, vol. 25, no. 2, pp. 120–132, Oct. 1997, doi: https://doi.org/10.1177/0305735697252003.
P. G. Dunn, B. de Ruyter, and D. G. Bouwhuis, “Toward a better understanding of the relation between music preference, listening behavior, and personality,” Psychology of Music, vol. 40, no. 4, pp. 411–428, Mar. 2011, doi: https://doi.org/10.1177/0305735610388897.
T. Schäfer, P. Sedlmeier, J. Krems, and H. Bruhn, “Determinants of Music Preference (Bestimmungsgrößen für Musikpräferenz)" Dissertation, 2008. Accessed: Jun. 29, 2024. [Online]. Available: https://monarch.qucosa.de/api/qucosa%3A19075/attachment/ATT-0/
“John Srivastava 1995 big5,” 1995. Available: http://jenni.uchicago.edu/econ-psych-traits/John_Srivastava_1995_big5.pdf
H. J. Pandya et al., “Label-free electrical sensing of bacteria in eye wash samples: A step towards point-of-care detection of pathogens in patients with infectious keratitis,” Biosensors and Bioelectronics, vol. 91, pp. 32–39, May 2017, doi: https://doi.org/10.1016/j.bios.2016.12.035.
B. Rammstedt and O. P. John, “Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German,” Journal of Research in Personality, vol. 41, no. 1, pp. 203–212, Feb. 2007, doi: https://doi.org/10.1016/j.jrp.2006.02.001.
GhavamiNejad P, GhavamiNejad A, Zheng H, Dhingra K, Samarikhalaj M, Poudineh M., “A Conductive Hydrogel Mi-croneedle‐Based Assay Integrating PEDOT: PSS and Ag‐Pt Nanoparticles for Real‐Time, Enzyme‐Less, and Electro-chemical Sensing of Glucose,” Advanced Healthcare Materials, vol. 12, no. 1, Oct. 2022, doi: https://doi.org/10.1002/adhm.202202362.
M. Safavieh et al., “Paper microchip with a graphene-modified silver nano-composite electrode for electrical sensing of microbial pathogens,” Nanoscale, vol. 9, no. 5, pp. 1852–1861, 2017, doi: https://doi.org/10.1039/c6nr06417e.
Y.-I. Tian, T. Kanade, and J. F. Cohn, “Recognizing action units for facial expression analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97–115, 2001, doi: https://doi.org/10.1109/34.908962.
B. Amos, B. Ludwiczuk, M. Satyanarayanan,"Openface: A general-purpose face recognition library with mobile applications,"CMU-CS-16-118, CMU School of Computer Science, Tech. Rep., 2016.
Pundlik, A.; Verma, S.; Dhingra, K. Neural Pathways Involved in Emotional Regulation and Emotional Intelligence. J. Knowl. Learn. Sci. Technol. 2024, 3 (3), 165-192. https://doi.org/10.60087/jklst. vol3.n3. p.165-192.
S. J. Philibotte, S. Spivack, N. H. Spilka, I. Passman, and P. Wallisch, “The Whole is Not Different From its Parts,” vol. 40, no. 3, pp. 220–236, Feb. 2023, doi: https://doi.org/10.1525/mp.2023.40.3.220.
J. Bernard, T. Ruppert, M. Scherer, Jörn Kohlhammer, and T. Schreck, “Content-based layouts for exploratory metadata search in scientific research data,” CiteSeer X (The Pennsylvania State University), Jan. 2012, doi: https://doi.org/10.1145/2232817.2232844.
R. Van Meteren and M. Van Someren, “Using Content-Based Filtering for Recommendation 1.” Available: https://users.ics.forth.gr/~potamias/mlnia/paper_6.pdf
F. Cacheda, V. Carneiro, D. Fernández, and V. Formoso, “Comparison of collaborative filtering algorithms,” ACM Transactions on the Web, vol. 5, no. 1, pp. 1–33, Feb. 2011, doi: https://doi.org/10.1145/1921591.1921593.
Mehta, A.; Alaiashy, O.; Kumar, P.; Tamilinian, V.; Besong, S.; Balpande, S.; Verma, S.; Dhingra, K. Advancing Model-Based Systems Engineering in Biomedical and Aerospace research: A Comprehensive Review and Future Directions. J. Knowl. Learn. Sci. Technol. 2024, 3 (4), 133-147. https://doi.org/10.60087/jklst.v3.n4.p133.
Chilmakuri, L.; Mishra, A. K.; Shokeen, D..; Gupta, P..; Wadhwa, H. H.; Dhingra, K..; Verma, S. A Wearable EMG Sensor for Continuous Wrist Neuromuscular Activity for Monitoring. J. Knowl. Learn. Sci. Technol. 2024, 3 (4), 148-159. https://doi.org/10.60087/jklst.v3.n4.p148.
E. Kobayashi, Takayasu Fushimi, K. Saito, and T. Ikeda, “Similarity Search by Generating Pivots Based on Manhattan Distance,” Lecture notes in computer science, pp. 435–446, Jan. 2014, doi: https://doi.org/10.1007/978-3-319-13560-1_35.
Chandna, R..; Bansal, A.; Kumar, A.; Hardia, S.; Daramola, O.; Sahu, A.; Verma, K.; Dhingra, K.; Verma, S. Skin Disease Classification Using Two Path Deep Transfer Learning Models. J. Knowl. Learn. Sci. Technol. 2024, 3 (4), 169-187. https://doi.org/10.60087/jklst.v3.n4.p169.
S. Niwattanakul, J. Singthongchai, E. Naenudorn, and S. Wanapu, “Using of Jaccard coefficient for keywords similarity,” ResearchGate, Mar. 2013, [Online]. Available: https://www.researchgate.net/publication/317248581_Using_of_Jaccard_Coefficient_for_Keywords_Similarity
V. Thada and V. Jaglan, “Comparison of Jaccard, Dice, Cosine Similarity Coefficient to Find Best Fitness Value for Web Retrieved Documents Using Genetic Algorithm.” Available: https://dknmu.org/uploads/file/6842.pdf
M. Norouzi, D. J. Fleet, and R. R. Salakhutdinov, “Hamming Distance Metric Learning,” Neural Information Processing Systems, 2012. https://proceedings.neurips.cc/paper/2012/hash/59b90e10
a220e2ebc542eb9d950b1e-Abstract.html (accessed Jun. 29, 2024).
C. Shao, P. Cui, P. Xun, Y. Peng, and X. Jiang, “Rank correlation between centrality metrics in complex networks: an empirical study,” Open Physics, vol. 16, no. 1, pp. 1009–1023, Dec. 2018, doi: https://doi.org/10.1515/phys-2018-0122.
F. Rahutomo, T. Kitasuka, and M. Aritsugi, “Semantic Cosine Similarity,” www.semanticscholar.org, 2012. https://www.semanticscholar.org/paper/Semantic-Cosine-Similarity-Rahutomo-Kitasuka/41ff3934f40c32ac8643270822de1c763e16c71b
P. K. Singh, P. K. D. Pramanik, and P. Choudhury, “A Comparative Study of Different Similarity Metrics in Highly Sparse Rating Dataset,” Data Management, Analytics and Innovation, pp. 45–60, Sep. 2018, doi: https://doi.org/10.1007/978-981-13-1274-8_4.
C.-C. Lu and V. S. Tseng, “A novel method for personalized music recommendation,” Expert Systems with Applications, vol. 36, no. 6, pp. 10035–10044, Aug. 2009, doi: https://doi.org/10.1016/j.eswa.2009.01.074.
A. Nanopoulos, D. Rafailidis, P. Symeonidis, and Y. Manolopoulos, “MusicBox: Personalized Music Recommendation Based on Cubic Analysis of Social Tags,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 18, no. 2, pp. 407–412, Feb. 2010, doi: https://doi.org/10.1109/tasl.2009.2033973.
A. B. R. Shatte, D. M. Hutchinson, and S. J. Teague, “Machine learning in mental health: a scoping review of methods and applications,” Psychological Medicine, vol. 49, no. 09, pp. 1426–1448, Feb. 2019, doi: https://doi.org/10.1017/s0033291719000151.
Copyright (c) 2024 International Journal of Engineering and Computer Science
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.