Saraga Audiovisual : A large multimodal open data collection for the analysis of Carnatic Music.

Shankar, Adithi - Plaja-Roglans, Genís - Nuttall, Thomas - Rocamora, Martín - Serra, Xavier

Resumen:

Carnatic music is a style of South Indian art music whose analysis using computational methods is an active area of research in Music Information Research (MIR). A core, open dataset for such analysis is the Saraga dataset, which includes multi-stem audio, expert annotations, and accompanying metadata. However, it has been noted that there are several limitations to the Saraga collections, and that additional relevant aspects of the tradition still need to be covered to facilitate musicologically important research lines. In this work, we present Saraga Audiovisual, a dataset that includes new and more diverse renditions of Carnatic vocal performances, totalling 42 concerts and more than 60 hours of music. A major contribution of this dataset is the inclusion of video recordings for all concerts, allowing for a wide range of multimodal analyses. We also provide high-quality human pose estimation data of the musicians extracted from the video footage, and perform benchmarking experiments for the different modalities to validate the utility of the novel collection. Saraga Audiovisual, along with access tools and results of our experiments, is made available for research purposes.

Detalles Bibliográficos
2024
Este trabajo fue realizado bajo el marco de "IA y Música: Cátedra en Inteligencia Artificial y Música (TSI-100929-2023-1)", financiado por la Secretaría de Estado de Digitalización e Inteligencia Artificial y la European Union-Next Generation EU, en el marco del programa Cátedras ENIA 2022 para la creación de cátedras universidad-empresa en IA.
Applications
Music videos
Multimodal music systems
Evaluation, datasets, and reproducibility
Novel datasets and use cases
Knowledge-driven approaches to MIR
Computational ethnomusicology
MIR tasks
Pattern matching and detection
Sound source separation
Computational musicology
Inglés
Universidad de la República
COLIBRI
https://ismir2024.ismir.net/
https://hdl.handle.net/20.500.12008/47319
Acceso abierto
Licencia Creative Commons Atribución (CC - By 4.0)