The goal of the indexing work package is to develop algorithms/techniques and provide the modules for the automatic extraction of signal descriptors of the musical contents. These descriptors are then used by the various SemanticHIFI functions for the browsing within and manipulation of recorded music tracks according to their musical content.
The targeted functionalities are music segmentation / summarization / visualization; browsing/searching music using high-level music descriptors (rhythm / tonal / timbre features); music remixing (by source separation using score alignment); browsing by lyrics (which requires the lyrics to be aligned to the audio by score alignment); audio identification; automatic learning of user-defined categories.
The complexity of the extraction from the audio signal of a specific features depends strongly on the kind of music considered (it is easier to extract melody from a monophonic signal, than from a complex mixture signal; it is easier to detect tempo for a pop track than for jazz/classical music). This explains that some functionalities of the work package are studied using several strategies (tempo/phase). Together with strategies of extraction of descriptors identified for the targeted SemanticHIFI features, the research on indexing also includes a mechanism for learning the audio characteristics of arbitrary categories defined by the user (EDS System).