A thesis submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science, School of Engineering and Technology

Thesis (Ph.D.) - Asian Institute of Technology, 2012

Musical sequences with actors dancing and lip-synching to songs sung by playback singers are integral parts, particularly of South Asian movies. Fans seek out movies for their songs and they often seek songs of a particular genre or those vocalized by particular actors. In fact, song and dance sequence of South Asian movies are an industry of their own. Given the huge numbers of movies produced in South Asia over the past decades, most of which are in digital archives, it is an important problem to automatically extract and categorize their musical sequences.It is this problem, falling within the modern discipline of multimedia data mining,that is addressed in this thesis. We develop a new theoretical approach and based upon it a complete software system for musical sequence extraction from movies. Our Method invokes an SVM-based classifier and makes as well a novel application of probabilistic timed automaton to distinguish musical sequences from non-musical. Our System analyzes both audio and video signals to give a classifier that not only extracts musical sequences but identifies their genre, and detects the vocalizing actors. This Thesis describes the underlying theory, the system from end to end, and a large batch of validating experiments.