Effective Multimedia Document Representations for Knowledge Discovery

Abstract

In recent years, the rapid advances in multimedia technology have led to grow the multimedia documents explosively. In order to utilize the multimodal information of multimedia documents, sophisticated knowledge discovery systems are required. The knowledge discovery systems require efficient multimedia mining methods to extract the meaningful and useful information from the huge volume of multimedia documents. The success of multimedia mining relies on the representation of multimedia documents and its multimodal contents. The appropriate representation of multimedia documents discovers the useful patterns that can be used to assist the multimedia mining methods in discovering the useful knowledge. The multimodal nature of multimedia objects is the challenging problem for the multimedia document representation, as the features of multimodal objects are in different space with different characteristics and dimensionalities. Representation of multimodal multimedia objects in a unified feature space helps the multimedia document representation and multimedia mining methods. The research work in this thesis proposes the multimedia data representation methods, multimedia document representations, and multimedia mining methods for the effective knowledge discovery in multimedia documents. In the first methodology, this thesis aims at the representation of multimodal multimedia objects in a unified feature space. We propose two multimedia data representation methods, Multimedia To Signal Conversion (MSC) and Multimedia to Image Conversion (MIC) to represent the multimedia objects in a unified domain. The MSC represents the multimedia objects in frequency domain by converting the multimedia objects as signal objects. The MIC converts the multimedia objects as image objects to represent them in spatial domain. The multimedia objects in unified domain are represented in the unified feature space using the features with similar dimensions and characteristics. Hence, both the multimedia data representation methods convert themultimodal multimedia documents as unified multimedia documents. The unified multimedia documents ease the representation of multimedia documents and improve the efficiency of multimedia mining methods. The proposed multimedia data representation methods are effectively used for knowledge extraction from multimedia documents. In the second methodology, this thesis presents the two multimedia document representations, Multimedia Suffix Tree Document (MSTD) and Multimedia Feature Pattern Tree (MFPT) to represent the unified multimedia documents. The MSTD represents the unified multimedia documents based on shared similar multimedia objects among the documents. The similarity between the multimedia objects depends on the similarity of the features. The MFPT represents the documents based on shared similar feature patterns of the multimedia objects. Both the representations are compact and provide the complete information of the documents. They function as the platform for the multimedia knowledge extraction methods. In the third methodology, this thesis explores the multimedia mining methods based on the MSTD and MFPT representations. The MSTD and MFPT based classification algorithms effectively classifies the multimedia documents. The multimedia documents are partitioned into clusters of same multimedia concepts using the MSTD and MFPT based clustering algorithms. The MSTD representation extracts the frequent multimedia patterns to generate the multimedia class association rules for classifying the multimedia documents. The MFPT representation extracts the sequential multimedia feature patterns to derive the multimedia class sequential rules that support the classification of multimedia documents based on the object characteristics. The efficacy of the proposed methods is evaluated by conducting the experiments with four datasets of multimodal multimedia documents. Experimental results demonstrate that the proposed multimedia data representation methods benefit the multimedia document representation and multimedia mining methods by representing the multimodal multimedia objectsin a unified feature space. The proposed multimedia document representations are effectively used to enhance the performance of multimedia mining methods in discovering the knowledge from multimedia documents.