The Semantic Web has become reality over the past couple of years. While certain practical topics—such as interoperability, etc.—have at least partially been addressed, scalability and expressivity issues regarding the utilisation of multimedia metadata on the Semantic Web are still widely neglected. However, existing Web (2.0) applications handling millions of multimedia assets are starting to take advantage of Semantic Web technologies.
This work contributes to design decisions regarding scalable and smart multimedia applications on the Semantic Web. Based on an analysis of practical issues—stemming from diverse projects and activities the author has participated in over the past four years—three areas have been identified, namely (i) performance and scalability issues on the data access level, (ii) the effective and efficient representation of multimedia content descriptions, and (iii) the deployment of multimedia metadata on the Semantic Web. The three research areas have as its common base the trade-off between expressivity and scalability.
We present our findings regarding scalable, yet expressive Semantic Web multimedia applications in a number of practical settings and discuss future directions, such as interlinking multimedia.
Contents
1. Introduction
1.1 Motivation
1.2 Problem Definition
1.2.1 Performance and Scalability Issues in Distributed Metadata Sources
1.2.2 Efficient and Effective Representation of Multimedia Metadata
1.2.3 Scaleable Multimedia Metadata Deployment on the Semantic Web
1.3 Reader’s Guide
1.4 What this work is NOT about
2. Related and Existing Work
2.1 Semantic Web Applications
2.1.1 Projects & Activities
2.2 Multimedia Applications
2.2.1 Smart Multimedia Content
2.2.2 Multimedia Metadata Deployment
2.2.3 Semantic Web Multimedia Applications—SWMA
2.2.4 Projects & Activities
2.3 Scalability and Expressivity
2.3.1 Infrastructure Level
2.3.2 Application Level
2.3.3 Projects & Activities
3. Multimedia Metadata
3.1 Multimedia Container Formats
3.1.1 eXtensible HyperText Markup Language–(X)HTML
3.1.2 Scalable Vector Graphics–SVG
3.1.3 Synchronized Multimedia Integration Language–SMIL
3.1.4 eXtensible 3D—X3D
3.2 Aspects of Multimedia Metadata
3.2.1 An Attempt of A Definition
3.2.2 Types of Metadata
3.2.3 Scope of Metadata
3.3 Multimedia Metadata Formats
3.3.1 Metadata for Still Images
3.3.2 Metadata for describing Audio Content
3.3.3 Metadata for describing Audio-Visual Content
3.3.4 Multimedia Content Description Interface–MPEG-7
3.3.5 Formats For Describing Specific Domains Or Workflows
3.3.6 Interoperability
4. Semantic Web
4.1 Logic and the Semantic Web
4.1.1 Knowledge Representation
4.1.2 Description Logics (DL)
4.1.3 Logic Programming (LP)
4.1.4 Integrating DL and LP
4.2 Semantic Web Vision
4.2.1 Synopsis
4.2.2 Current State
4.2.3 Future
4.2.4 Related Fields
4.3 Semantic Web Stack
4.3.1 Encoding & Addressing
4.3.2 Data Structure and Exchange
4.3.3 Data Model
4.3.4 Ontologies, Rules & Query
4.3.5 Trust & Data Provenance
4.3.6 Semantic Web Issues
4.4 Semantic Web Vocabularies
4.4.1 Generic Vocabularies
4.4.2 Social Vocabularies
4.4.3 Spatio-temporal Vocabularies
4.4.4 Other Vocabularies
4.5 Linked Data
4.6 Web 3.0
4.6.1 Web 2.0: Ajax & Mashups
4.6.2 Metadata in HTML
4.6.3 Web 2.0 + Semantic Web = Web 3.0?
4.7 Conclusion
5. Creating Smart Content Descriptions
5.1 Information Flow and Media Semantic Web Stack
5.2 Extraction vs. Annotation
5.2.1 Extraction
5.2.2 Annotation
5.3 How To Deal with the Semantic Gap
5.3.1 Low-level Feature Based Approach
5.3.2 Model-based Approach
5.3.3 Semantic Web Approach
5.3.4 Hybrid Approach
5.4 Multimedia Ontology Engineering
5.4.1 Methodologies
5.4.2 Ontology Engineering Tools
5.4.3 Review of Existing Multimedia Ontologies
6. Scaleable yet Expressive Content Descriptions
6.1 Introduction
6.2 Motivation and Scenarios
6.3 Requirements for the Description of Multimedia Assets
6.4 Environment Analysis: The Semantic Web
6.5 Multimedia Assets on the Semantic Web
6.6 Formal Descriptions of Multimedia Assets
6.6.1 Ontology Languages
6.6.2 Rules
6.6.3 Comparing Formal Descriptions Regarding the Requirements
6.7 Conclusions
7. Rational & Common Concepts
7.1 The Semantic Web Stack regarding SWMA
7.2 Design Principles and Common Concepts
7.2.1 Occam’s Razor
7.2.2 Follow-your-nose
7.2.3 Reuse & Layering
7.3 Expressivity on the Semantic Web
7.4 Scalability on the Semantic Web
7.5 Conclusion
8. A Performance and Scalability Metric for Virtual RDF Graphs
8.1 Motivation
8.2 Related and Existing Work
8.3 Virtual RDF Graphs
8.3.1 Types Of Sources
8.3.2 Characteristics Of Sources
8.4 A Metric for Virtual RDF Graphs
8.5 Conclusion
8.6 Acknowledgements
9. Media Semantics Mapping
9.1 Environment
9.2 Related Work
9.3 Media Semantics Mapping
9.3.1 Data and Metadata
9.3.2 Media Semantics
9.3.3 Spaces of Abstraction
9.3.4 Built-in rules
9.3.5 User-defined rules
9.3.6 The MSM Knowledge Base
9.4 Applying the Media Semantics Mapping
9.5 Mapping the NM2 Workflow to the Canonical Model
9.5.1 The NM2 Workflow
9.5.2 Authoring Of Non-linear Stories
9.5.3 Example NM2 Productions
9.5.4 Lessons Learned
9.5.5 The NM2 Workflow in Terms of Canonical Processes
9.6 Discussion
10. Efficient Multimedia Metadata Deployment
10.1 Motivation
10.1.1 Last Mile of Multimedia Metadata Deployment
10.1.2 Related Work
10.1.3 Design Principles
10.2 Use Cases
10.2.1 Use Case: Annotate and Share Photos Online
10.2.2 Use Case: Purchasing Music Online
10.2.3 Use Case: Describing the Structure of a Video
10.2.4 Use Case: Publishing Professional Content with Metadata
10.2.5 Use Case: Expressing and Using Complex Rights Information
10.2.6 Use Case: Detailed Description of Large Media Assets
10.2.7 Use Case: Cultural Heritage
10.2.8 Derived Requirements from the Use Cases
10.3 RDFa-deployed Multimedia Metadata
10.3.1 ramm.x Vocabulary
10.3.2 ramm.x extensions
10.3.3 Processing ramm.x Descriptions
10.4 Examples
10.4.1 Deploying a Still Image along with Exif Metadata
10.4.2 An Example from Cultural Heritage
10.5 Conclusion and Future Work
11. Conclusions
12. Outlook
12.1 Semantic Web multimedia applications now and in 10 years time
12.1.1 Emerging Metadata
12.1.2 Advanced Annotation Techniques
12.1.3 Interactive Media
12.2 Future Work
12.2.1 Meshups and More
12.2.2 Multimedia and the Web of Data
Objectives and Topics
This thesis aims to address the challenges of building scalable and expressive Semantic Web multimedia applications. The primary research focus lies on solving practical problems associated with the efficient access, representation, and deployment of multimedia metadata on a Web-scale, specifically targeting the trade-off between expressivity and scalability in various application scenarios.
- Performance and scalability in distributed multimedia metadata sources.
- Effective and efficient representation of multimedia content and metadata vocabularies.
- Deployment of multimedia metadata on the Semantic Web, particularly bridging the gap between existing formats and RDF.
- Engineering principles for designing Semantic Web multimedia applications.
- Evaluation of formal description paradigms and ontology engineering methodologies for media assets.
Excerpt from the Book
Example 1.1 (Low-level feature description of a media asset with RDF).
A video clip with a duration of one hour is described with MPEG-7. Several visual low-level features (F) as colour, shape, texture, etc. are extracted for a number of spatial segments (S) per key frame (K). A multimedia ontology is then used to represent the MPEG-7 descriptors formally (on basis of RDF); an average number of RDF triples is assumed for each descriptor (TD). An estimation of the resulting RDF graph size then is F·K·S·TD. Let us assume that we want to capture 10 features, some 1000 key frames may exist, 10 spatial segments are marked up, and finally 10 triples are required per descriptor. This yields a total RDF graph size of 1 million triples—just for describing the low-level features of an hour of video footage.
Summary of Chapters
1. Introduction: This chapter provides motivation, defines the core research questions regarding the expressivity-scalability trade-off, and introduces the problem scope.
2. Related and Existing Work: This chapter reviews the current landscape of Semantic Web and multimedia applications, critically examining projects that have addressed similar scalability and expressivity issues.
3. Multimedia Metadata: This chapter provides a comprehensive overview of existing container formats and metadata standards, discussing their suitability for the Semantic Web.
4. Semantic Web: This chapter covers the logical foundations of the Semantic Web, detailing knowledge representation, description logics, and the semantic stack.
5. Creating Smart Content Descriptions: This chapter elaborates on the information flow in multimedia applications, exploring extraction versus annotation and multimedia ontology engineering.
6. Scaleable yet Expressive Content Descriptions: This chapter analyzes the requirements for multimedia assets and provides a comparison of formal description paradigms like OWL vs. WSML.
7. Rational & Common Concepts: This chapter defines design principles for engineering Semantic Web multimedia applications, such as Occam's Razor and the Follow-your-nose principle.
8. A Performance and Scalability Metric for Virtual RDF Graphs: This chapter defines a performance metric for accessing non-native RDF sources and demonstrates its application.
9. Media Semantics Mapping: This chapter discusses bridging the Semantic Gap through mapping low-level MPEG-7 features to formal semantic concepts in a non-linear production environment.
10. Efficient Multimedia Metadata Deployment: This chapter introduces the ramm.x framework for deploying existing multimedia metadata formats on the Semantic Web using RDFa.
11. Conclusions: This chapter summarizes the contributions of the thesis, reviewing the developed metrics, ontologies, and deployment frameworks.
12. Outlook: This chapter discusses future trends in Semantic Web multimedia applications and identifies open research areas.
Keywords
Semantic Web, Multimedia Applications, Scalability, Expressivity, Metadata, RDF, MPEG-7, Ontology Engineering, Multimedia Metadata Deployment, Semantic Gap, RDFa, Linked Data, Web 3.0, Knowledge Representation, Multimedia Semantics
Frequently Asked Questions
What is the core focus of this doctoral thesis?
The thesis focuses on addressing the expressivity-scalability trade-off when building smart and scalable multimedia applications on the Semantic Web.
What are the three main research areas identified?
The research addresses (i) performance and scalability in distributed data access, (ii) efficient representation of multimedia descriptions, and (iii) the deployment of multimedia metadata on the Semantic Web.
What is the primary objective of the work?
The goal is to provide design decisions and frameworks to enable effective utilization of multimedia assets on the Semantic Web while ensuring they remain scalable.
Which scientific methods are primarily employed?
The work utilizes a multidimensional analysis of practical project requirements, formal analysis of ontology languages, and experimental evaluation through metrics and prototypes (e.g., PSIMeter).
What does the main body of the work cover?
The main body systematically covers the fundamentals of the Semantic Web and multimedia metadata, defines engineering principles, introduces a scalability metric for virtual RDF graphs, describes media semantics mapping, and presents the ramm.x framework for metadata deployment.
Which keywords best characterize the research?
Core keywords include Semantic Web, Multimedia Metadata, Scalability, Expressivity, RDF, Ontology Engineering, and Multimedia Metadata Deployment.
What is the "Semantic Gap" in this context?
It refers to the lack of coincidence between automatically extracted low-level features from multimedia content and the high-level semantic interpretation required by human users.
How does the author propose bridging the "last mile" of metadata deployment?
The author proposes the ramm.x framework, which uses RDFa to embed or reference existing multimedia metadata formats (like Exif or MPEG-7) in Web documents, making them accessible to Semantic Web agents without inventing entirely new formats.
- Citation du texte
- Dr. Michael Hausenblas (Auteur), 2009, Building Scalable and Smart Multimedia Applications on the Semantic Web, Munich, GRIN Verlag, https://www.grin.com/document/126801
