1. What is MIRROR?
MIRROR (MPEG-7 Image Retrieval Refinement based On Relevance feedback)
is a platform for content-based image retrieval (CBIR) research and development using MPEG-7 technologies.
MIRROR supports several MPEG-7 visual descriptors:
- Color Descriptors:
- Dominant Color Descriptor (DCD)
- Scalable Color Descriptor (SCD)
- Color Layout Descriptor (CLD)
- Color Structure Descriptor (CSD)
- Texture Descriptors:
- Edge Historgram Discriptor (EHD)
- Homogeneous Texture Descriptor (HTD)
The system core is based on MPEG-7 Experimentation Mode (XM) with web-based user interface for
query by image example retrieval. A new Merged Color Palette approach for DCD similarity measure
and relevance feedback are also developed in this system.
The system is highly modularized, new algorithms, new ground truth set, and even new image database can be added easily.
2. Major functions of MIRROR
MIRROR mainly supports the following image retrieval functions:
- Hierarchical image browsing
- (Click on the directory tree in the left column)
- Content based image retrieval by example image
- Relevance feddback retrieval
- Auotmatic performance meaure using ANMRR
- Random browing
3. System Architecture
MIRROR consists of three main modules as shown in Fig.1.
Feature Extraction module extracts the descriptor from the original image and
converts the information into MPEG-7 data stream. Similarity Measure module uses
the user reference image as a query to search relevant images in the database
based on the pre-generated MPEG-7 data. Relevance Feedback module receives user
feedback on relevant retrieval and generates a new query for Similarity Measure module for a new retrieval.
Fig. 1: Architectural structure of MIRROR
The extraction tools of MIRROR are based on MPEG-7 Experimentation Model (XM).
The image features are extracted from the image collection using the extraction tools
for indexing and subsequent image retrieval. Image features are described by MPEG-7
Description Definition Language (DDL). Multiple MPEG-7 data streams are required if different
descriptors are employed simultaneously. The MPEG-7 data stream is usually generated offline
and stored in files for subsequent image retrieval process due to the huge amount of image data involved.
Each image in the database is represented by various descriptors. Given a query image and selected descriptor, similarity retrieval involves searching the database
for similar features as the input query by reading the corresponding MPEG-7 data.
Various matching tools are defined for each descriptor and description scheme.
Particularly, MIRROR improves the retrieval performance by using a new matching tool
-- Merged Palette Histogram Similarity Measure for DCD.
4. User Interface
MIRROR employs the query by example paradigm. Fig. 2 shows the web-based user interface of MIRROR.
Fig. 2: The reference query image and the initial query results.
To start the search, the user chooses a reference image from a set of example images.
The user may browse the category tree (top left column of Fig. 2) for other set of images.
After choosing the reference image and search method, the system returns the images with the highest ranks
(right column of Fig. 2). The query image is shown at the top of the screen.
Retrieved images are shown at the center of the screen in similarity descending order.
This allows users to select relevant images to guide the next search.
The system usually retrieves relevant images and false matches. The user checks the images that are
more relevant with what he/she is looking for and iterates the process until desired images are found.
During interactive search, the user selects or unselects the example image(s).
The user-selected relevant images appear in the upper row of the screen as shown in Fig. 3.
Fig. 3: Relevance feedback result after 4 more images are selected by the user from the initial retrieval result.
5. Evaluation mode
For easier comparison, there is a "Common Color Dataset" defined as a reference image database,
also have a pre-defined "Common Color Query" with ground truth set.
MIRROR has an evaluation mode for verifying the results. Just click on the "Evaluation Mode" link from left panel,
and follow the steps.
Fig. 4: MIRROR's Evaluation mode
6. Random Mode
If you want to try more result of MPEG-7 descriptors, you may use random image mode to pick
some random images for query. Just click on "Random Image" in left panel.
Then choose a reference image and search method just like the browsing mode.
Fig. 5: MIRROR's Random image mode
7. Research Team
MIRROR is developed by Dr. Lai-Man Po's postgraduate and undergrauate research projects.
The team members are:
- Academic Staff:
- Dr Lai-Man Po
- Dr Kwok-Wai Cheung
- Postgraduate student:
- Mr Ka-Man Wong (MPhil 2001 - 2004)
- Undergraduate students:
- Mr Liu Tak Shing, Sam (BECE 2002)
- Ms Lam Hiu Hung, Christy (BSIT 2002)
- Mr Chan Kin Yeung (BECE 2003)
- Mr Lau Chung Hei, Philip (BEIE 2004)
- Mr Luk Wai Lung (BEIE 2004)
Last update: 26 August 2006