Authors

Shahd Hesham,Samira Refaat,Rawan Khaled,Dalia Yasser,Dr.Fatma Helmy,Eng.Nada Ayman



Publishing Date

December 28, 2020


Abstract

The main idea of this project is to build a virtual tour guide that focuses on monument & landmark recognition inside of Egypt and explaining the history and the stories behind them. Based on the rich history of Egypt, this system holds a great value where millions of people from different nationalities and backgrounds visit Egypt every year; to learn more about its magnificent stories and secrets. The goal of the project is to enhance these tourists’ experience and make it more enjoyable. To do that We will build our system using both the traditional and non-traditional machine learning techniques to detect the landmarks, then we will implement an augmented reality layer to start acting as a guide through either audio or text.

1.1 Background

This document is intended to delineate the document details. Furthermore, the documentation is used to guide the developers, and to be a product approval record for the needed functions. This document will explain the software implementation. The software implementation covers the algorithms and methods used. The monument recognition network is intended to give full answers to the tourists to help them with understanding the arts they see. The application will contain dataset from different and various Egyptian places and monuments and will permit the user to identify the monument by accessing the camera (live-stream) or uploading a video.

1.2 Scope of this document

Monument Recognition is to detect the historical places without the need of a tour guide or even declarative signs. It will play a vital role in improving the touristic experiences in regions that are not popular enough. Through a mobile application, the user would gain knowledge in either text or audio forms.

The phases of creating the system are gathering information and collecting data, followed by pre-processing phase, processing, testing the system and finally, deployment.

1.3 System Overview

The product of the system is an application that is intended to be used by anyone who needs to get information about certain monuments and places in Egypt, by the usage of images, videos, camera, and location. It contains the following steps:

– Preparing a dataset collecting the images of the historical sites we are targeting for our projects first phase

-Using data augmentation cite{mikolajczyk2018data} to enhance and enlarge the dataset that we have collected

-Building Classification models by firstly dividing our data into 2 sets training and testing set we are working with 75% and 25% respectively.

-Training the data using two approaches:

Deep Learning models & Machine Learning Models

-Testing the models each classifier will be tested to evaluate its performance using the testing dataset. To be able to choose the best classifier that we will build our project based on, Keeping an eye especially on the Capsule Neural Network results

– Evaluating the classifiers’ results comes next as it is very important; cross-validation is used to assess the efficiency of various techniques and compare the results with the results of the testing phase.

– Deploying the project proposed would be our last phase in the project. A video that contains many monuments is the input to our application. The purpose of the application is to break the video into frames and identify the monuments in each of them. The planned output for the end-user is the name and overview of each monument. In this area, we will face many challenges, such as identifying monuments that are partly obscured by objects. An Augmented reality layer will be added to act as a tour guide where it will present both audio and text.

1.4 System Scope

-Collecting a dataset of Egyptian monuments. We will also keep in mind all the requirements for benchmark datasets that are: relevant representative, non-redundant, scalable, reusable when building our dataset. We build it by the use of Google images and other search engines. Also, we would collect the images and videos ourselves if needed.

– Determining features needed to be extracted for our models.

– Offline & Online video processing in the deployment phase

-Recognition using conventional and non-traditional machine learning and deep learning methods for historical monuments.

-Using neural network capsule and other state-of-the-art techniques to boost the findings obtained by CNN and other techniques.

-Recognition for the whole monument even if it is blocked by another object.

– Adding an augmented reality layer after recognition is done.