Authors
Members: Maryam Victor, Pierre Malak, Abdallah Khayrat, Seifeldin Ahmed
Supervisors: Eng.Haytham, Eng.Verina Adel, Dr. Mohamed El Shalakani
Publishing Date
8th of January 2022
Abstract
As robots and Artificial intelligence combined together provide us with the ultimate technology to replace individuals and achieve tasks that were impossible to achieve when done by people. The main idea of our project is to develop a semi-humanoid robot for on-campus surveillance using deep learning. It can interact with users using face recognition and recognises commands. Its main purpose is to detect unwanted abnormal and unethical behaviours between university students such as fighting or playing cards or smoking. Also, give a ticket/notification accordingly. It also helps with the services that students normally need such as looking for the route to their destination or retrieving their schedules.
1.1 Purpose of this document
The purpose of this Software document (SRS) is to outline and specify the requirements of this system: ‘Learning-based semi-humanoid robot for on-campus surveillance’. Which detects abnormal and unethical activities between students. This document shows the components, the software implementation and gives a clear system overview. This document is intended for the developers of this system and the graduation projects evaluators.
1.2 Scope of this document
This document will include vivid explanations of the system’s architecture, workflows. It provides a clear and detailed description of the functional and non-functional requirements of our system which is a learning based Semi-Humanoid robot for On-Campus surveillance using deep learning.The document shows the various stages that the software goes through before being deployed on the pepper robot and working on-campus.
1.3 System Overview
This system is a software for a robot that will be used on-campus to help students, staff and security it uses the university database to identify the students faces and know their IDs’ and it saves the report on the server each day.
To prepare the data we:
• Customized data set for the playing cards to have a big data set that include all the possible cards as UNO cards for example and to label it.[7]
• We labeled data set to use YOLO algorithm.[8]
The pre-processing state:
• We separated the videos into frames to be used in the fighting model.
• The images were labeled to be used in the cards, smoking and face detection models.
• The sound was detected by speech-recognition to be used voice commands.
The code overview:
• Fighting model: It is a deep learning model for fighting detection. A dataset consisting of videos is used. then 20 frames are extracted from each video by tensorflow. CNN using a pre trained keras model (VGG19) is applied. Then the output goes to LSTM. The data is splited into training ( 80 percent) and testing( 20 percent). then the model is trained and tested and evaluated.
• Smoking detection is using the YOLO algorithm to detect the cigarette in hands by training the model with a pre-trained model, concentrating on the batch size and the number of epochs to get the higher accuracy in the model, and in the detection can be used by sending 0 to realtime capture or sending the URL of the video or the picture to detect.py file.
• Playing-Cards detection is using the YOLOV5 algorithm to detect the behaviour by training the model with a pre-trained model from YOLOV5m, to get the weights file which we will be used to detect, the YOLOV5 is working by labelling your own customized dataset and saving its annotation format into YOLOV5 PyTorch, to use its labels to train the model using train.py file, then validate it by val.py file sending the parameters of the trained weight and the validation dataset.
1.4 System Scope
The purpose of this system is to have a semi-humanoid robot that uses deep learning to detect unwanted abnormal and unethical activities between students on campus. It helps by doing detection as well as recognising the actor and giving tickets/notifications accordingly. Also, providing students with some services they normally need such as finding the route to their destination on campus or retrieving their schedules.