Docsity
Log inSign up
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search Store documents

The best documents sold by students who completed their studies

Search through all study resources
Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

University Rankings

Discover the best universities in your country according to Docsity users

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

From our blog

Exams and Study
Go to the blog

Sleep Drowsiness Detection thesis, Thesis of Computer Science

JSS Science and Technology UniversityComputer Science

Sleep Drowsiness Sleep Drowsiness Detection thesis for final year

Typology: Thesis

2019/2020

Uploaded on 09/20/2020

ansh-sri
ansh-sri 🇮🇳

4

(2)

6 documents

1 / 17

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Rizvi College of Engineering
Department of Computer Engineering
Project Synopsis Report
On
REAL TIME SLEEP/DROWSINESS DETECTION
Submitted in partial fulfilment of the requirements
of the degree of
Bachelors of Engineering
by
Rizvi Khurram Abbas (40)
Ansari Mohd Hasim(06)
Roshan Santaram Tavhare (66)
Shaikh Rizwanul Amjad (56)
Guide:
Dr. VARSHA SHAH
University of Mumbai
2018 - 2019
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff

Partial preview of the text

Download Sleep Drowsiness Detection thesis and more Thesis Computer Science in PDF only on Docsity!

Rizvi College of Engineering

Department of Computer Engineering

Project Synopsis Report

On

REAL TIME SLEEP/DROWSINESS DETECTION

Submitted in partial fulfilment of the requirements of the degree of

Bachelors of Engineering

by

Rizvi Khurram Abbas ( 40 )

Ansari Mohd Hasim( 06 )

Roshan Santaram Tavhare ( 66 )

Shaikh Rizwanul Amjad (56)

Guide: Dr. VARSHA SHAH

University of Mumbai

Abstract The main idea behind this project is to develop a nonintrusive system which can detect fatigue of any human and can issue a timely warning. Due to the drowsiness of the employee they are not able to meet the deadlines of the project and can cause an increase in cost to the company. This system will monitor the employee eyes using a camera and by developing an algorithm we can detect symptoms of employee fatigue early enough to avoid the person from sleeping. So, this project will be helpful in detecting employee fatigue in advance and will give warning output in form of notification or pop-ups. Moreover, the warning will be deactivated manually rather than automatically. For this purpose, a de- activation dialog will be generated which will contain some simple mathematical operation which when answered correctly will dismiss the warning. Moreover, if employee feels drowsy there is possibility of incorrect response to the dialog. We can judge this by plotting a graph in time domain. If all the three input variables show a possibility of fatigue at one moment then a Warning signal is given in form of text and sound. This will directly give an indication of drowsiness/fatigue which can be further used as record of employee performance.

Rizvi College of Engineering,

Off Carter Road, Bandra(W), Mumbai- 400050

Index

Sr.no Title Page no 1 Introduction.................................. 01 2 Problem Statement & Objectives........................ 02 3 Literature Survey................................. 03 3.1 Seeing Machines face LAB 3.1.1 Advantages 3.2 LC Technologies, Inc 3.2.1 Advantages 4 Proposed Methodology................................ 04 4.1 Methodology 4.1.1 Physiological level approach 4.1.2 Behavioral based approach 4.2 TensorFlow 4.3 Machine learning: 4.4 Machine learning categories 4.4.1 Supervised learning 4.4.2 Unsupervised learning 4.5 OpenCV....................................... .... 05 4.5.1 Algorithm 4.6 Face Detection........................ ..... …......................... 06

  1. 7 Eye detection......... … ........................................ ....... 07
    1. 7 .1 Recognition of Eye's State 4.7.2 Eye Blink Detection.. ................. …............................ 08

4.8 Drowsiness Detection. ... .... ............... .................. .......... ... 09 5 Plan of Work & Project Status................... ..................... 10 5.1 Proposed Modules 5.2 Scheduling References .. ….......... ........... …........................................ 12

Chapter 2 Problem Statement In earlier phases we have seen many of the project to meet failure due to some unexpected reasons. Drowsiness can also be considered one of the problems through which companies faces failure of the project. Firstly, sleepy workers are less productive, they react more slowly, make more mistakes and possible forget to do things which can lead to adverse impact on the productivity of the particular organization. Also, sleepy employees have worse adaptive performance. This mean that they will not be able to figure out how to handle the changing situation and novel challenges, things which are becoming increasingly common in the modern work place. Additionally, they will have trouble with multitasking or quickly switching between different tasks, also common elements of the modern work places. And according to the fatigue calculator developed by sleep matters initiatives at Brigham health for the national safety council estimated that an average sized fortune 500 company with approximately 52000 employees is losing about $80 million annually due to fatigued employees. This kind of loss may also lead to vanish of the company itself. So, there should be system or software to keep a real time record and alert to the respective person so that the give work is completed on time. Objectives Employee Sleep Detection System (ESDS) have been proposed as specific countermeasures to control the drowsiness associated with employee. Our System will employ a variety of techniques for detecting employee’s drowsiness while working and notify an employee when critical drowsiness levels are reached. However, the detection of employee fatigue using valid, unobtrusive, and objective measures remains a significant challenge. Detection techniques may use ocular or facial characteristics.

Chapter 3 Literature Survey Technological approaches for detecting and monitoring ocular and facial characteristics continue to emerge and many are now in the development, validation testing, or early implementation stages. In this section, some currently available drowsiness monitoring devices, as well as technologies that will be available in the near future are discussed. 3.1 Seeing Machines face LAB Seeing Machines face LAB provides head and face tracking as well as eye, eyelid, and gaze tracking for human subjects using a non-contact, video-based sensor. 3.1.1 Advantages

  1. Face LAB has a flexible and mobile tracking system and a wide field of view that enables analysis of naturalistic behavior, including head pose, gaze direction, and eyelid closure, in real time under real-world conditions without the use of wires, magnets, or headgear. Thus, it is a tool that has great promise for analyzing employee behavior in simulators and test vehicles.
  2. It is not sensitive to sudden movement or obstruction, and it recovers immediately if a subject leaves the field of view. The device reportedly works in bright sunlight or at night, with subjects close to the camera or several feet away. 3.2 LC Technologies, Inc. LC Technologies, Inc. has developed an eye tracking technology that is both an eye operated computer for control and communication and a device for monitoring and recording eye motion and related eye data. 3.2.1 Advantages The goal of the system is to monitor the employee’s eye point-of-regard, saccadic and fixation activity, and percentage eyelid closure reliably and in real time. 3.2.2 Disadvantages In most cases, eye tracking works with eyeglasses and contact lenses since the calibration procedure accounts for the refractive properties of the lenses. However, eyeglasses tilted significantly downward, hard contact lenses, and sunglasses may cause problems for the system (but sunglasses are usually not permitted in office premises).

4.5 OpenCV: OpenCV stands for Open Source Computer Vision. It's an Open Source BSD licensed library that includes hundreds of advanced Computer Vision algorithms that are optimized to use hardware acceleration. OpenCV is commonly used for machine learning, image processing, image manipulation, and much more. OpenCV has a modular structure. There are shared and static libraries and a CV Namespace. In short, OpenCV is used in our application to easily load bitmap files that contain landscaping pictures and perform a blend operation between two pictures so that one picture can be seen in the background of another picture. This image manipulation is easily performed in a few lines of code using OpenCV versus other methods. OpenCV.org is a must if you want to explore and dive deeper into image processing and machine learning in general.

4.5.1 Algorithm:

  1. Image sequence input to camera.
  2. Face detection
  3. Locating eyes
  4. Eye state recognition using Binary pattern method and Edge detection method.
  5. If eyes are closed and continues to be closed for predefine threshold, Drowsiness state is detected
  6. Else normal state
  7. Repeat the process 4 .6 Face Detection: Object Detection using Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, "Rapid Object Detection using a Boosted Cascade of Simple Features" in 2001. It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images. Here we will work with face detection. Initially, the algorithm needs a lot of positive images (images of faces) and negative images (images without faces) to train the classifier. Then we need to extract features from it. For this, Haar features shown in the below image are used. They are just like our convolutional kernel. Each feature is a single value obtained by subtracting sum of pixels under the white rectangle from sum of pixels under the black rectangle. A cascaded Adaboost classifier with the Haar-like features is exploited to find out the face region. First, the compensated image is segmented into numbers of rectangle areas, at any position and scale within the original image. Due to the difference of facial feature, Haar-like feature is efficient for real-time face detection. These can be calculated according to the difference of sum of pixel values within rectangle areas. The features can be represented by the different composition of the black region and white region. A cascaded Adaboost classifier is

where σ is a scale parameter. Then, the magnitude and orientation of the edge are calculated by the differential filter. The final edge image is obtained by edge information of multiple scale σ. Finally, the numbers of edge points are summed for recognizing the eye's state. 4 .7.2 Eye Blink Detection:

  1. Grayscale Conversion: The coloured eye image is first converted to grayscale. Gray scale conversion algorithms use to convert the coloured image to grayscale. Corner Detection Corners are defined as intersection of two edges. We propose an eye blink detection algorithm that uses the two eye corner points and one point at the lower eye lid. To detect these points, Harris Corner Detector has been used. The reason for using Harris Corner Detector is one of the most used corner and interest point detector and is invariant to illumination variation, image noise, scale and rotation. This corner detector makes use of the fact that a corner is simply the point where two edges intersect. In other words, it is the point at which the two edges change direction. The image gradient has an increased variation in both directions, which can be used to detect it. This “variation” is determined by Harris Corner Detector. On applying the Harris Corner Detector on the input eye image, we get the points as indicated in fig.,
  1. Midpoint Calculation A midpoint is defined as the middle or Centre point of a line segment. Once all the required points have been found, the next step would be to find midpoint between the two upper corner points. Let (X1, Y1) be the coordinates of upper left corner and (X2,Y2) be the coordinates of the upper right corner. A line segment is drawn between these two points. The midpoint of this line segment can be calculated using the following formula,
  2. Distance Calculation: Distance is a mathematical description of how far objects are from each other. As next step, we find distance of the midpoint from the point at lower eyelid. In analytic geometry, distance between two or more points is calculated by using the distance formula given by the Pythagorean Theorem. The distance between two points (X1, Y1) and (X2, Y2) is given as: 4)Eye State Determination: Finally, the decision for the eye state is made on the basis of distance ’d’ calculated in the previous step. If the distance is zero or is close to zero, the eye state is classified as “closed” otherwise the eye state is identified as “open”. 4.8 Drowsiness Detection: The last step of the algorithm is to determine the person’s condition on the basis of a pre-set condition for drowsiness. The average blink duration of a person is 100-400 milliseconds. This is 0.1-0.4 of a second. Hence if a person is drowsy his eye closure must be beyond this interval. We set a time frame of 5 seconds. If the eyes remain closed for five or more seconds, drowsiness is detected and alert pop regarding this is triggered.

5.2 Scheduling The following table shows the expected flow of work for the accomplishment of the required result No Description Duration (in weeks) Complexity (out of 10) Status 1 Researching algorithms that aid in fatigue monitoring 1 7 Done 2 Literature Survey of Seeing Machines face LAB 1 5 Done 3 Literature Survey of LC Technologies, Inc. 1 5 Done 4 In-depth understanding and study of python Language along with a Machine Learning Framework known as TensorFlow. 1 7 Done 5 Implementation of the Canny Algorithm for Eye Detection and Image Processing. 1 - Pending 6 Actual implementation of the Drowsiness Detection System 1 - Pending 7 Writing, compiling and execution of the codes necessary for the implementation of the Drowsiness Detection System

1 -^

Pending

References [1] https://ieeexplore.ieee.org/document/ [2] https://sci-hub.tw/https://ieeexplore.ieee.org/document/7396336?reload=true [3] www.google.com [ 4 ] https://www.codeproject.com/Articles/26897/TrackEye-Real-Time-Tracking-Of-Human-Eyes- Using-a [ 5 ] https://realpython.com/face-recognition-with-python/ [ 6 ] https://www.pyimagesearch.com/2017/04/24/eye-blink-detection-opencv-python-dlib/ [ 7 ] https://www.codeproject.com/Articles/26897/TrackEye-Real-Time-Tracking-Of-Human-Eyes- Using-a [ 8 ] https://github.com/tahaemara/sleep-detection/blob/master/README.md [ 9 ] https://ieeexplore.ieee.org/document/7396336?reload=true [ 10 ] https://github.com/mans-men/eye-blink-detection-demo [ 11 ] https://docs.opencv.org/3.4/d7/d8b/tutorial_py_face_detection.html [ 12 ] https://www.learnopencv.com/training-better-haar-lbp-cascade-eye-detector-opencv/ [ 13 ] https://blog.goodaudience.com/real-time-face-and-eyes-detection-with-opencv-54d9ccfee6a [ 14 ] https://www.seeingmachines.com/technology/ [ 15 ] https://eyegaze.com/tag/lc-technologies/