A Brief Description
A Brief Description
How it works?
Detailed Circuit Diagram
Detailed Pin Diagram
Group Members Information
In this project we have designed and implemented a gesture responsive mouse(GRM). It simply tracks the movement of human hand and moves the cursor accordingly. What makes our project interesting is that we have used sonar sensors to track the movement of the human hand.
HC-SR04 Ultrasonic Sensor
In our GRM(Gesture Responsive Mouse) we have used eight Ultrasonic sensors. Four sensors are placed horizontally to track the vertical movement of
the human hand and three sensors are placed vertically to track the horizontal movement of the human hand. All these sensors can update
the distance of an object in front of them in real time. We used this information to keep track of the position of the human hand and
receive the data using an Arduino Mega. We then sent the data to the computer through an usb cable to the serial port of the computer.
In the computer a python script is run. This script uses two python libraries "PySerial" and "PyAutoGUI". With the help of pyserial
python reads the data from the serial port of the computer. And using the functions of pyautogui the mouse cursor is moved accordingly.
The eighth sensor is used to generate the click. We implemented two click options. "Double Click" and "Right Click". If the hand is placed < 8 cm then right click is generated. If hand is placed between 8 to 15 cm then double click is generated.
fig.1 Block Diagram
2. Collect Realtime data of the hand gesture.
3. Apply necessary filtering to reduce the noise and fluctuation in the reading.
4. Send the data to the computer using USB serial port.
5. Read the data in the laptop using python script.
6. Move the mouse or generate click cursor according to the reading.
7. Go to step (2)
Fig.2 Circuit Diagram (Gesture Responsive Mouse)
fig.3 Pin Diagram for Gesture Responsive Mouse
Mouse is a very movement sensetive device. Its very fast and has to be accurate. While mouse is put on top of a mouse pad or a table its movement is very controlled and its really easy to move around without any shaking. But one of the major problem that we faced in our project is that, in our project humand hand is suppose to be moving in front of the sensors. As there are no rigid support human hand shakes a lot. In this natural shaking of hand the mouse cursor moves very abruptly. To reduce the abrupt change in reading we had to implement many level of filtering. We only took the values of the mouse movement in a specific direction if two consecutive values had the same directional flag.
Even in arduino the rate of reading data is very fast with respect to hand movement. So we put a 3ms (Mili Second) delay between each reading of the loop() function. This caused the mouse to respond a little slow but it reduced the abrupt movement of the cursor to a great extent.
As,sonar sensors produce sound waves in funnel shape and measure the object distance from the echo of sound. It produces the echo in a 15 degree angle, not as a straight line. So there were many overlapping region where the may have caused extra noise.
When the arduino sent data using the serial port, it sent the data as a byte stream. So with our data there were also carriage returns and newline carecters. So we first read the data as a byte stream, converted it into UTF-8 string and then chopped of all the newline and carriage return from the string. Then strip the "x" and "y" coordinate of the hand, convert it to float and moved the cursor accordingly.
There was a small amount of noise in the reading of the Ultrasonic sensor itself. Even if an object was set still in front of it, the sensors reading would change very slightly. Between 0.1 to 0.5 cm. But as 1 cm is equal to almost 38 pixel, a small amount of fluctuation in reading caused the mouse cursor to move very unpredictably.
Section A2: Group 5
M. Z. Ferdous (1205081)
Md. Safiullah Sabuj (1205085)
Muhit Sarwar (1205097)
Mujahid Al Rafi (1205103)
Tarhib Al Azad (1205105)
Mohammad Saifur Rahman
Md. Aashikur Rahman Azim
Md. Iftekharul Islam Sakib
Back to top