Gesture Responsive Mouse(GRM)







A Brief Description

Required Equipments

How it works?

Block Diagram

Algorithm

Detailed Circuit Diagram

Detailed Pin Diagram

Problems Faced

Necessary Links

Group Members Information

Course Teacher






A Brief Description


In this project we have designed and implemented a gesture responsive mouse(GRM). It simply tracks the movement of human hand and moves the cursor accordingly. What makes our project interesting is that we have used sonar sensors to track the movement of the human hand.






Required Equipments


  1. Arduino Mega

  2. HC-SR04 Ultrasonic Sensor

  3. Breadboard






How it works?


In our GRM(Gesture Responsive Mouse) we have used eight Ultrasonic sensors. Four sensors are placed horizontally to track the vertical movement of the human hand and three sensors are placed vertically to track the horizontal movement of the human hand. All these sensors can update the distance of an object in front of them in real time. We used this information to keep track of the position of the human hand and receive the data using an Arduino Mega. We then sent the data to the computer through an usb cable to the serial port of the computer. In the computer a python script is run. This script uses two python libraries "PySerial" and "PyAutoGUI". With the help of pyserial python reads the data from the serial port of the computer. And using the functions of pyautogui the mouse cursor is moved accordingly.
The eighth sensor is used to generate the click. We implemented two click options. "Double Click" and "Right Click". If the hand is placed < 8 cm then right click is generated. If hand is placed between 8 to 15 cm then double click is generated.




Block Diagram




                                                                       pin diagram

                                                                                                                                               fig.1 Block Diagram






Algorithm




       1.Start
       2. Collect Realtime data of the hand gesture.
       3. Apply necessary filtering to reduce the noise and fluctuation in the reading.
       4. Send the data to the computer using USB serial port.
       5. Read the data in the laptop using python script.
       6. Move the mouse or generate click cursor according to the reading.
       7. Go to step (2)
                            






Detailed Circuit Diagram




                                                                       pin diagram

                                                                                                                             Fig.2 Circuit Diagram (Gesture Responsive Mouse)




Detailed Pin Diagram




                                                                       pin diagram

                                                                                                                           fig.3 Pin Diagram for Gesture Responsive Mouse






Problems Faced


  1. Mouse is a very movement sensetive device. Its very fast and has to be accurate. While mouse is put on top of a mouse pad or a table its movement is very controlled and its really easy to move around without any shaking. But one of the major problem that we faced in our project is that, in our project humand hand is suppose to be moving in front of the sensors. As there are no rigid support human hand shakes a lot. In this natural shaking of hand the mouse cursor moves very abruptly. To reduce the abrupt change in reading we had to implement many level of filtering. We only took the values of the mouse movement in a specific direction if two consecutive values had the same directional flag.

  2. Even in arduino the rate of reading data is very fast with respect to hand movement. So we put a 3ms (Mili Second) delay between each reading of the loop() function. This caused the mouse to respond a little slow but it reduced the abrupt movement of the cursor to a great extent.

  3. As,sonar sensors produce sound waves in funnel shape and measure the object distance from the echo of sound. It produces the echo in a 15 degree angle, not as a straight line. So there were many overlapping region where the may have caused extra noise.

  4. When the arduino sent data using the serial port, it sent the data as a byte stream. So with our data there were also carriage returns and newline carecters. So we first read the data as a byte stream, converted it into UTF-8 string and then chopped of all the newline and carriage return from the string. Then strip the "x" and "y" coordinate of the hand, convert it to float and moved the cursor accordingly.

  5. There was a small amount of noise in the reading of the Ultrasonic sensor itself. Even if an object was set still in front of it, the sensors reading would change very slightly. Between 0.1 to 0.5 cm. But as 1 cm is equal to almost 38 pixel, a small amount of fluctuation in reading caused the mouse cursor to move very unpredictably.










Necessary Links


Youtube Video






Group Members Information


Section A2: Group 5






Course Teachers


  1. Mohammad Saifur Rahman

  2. Md. Aashikur Rahman Azim

  3. Md. Iftekharul Islam Sakib





Back to top