A Brief Description
A Brief Description
How it works?
Detailed Pin Diagram
Group Members Information
In this Project, we have designed an embedded system which converts sign languages to sound forms in order to establish communication with the outside
world for the people who are deaf, people who can hear but cannot physically speak, or have troubles with spoken languages due to some other disabilities.
A sign language is a language which majorly used for mutual communication to express meaning, as opposed to acoustic sound patterns. Sign language involves simultaneously combining hand shapes, orientation and movement of the hands, arms or body, and facial expressions to express a speaker's thoughts.
A common misconception is that all sign languages are the same worldwide or that sign language is international. Each country generally has its own, native sign language, and some have more than one. In out project we have interpreted 17 common moves in Bengali signed languages and necessary resources are collected from Center for Disablity in Development.
Arduino Mega 2560
MPU 6050 ( Accelerometer, Gyroscope, Temperature sensor )
SD Card Adapter
LM 386 Audio Amplifier
8 ohm Speaker
5V Fixed Positive Power Supply Regulator 7805
16 * 2 LCD
Sign language involves the orientation and movements of hand. So to tracking the condition of our hand we use gyroscope, for the condition of finger we use flex sensor.
We integrate the 2 components with a hand sock rigidly
so that wearing the hand sock when our hand is moved or our finder is bent flex sensor and gyroscope face some effects.
Flex sensor is a variable resistance. When it is bent the resistance is increased. The resistance will be increased in proportion to how much it is bent. Measuring the resistance we can know the condition of our fingers.
Using Accelerometer and Gyroscope we can track the transitional and rotational motion respectively along 3 axis. Both Accelerometer and Gyroscope has 3 registers. For example, if we move the hand along positive x - axis accelerometer will give positive value and if we move the hand along negative x - axis it will give negative value in the corresponding register. The higher the acceleration we will get more value. For rotational movement gyroscope can detect clockwise and anti-clockwise rotation and output positive or negative value according to angular velocity.
We store some audio files in the SD card. The audio files are saved in .wav format. The files will be played through arduino analog output pin and speaker. The volume of the sound is very low so we use amplifier to amplify the sound wave. The file will be played to express what the disabled man want to express. For playing .wav files we include TMRpcm library in arduino. To convert .M4A file to .wav we use the online converter audio.online-convert
When arduino senses some output from accelerometer ans gyroscope it will match the movement and orientation of our hand and finger with the movement and orientation for 17 moves which we are implemented. If it matches with a move in a significant threshold arduino will play the corresponding audio files from memory and display the move on LCD. Otherwise it will say invalid move.
2. MPU6050 is initialized
3. SD card presence is checked
4. Necessary variables and interrupt are initialized
5. Flex sensors values are read and reads are encoded
6. Time count started
7. If user is ready
8. If 2sec is not over
9. If data is taken threshold times goto
10. Data is taken from MPU6050 when device has enough data
11. Data is checked against some criteria and either accepted and encoded or discarded
12. Time count is taken again to see if 2sec is over, if not over goto 7
13. Taken data are matched against reference movement data to find the best match
14. If match found
15. Word corresponding given sign is printed on lcd
16. Word corresponding given sign is played on speaker
18. Lcd print indicates there is an invalid move
19. Else go to (5)
Arduino only supports .wav audio file. To convert .M4A to .wav and to get the best quality we have tuned the convertion parameters such as sampling rate, bit resolution, pcm format. Arduino can't play audio files of sampling rate above 16000 hz or less 1000Hz. Bit resolution must be 8 bit. In conclusion we found that, for 11025 Hz sampling rate, 8 bit resolution we get the best quality.
Noise is a common phenomena while playing audio files. To resolve the problem, we tried several methods.
While playing a .wav file first we get a high frequency sound. This sound made our project bitter. Using a low pass filter, this situation was improved.
Adjusting the brightness of lcd was another problem. Firstly, we supply the voltage using resistor and consequently the brightness was very low. Then using potentiometer ( variable resistance ) we tuned the supply voltage and the problem was solved.
Some simple sign were not implemented because reference movement is same as another sign. This could be solved if more flex sensors could be used.
At first raw data from accelerometer and gyroscope (MPU6050) was read. Due to gravity, accelerometer data was not suitable for use. Later, effect of gravity was removed from the accelerations along each axis using a library.
Different threshold values are used throughout the program. They are set from experiments. So, the system do not perform correctly all time but trial by different person shows it performs correctly almost all the time.
Wires used to connect the gloves with arduino were not flexible enough. So, hand movement was not free.
Section A1: Group 4
Md. Rashedul Islam (1205013)
Mujtahid Al-Islam Akon (1205017)
Jaiaid Mobin (1205019)
Md. Touhidul Islam (1205031)
Md. Mohimenul Kabir (1205045)
Md. Aashikur Rahman Azim
Md. Iftekharul Islam Sakib
Md. Tarikul Islam Papon
Back to top