TEAM 12: HandTalk Team
Spring 2008
MEMBERS
PROJECT CONCEPT
HandTalk is a wireless glove designed to recognize basic American Sign Language (ASL) hand signs and voice them by interfacing with a Java enabled cellphone/PDA.
MOTIVATION
We feel that HandTalk could greatly aid the deaf more comfortably communicate with those unfamiliar with ASL.
HANDTALK V0.1 DEMO
Checkout our first end to end test for our glove. It shows the result for an input to our system from beginning to end. More features are yet to come.
COMPETITIVE ANALYSIS
- KMI Embedded RF Finger System
- Similar to our system
- Uses RF to communicate
- Not very portable
- Demo video here
|
- Metamotion Data Glove
- Very precise
- Very expensive
- Not portable
- Website
|
![](images/metamotion.jpg) |
- Berkeley Acceleration Sensing Glove
- Not portable
- Uses 3D Accelerometers
- Website
|
![](images/berkeley.jpg) |
TECHNICAL SPECIFICATIONS
Hardware:
- Flex Sensor x 10 ($10 each): Jameco Electronics -
Odeg:1OK
- Nokia N95
- BlueSentry Bluetooth Data Acquisition Module from Roving Networks
Software:
- Nuance RealSpeak Text to Speech Engine ($10)
- TALKS - for interim use
Protocols:
- Wireless: Bluetooth for glove to communicate with the cellphone/PDA
REQUIREMENTS
- Glove needs to recognize the ASL alphabet
- Glove needs to be portable and have low power consumption
- Glove needs to have a simple mechanism to switch between the ready and standby state
- System needs to recognize and voice sign within 2 seconds
- The support application on the cellphone/PDA needs to be compatible with most cellphones/PDAs currently on the market
ARCHITECTURE
USE CASES (INTERACTION DIAGRAMS)
SYSTEM STATES & TRANSITIONS
To be completed.
RISKS & MITIGATION STRATEGIES
Risks |
Mitigation |
Bluetooth is tough to implement |
Module has complete implementation on-board, so there's no real coding required |
Possible low resolution from the sensors + A2D |
- Use our own alphabet/gestures - possibly change application
- Expand the voltage range by looking at more complicated analog circuit setups - we're using a simple voltage divider
|
Fewer than ideal number of I/O pins (8 vs. 10+) |
- 2 flex sensors for each of 3 fingers + 1 sensor for 2 fingers - we can only decide which through user testing
- Use 3D accelerometer + 1 flex sensor per finger - involves rethinking of application
|
ERROR HANDLING
To be completed.
IMPLEMENTATION DETAILS
To be completed.
TEST CASES
- Bluesentry was connected and we were streaming data when the connection was suddenly lost.
- User should be notified and we should go back to the connections page.
- Bluesentry was connected but suddenly dies.
- Notify user and stop program. User can press exit or start BlueSentry again and try
reconnecting. Go back to Discoverer page.
- SD Card testing -
- Tried to load a file that was in an invalid format/wrong file.
Action - Die gracefully.
- User forgets to save a file/overwriting a file? (currently throws exception)
- Different dictionaries - same signature in two different libraries might mean separate things. The data outputted
should not be corrupted.
- Specifying file path - check the path. (have SD card path hardcoded and then just append user.
- User Input fields - trying to screw up all input fields. Check if bluetooth address is not
the BlueSentry device.
- Learning interface - check if they are adding new words/conflicts and notify appropriately.
- START-STOP-START cycle of the connection handler with our bluetooth device should work properly.
- Red button of death - deal with finding the function that would be called in that scenario.
- Check for Memory Leaks
EXPERIMENTAL EVALUATION
To be completed.
LESSONS LEARNED
To be completed.
FUN STUFF
To be completed.
REFERENCES
- Project Proposal and Requirements,
Team Project Presentation, February 1, 2008
- Design & Architecture,
Team Project Presentation, February 15, 2008
- Mid-semester Project Status,
Team Project Presentation, Date TBD
- Test Plan & Experimental Validation
Team Project Presentation, March 28, 2008
- Final Project Presentation,
Team Project Presentation, Date TBD
- Project Poster,
May 2, 2008
Back to the top of this page
18-549 course home page