How It Works: Finger Sign Recognition Systems
Modern technology is rapidly altering and enhancing all aspects of our lives. New developments continue to remove barriers for people who are speech and hearing impaired. By using artificial intelligence, researchers continue to develop both hardware and software that improves the way deaf individuals learn and communicate. Sign language can now be captured, recognized and translated to speech for the benefit of blind people. Research also continues to analyze speech and convert it to either sign or text on a computer screen for the hearing impaired.
Other People Are Reading
-
Artificial Intelligence
-
Modern research has seen the development of computers that can recognize sign language. Such projects as CyberGloves, which incorporates virtual reality sensors to capture both isolated and continuous hand signs, and the Markov Modeling and neural network systems, have made a significant impact in this regard. The ICICLE (Interactive Computer Identification and Correction of Language Errors) project uses intelligent computer-aided instruction to build a tutorial system for hard-of-hearing or deaf children. ICICLE analyzes their English writing and makes tailored lessons and recommendations for these learners.
Select Challenges
-
Sign languages are not international, and a number of countries use their own unique sign languages. American Sign Language (ASL), for example, is not simply a visual representation of English, but has its own specific rules and grammar. The sign system, Signed English, however, makes use of ASL signs but uses them in English order. It is easier, however, to develop software for this transliteration process than for natural language processing (NLP) that is necessary for translating between ASL and English. Signing is a duel process and requires both the ability or skill to read signs and the skill to make or render signs. Despite the considerable achievements obtained with artificial intelligence, more progress has been made regarding computers rendering signs as opposed to them reading signs.
-
Capturing Signs
-
By using a capture glove, the computer is able to track the signs an individual makes with their fingers. Through software, these signals are expressed in written or spoken language. Such a glove, designed by Ryan Patterson in 2002, sensed the hand movements of the sign language alphabet and wirelessly transmitted the data to a portable device, from where it was displayed on a computer screen. A more modern glove, the CyberGlove, incorporates between 18 and 22 sensors and is connected to a computer through a serial cable, while the VPL Data Glove contains fiber optic transducers that measure finger flex angles.
Computer Vision
-
Computers and cameras are also being used to capture hand movements and thereby digitize signs. In 1992, a camera was designed that could capture finger signs from a glove with markings on the tip of each finger. Two years later, this camera was modified to capture a ring of color around each joint on the signer's hand. More recently, a camera vision system is used together with wireless accelerometers that have been mounted in a bracelet or watch to measure hand rotation and movement.
-
References
Resources
- Photo Credit wegweiser finger image by Jörg Jahn from Fotolia.com
