Cyber-Eye and Scented E-mail
August 30, 2012
A group of Gdańsk University of Technology researchers led by Prof. Andrzej Czyżewski are working to develop a range of multimodal computer interfaces for use in education, medicine, defense and industry.
Innovative computer technology used in education and medicine has long been a Polish specialty. Czyżewski, who heads the Gdańsk University of Technology’s Multimedia Systems Department, has a long list of inventions to his name that he has developed over the past 20 years. These inventions are based on the use of information technology to support imperfect human senses—hearing, sight, smell and touch.
“We don’t create things exclusively with a commercial use in mind,” says Czyżewski. He adds that the researchers’ work has benefited a number of nonprofit causes. For example, the researchers have developed a method to examine the level of awareness of comatose patients in a hospice in the city of Toruń. This method is dubbed Cyber-Eye and Czyżewski calls it a revolution in palliative medicine.
The method, the first of its kind in the world, according to Czyżewski, is based on tracing the eye movements of comatose patients using infrared technology. It turns out that many such people have contact with the world around them, Czyżewski says.
As part of the latest project, Czyżewski says the researchers are looking to develop software capable of interpreting a patient’s gestures. The system could help deaf and hearing-impaired people by translating sign language into speech.
The system could also be used by physically disabled people to give instructions to the computer without using the mouse and keyboard.
Other uses are also possible, for example in diagnosing Parkinson’s disease. The idea came from Prof. Jarosław Sławek from the Medical University of Gdańsk, who collaborates with the Multimedia Systems Department.
The gesture-interpreting software created by the researchers would help determine the stage of disease and thus improve the effectiveness of treatment.
Encouraged by Sławek, the researchers have also started to test the efficiency of the sense of smell in patients. The keenness of people’s sense of smell makes it possible to predict the occurrence of symptoms of Parkinson’s disease up to 10 years in advance, according to Sławek.
The gesture-recognition software has been expanded to include an aroma interface. This dual diagnosing tool will make it possible to test the olfactory sensitivity of patients in the same way that their hearing can be tested, says Czyżewski.
As an experiment, the researchers are sending “fragrant files” by email to 50 recipients equipped with special software. They have also designed special applications in the form of interactive boards that allow children with ADHD syndrome to focus their attention. Fifty such demonstration interfaces have already been sent to schools across Poland, Czyżewski says.
The researchers’ multimodal interfaces also make it possible to examine and treat binocular vision disorders, which are routinely diagnosed in about 12 percent of school students.
To be effective in what they do, people need to use their senses symmetrically in terms of how both their eyes, ears and nostrils work, Czyżewski says. Those unable to use their senses symmetrically lack so-called stereoscopic fusion. Some of these patients cannot see three-dimensional images or see them poorly. The results can be tragic—for example when someone is driving a car or working at heights.
Until now, doctors used an expensive device called synoptophor to examine patients with this disorder. The method developed by Czyżewski’s team offers new opportunities. By showing patients simple exercises, it teaches them to see spatially.
Apart from Czyżewski, several other researchers are involved in work to develop and implement a range of multimodal computer interfaces. Michał Lech is dealing with gestures, Łukasz Kosikowski is handling sight, and Katarzyna Kaszuba is focusing on cerebral hemispheres. Czyżewski oversees the project as a whole.
An accomplished innovator, Czyżewski has won a number of prestigious awards, including the Man of the Year in Science title from the American Biographical Institute in 2009. At the Concours Lepine 2010 trade show in Paris, he got a medal for his system for examining and treating binocular vision disorders and for his Virtual Touchpad smart monitoring system. He has also won prizes at the Poznań International Fair, including a gold medal for transferring his research results to business practice.
Czyżewski also won gold medals at last year’s Technicon Innowacje 2011 trade fair in Gdańsk and at the International Eureka Contest held as part of the Brussels Innova exhibition. The medals were for his system for improving the hearing of patients with disorders of the central nervous system.
This last invention has made it onto a list of key information and communication technology projects, and Czyżewski got zl.7.2 million for carrying out the project from the European Union’s Innovative Economy Operational Program. The funds were granted for five years.
The researchers plan to create what they call 50 technology demonstrators and at least 50 publications connected with the project. A total of 250 devices will be produced for tests in a number of areas. The devices will be tested by psychological and pedagogical centers, schools, healthcare institutions, and other users.
Several devices related to the project have already hit the market. These include a smart pen and an aid for people with a stutter. They were designed thanks to cooperation between Gdańsk University of Technology researchers and the Young Digital Planet company, Poland’s largest producer and publisher of educational computer programs.
Another product awaiting its market launch is a device that slows speech to help people who have problems understanding others when they speak fast. Hearing is not only about the ear but also about the brain, which is where the problem may lie, Czyżewski says. People of all ages may have problems understanding fast speech. Human speech contains many unnecessary sounds, Czyżewski adds. These can be removed using digital technology, and the desired message will reach the listener in real time, slower and understandable. “It’s enough to slip on a pair of small earphones and connect them to a computer with our application,” Czyżewski says.
The researchers have already found a producer at the Gdańsk Science and Technology Park, with which the Gdańsk University of Technology is in close contact. The producer has constructed a miniature—cell phone-sized—computer to work with applications for advanced sound processing. Work to upgrade the device is under way.