Student developing way for the blind to play ping pong using sound
Object tracking cameras combined with speakers provide real-time audio feedback in three dimensions, creating an acoustic "image”. Image: Phoebe Peng
Working with ARIA Research, a startup co-led by the University of Sydney and the University of Technology Sydney which is developing vision for the blind, Phoebe Peng is using technologically advanced human echolocation that enables blind users to perceive their surroundings in precise detail.
The process uses event cameras which, unlike normal cameras that capture complete images of a scene, track changes in an image over time, making them ideal for the observation of small objects like table tennis balls. The images are then processed into sound using a specialised algorithm. This is then communicated back to the player via an array of loudspeakers, ultimately with the aim of allowing players to track the ball and movements acoustically.
“Table tennis or ping pong has been played for decades as a more accessible version of tennis. The sport is particularly beginner-friendly while maintaining a rich level of competitive play. However, like many sports, it remains inaccessible for people who are blind or have low vision,” said Ms Peng, who will soon complete a Bachelor of Engineering (Honours) in Software Engineering, and is completing her Honours thesis under the supervision of Professor Craig Jin from the School of Electrical and Computer Engineering.
A researcher testing out the system. Credit: Phoebe Peng
According to Peng, table tennis makes a perfect test case for the kind of technology being developed by ARIA Research.
“The small size of the ball and table, along with the movement of the ball in 3D space, are things that make table tennis difficult to play for those with low vision and complete blindness,” said Peng, who is completing the work as part of her Honours’ thesis. “Making this sport more accessible while also exploring the potential of event cameras were my two biggest motivators,” she said.
In one study, using two perfectly positioned cameras, Ms Peng was able to identify and track a ball in three dimensions in real time. She then fed that data into an algorithm controlling an array of loudspeakers along the sides of the table, which created a sound field matching the position of the ball.
While the results are promising, more experimentation will be needed before the system will be ready for actual play.
Researchers in the lab, testing out the system. Image: Phoebe Peng
“An ongoing technical challenge is the matter of human perception of sound,” said Ms Peng. “There are limitations on how accurately people can perceive sound localisation. What type of sound should be used? Should the sound be continuous? This is what we’ll be tackling in the next stage of development.”
Over the past three years, the ARIA Research project has been developed in collaboration with the University of Sydney and University of Technology Sydney’s faculties of engineering, Blind Citizens Australia and World Access for the Blind Australia, with the support of the Australian Federal Government.
Ms Peng presented her work as part of Acoustics 2023 Sydney conference at the International Convention Centre Sydney.
