News

Rewards program helps robots think for themselves



14 October 2013




Intelligent machines are a favourite subject for science fiction writers, who have mined into the rich potential for mayhem when robots with brains rebel against their human masters.

In real life, robots are often assigned mundane tasks that are too boring or dangerous for human hands. But Fabio Ramos is helping to produce a new breed of intelligent robots with applications more ambitious than vacuuming the floor or dealing with unexploded bombs.

A Brazilian graduate from the University of Sao Paulo, Dr Ramos is now a Senior Lecturer and an Australian Research Council fellow at the School of Information Technologies and Australian Centre for Field Robotics. Based in the area of robotics and artificial intelligence, he writes programs that effectively train machines how to learn for themselves.

"We devise methods that allow machines to learn on their own, based on their interaction with the environment," he explains.

"They are motivated by little rewards that we write into the program. For example, if a robot takes an action that makes it bump into a chair, it starts to learn how to avoid chairs next time."

'We devise methods that allow machines to learn on their own, based on their interaction with the environment' (Photo: Ted Sealey)
'We devise methods that allow machines to learn on their own, based on their interaction with the environment' (Photo: Ted Sealey)

The robots' capacity for self-learning means that they can start to take on tasks of wider significance. For example, Dr Ramos has helped to develop a blimp that can automatically log data about air quality in the Hunter Valley.

He explains: "Robots follow a process. They scan the environment, collect data, process information and take appropriate action based on that information. On pollution monitoring problems, the robots learn the best areas to monitor pollution such as near train lines or coal mines.

"For every research project I undertake, I always have an application in mind."

The machine learning process involves an area of statistics called data fusion that seeks out relationships between data sets.

Another area of his work involves big data and geothermal exploration. Located five kilometres underground, geothermal rocks are a good source of renewable energy. But there is a catch - drilling operations cost up to $20 million each time.

Enter Dr Ramos and his team. Using data fusion, they help to accurately predict the most likely geothermal targets.

He explains: "There's a lot of data in Australia about what's underneath the earth," Dr Ramos says. "With clusters of computers that analyse data from the entire continent, we use a complex statistical model that attempts to put all sources of information together to minimise the risk involved in drilling."

While robots are not yet generally used for geothermal exploration, Dr Ramos hopes that they will be one day. "This technology could solve our energy problems," he says. And there's another useful side-product: it could also be used to find gold, diamonds and oil.

Dr Ramos adds that data fusion and prediction could even be used to formulate national policies. "The techniques that our machines use to analyse big data can be applied in politics," he says. "This way, governments can exclude extraneous factors and determine the best policies to implement and minimise political risk."

His work continues to strengthen the research relationship between the University of Sydney and Latin America and has attracted interest from the Aeronautics Institute of Technology in Sao Paulo, which is also interested in the processes used to monitor pollution.

Dr Ramos came to Australia from South America in 2003, attracted by the state-of-art robotics lab at the University of Sydney. Despite the distance between them, the two continents share similarities that appeal to Dr Ramos. He says: "The lifestyles are similar, but more importantly, higher education here is equally research-oriented."

Dr Fabio Ramos is helping to produce a new breed of intelligent robots (Photo: Ted Sealey)
Dr Fabio Ramos is helping to produce a new breed of intelligent robots (Photo: Ted Sealey)


Contact: Richard North

Phone: 02 9351 3191

Email: 1010021f321e4b5b1b3c474723311f0916283c3d67262f16412342