AI can help machines and people grow together Tracing the growth of FORPHEUS—from a robot capable of playing a table tennis rally to an AI-equipped robot that grows together with people

FORPHEUS keeps evolving every year as a state-of-the-art machine that presages a future in which people and machines live and work in harmony. Why was this robot created? What kind of evolution will he show this year? To answer these questions, the leaders who developed each generation of FORPHEUS will share the whole picture of its development.
This final episode features the third generation FORPHEUS unveiled at CEATEC JAPAN 2016. We'll examine how the third generation FORPHEUS evolved by adopting deep learning, an artificial intelligence (AI) technology, and the key to its adoption. The advancement of FORPHEUS will continue, as the human-machine relationship undergoes a continuous and unending evolution.

The third generation FORPHEUS lets us take an advance look at the future in which people and machines can grow together. Taku Oya, the leader of the third generation FORPHEUS development team, shared its improvement points and difficulties involved during the development process.

121-001.jpg

The previous second generation FORPHEUS served as a coach that could draw out the capabilities of the human player. However, OMRON perceives that the harmony between humans and machines that we aim for should enable mutual growth. The new FORPHEUS includes technology that enables people to inspire the growth of the machine. FORPHEUS didn't have this in the past, so it's a major improvement point.
Specifically, the third generation FORPHEUS makes use of deep learning, an AI technology. To study this technology, we invited a large group of table tennis players, including both advanced players and beginners, to OMRON's Keihanna Technology Innovation Center. Each played a rally with FORPHEUS for three minutes, from which we gathered various data. As a result, we accumulated rally data from more than 100 people and incorporated that data into the deep learning model.
One thing we discovered while allowing FORPHEUS to learn was that players whose head and shoulders did not move much could continue fast rallies. FORPHEUS can now detect not only the speed of the ball but the player's facial expressions and skeleton information to determine his/her body movements during a rally. By so doing, FORPHEUS judges the degree of harmonization with the player. When a player has improved his/her skills while enjoying a continuing rally, FORPHEUS also changes the level of return shots from "easy" to "normal" then to "hard" to match the player's skill.

121-002.jpgRecognizing skeleton information

For the application of deep learning, we sought the advice of Associate Professor Takayoshi Yamashita of Chubu University, a renowned expert in this area. We also employed OMRON's AI-based driver monitoring technology.
When playing table tennis, a rally may sometimes be interrupted with the ball rolling on the floor. This data must be removed, which is in fact the most difficult part of data collection. If data unrelated to the rally is incorporated into the deep learning model, the model learns erroneously. It takes a long time and tremendous effort to remove this unnecessary data. Following the previous generation, the technology to make what the machine is thinking understandable, as well as its ability to motivate people, have also been improved further.
There is an OMRON technology called "OKAO," which detects facial expressions. Using this technology, FORPHEUS monitors the player's facial expression during a rally. When a smile appears on the player's face, FORPHEUS takes it as a sign that harmonization with the player is gradually improving. This makes FORPHEUS happy, and lets it motivate the player to continue the rally.
To express FORPHEUS's emotions, the colors of its head or body change or the harmonization gauge on the table tennis net reads full. FORPHEUS communicates with the player through this feedback to motivate one another.

121-003.jpg

FORPHEUS changes its expression when the degree of harmonization increases

Continuing a rally by itself will not help promote harmony between FORPHEUS and the player. So, FORPHEUS also interprets how the player feels from his/her facial expressions while also determining the improvement of the player's rally skills.
Our ongoing development focuses on allowing both people and machines to have a common goal of promoting harmony between them. This in turn will allow both the person and the machine to achieve mutual growth.
Technology to realize harmony between humans and machines is now being perfected in order to unleash the possibilities of people. We hope more and more people around the world will feel through experience that such an exciting future is within their reach. We are also seeking to collaborate with partners around the world to promote the development of technologies capable of solving social issues, which need to be addressed to make this exciting future a reality.
FORPHEUS was not created merely for playing table tennis. It shows the way to reach a future in which machines can bring out the full potential and maximum performance of people. The relationship between people and machines will change dramatically in the upcoming ten years or so. It is our hope that FORPHEUS will provide clues for contemplating the human-machine relationship that awaits us in the future.
We will continue taking on the unending challenges of creating a better future for all.