August 24, 2020

A Big Step Towards the Future Where Machines Can Unlock Human Potential

-The 6th Generation FORPHEUS Motivates Players with AI

OMRON's vision of the future of man-machine interaction is the "harmony between humans and machines", where machines bring out human abilities and creativity to the fullest extent. The table tennis robot FORPHEUS, which embodies this vision, was born in 2013 as a utilization of OMRON's core technology "Sensing & Control + Think". It has continued to evolve with the concept for each generation, and in 2020, we have reached the 6th generation of the robot.

fprpheus_02.jpg
Evolution of FORPHEUS from the 1st to 6th Generation

In addition to the personal coaching function in the 5th generation to improve players' skills, the 6th generation FORPHEUS challenged to develop an AI that would increase players' motivation. By collaborating with a company in a different industry, we worked to create a machine that "further brings out motivation to accelerate player's growth".

The Challenge for Technologies to Accelerate Players' Growth

Satoshi Yase, a member of OMRON's Technology and Intellectual Property Division, who has served as project leader for two consecutive years since the 5th generation FORPHEUS, says that the development of the 6th generation focused on "improving player's motivation to grow" and got a lot of work on interactions where the machine accelerates players' growth. In the 5th generation, the hardware and algorithms have been thoroughly improved, giving the machine a high level of table tennis skill that allows for fast rallies with experienced players. It has also evolved to the point where it is possible to provide "personal coaching," which encourages players to improve their skills by giving them advice and ball returns tailored to the skill level of their opponents. The 6th generation challenged to accelerate players' skill development from a mental perspective.

開発メンバー.jpg

The 6th Generation FORPHEUS Development Team Members (The Right Most Person is Yase)

A Meta-AI that Generates a "rallying plan" that Infers Emotions, Fortifies and Maintains Players' Motivation

The key to accelerating a player's growth is to maintain a high level of motivation. In sports psychology, the state of total immersion in play is called "flow" and the state of extreme concentration is called the "zone", and athletes who reach these states of mind are at their greatest ability. It takes a high level of motivation to reach this flow & zone.

Now, to motivate players, it is necessary to know the mental state of players in real time and plan rallies accordingly. For example, a player who is confused because the challenge is too difficult should be treated with an easy ball. And a player who is bored because the challenge is too easy should be treated with a tough ball. In other words, you have to read the emotions of those players and give the most appropriate response at the right time.

That's why we turned our attention to the "Meta-AI" developed by a game company SQUARE ENIX. The Meta-AI monitors the entire game while guessing the player's psychology and initiates events at the right time. For example, it can tell an enemy character to deliberately show an opening, cleverly avoid an attack, or launch a powerful attack. By constantly adjusting the balance of the game, the player becomes more engaged and more immersed in the game, and that's what we call "hooked".

This is how SQUARE ENIX and OMRON teamed up to develop the 6th generation FORPHEUS. The two companies discussed the characteristics of the game of table tennis and how it could be handled by Meta-AI. In a joint workshop, we tested our hypothesis and confirmed actual emotional changes while actually playing table tennis. OMRON utilizes its proprietary people-measurement technology to estimate a player's skills and emotions in real time and understand his or her state of mind by using cameras to sense vital data such as facial expressions, gaze and heartbeat. Based on these results, the Meta-AI analyzes how to motivate players' emotions and translates them into rally plans. It then repeats the process of returning the ball accurately by changing the course, pitch type and speed according to these rally plans.

This 6th generation FORPHEUS, which enables motivational interactions, was first unveiled at the Consumer Electronics Show (CES) 2020, one of the world's largest consumer electronics trade shows, in January 2020.

The Arrival of the Future, Harmony between Human and Machine

Project leader Yase says, "The AI that we have developed here will be able to work with machines that match human's psychological state. 'The AI that boots motivation', which has been developed in this project, will enable machines to work in accordance with human's psychological conditions. This can be applied to a variety of social situations. For example, it can be applied to technical training and work support in the field of manufacturing in factories, and to motivate elderly people with reduced physical function and physical strength in the medical and nursing care fields during their rehabilitation process".

fprpheus_04.jpg
Scenes of the Future Usages (Expected Situations)

FORPHEUS will continue to evolve towards a future where machines and human continue to improve and grow together.

fprpheus_05.jpg

Members Working for New Development