Robots are gradually displaying behaviors similar to humans, but their previously unnatural smiles during interactions often prompted repulsion rather than warmth. However, with the introduction of a new type of robot, Emo, this situation has seen a significant improvement. Emo possesses advanced facial expression analysis capabilities, able to predict the expressions that will appear 0.839 seconds before humans display them. This results in more natural interactions, avoiding awkwardness and fear.
Currently, most robots only react after humans show an expression, a delay that makes them appear insincere. In reality, non-verbal communication such as smiling plays a crucial role in human interactions. Yuhang Hu, a doctoral student at the Creative Machines Lab at Columbia University says: “Accurately predicting human facial expressions would be a breakthrough in the field of human-robot interaction.”
The team Yuhang Hu belongs to had developed Eva in 2021, a robot capable of mimicking human facial expressions. Emo has built upon this and made further progress, achieving real-time prediction and mimicry of human expressions.
To accomplish this, researchers first needed to create a robot facial skin capable of simulating real human facial muscle movements. They then trained the robot with artificial intelligence to learn the nuances of human expressions and to respond swiftly. Emo represents a significant hardware advancement, with 26 independent actuators in comparison to Eva’s 10, these actuators can more accurately control facial movements and produce a richer and more natural expression simulation.
Emo’s eyes are equipped with high-resolution RGB cameras, emulating the human eye’s ability to perceive colors, not only capturing accurate visual information, but also tracking the gaze of its interlocutors. The camera captures red, green, and blue color information, processes it comprehensively for various data analyses.
The robot’s facial skin is made of soft silicone, secured onto the hardware structure with 30 magnets, allowing for precise simulation of human facial expressions.
The robot’s ability to present itself in different appearances is thanks to its unique magnet connection design. This design not only provides the robot with a variety of easily interchangeable “skins” but also greatly facilitates its maintenance and care. Here, we can see Emo’s interchangeable facial “skins,” displaying a variety of color changes, including blue, green, and purple.
Columbia University Professor of Mechanical Engineering Hod Lipson explains why the robot’s face adopts a blue color. This is to avoid the psychological phenomenon known as the “uncanny valley.” The uncanny valley theory suggests that as robots look increasingly human, people’s sense of closeness typically improves. However, humanoid robots with overly realistic features, yet subtle differences, tend to cause discomfort or even fear in observers. On the other side of the uncanny valley, if a robot’s appearance completely mimics a human’s, it can regain people’s affection. Therefore, the research team decided to give the robot a non-human blue appearance, with the purpose of not making the robot look too much like a human.
On the basis of the hardware and driving devices, the research team has developed two independent artificial intelligence models for robots. One model is responsible for analyzing and predicting the subtle facial expression cues of humans. This was achieved by watching facial expressions in online videos and conducting frame-by-frame analysis, after hours of training. The other model is in charge of calculating the robot’s facial movements in real time, in order to quickly produce the corresponding expressions. By “looking in the mirror,” that is, facing the RBG camera, the robot continuously monitors its own expressions, and through the model, learns the relationship between the subtle changes in its own face and specific facial movement commands. This process is called “self-modeling.”
During the experiments, necessary restrictions were placed on the robot’s movements to avoid damage from collision to the head or tearing of the facial skin. Such robots, despite having a long way to go on the path of non-verbal communication, have already begun to show a humanized side with their smiles. Professor Lipson stated that although he is a well-informed robot expert who usually does not smile at machines, he naturally returned the smile when the robot smiled at him, which indeed is a very important advancement.
The recent research progress shows that the research team is actively developing technologies to expand the robot’s expressive repertoire, they are focusing not only on how to synchronize the robot’s smile with humans, but also exploring other expressions and lip movements. Currently, the robot Emo’s skills are limited to analyzing human facial expressions, and it cannot dialogue with humans by parsing language text.
To achieve more natural human-machine communication, researchers plan to integrate the robot with advanced large-scale language models such as ChatGPT. They hope this will significantly enhance the fluency and naturalness of human-machine interactions.
Indeed the technological development has broad prospects and may bring many positive applications, however, its potential ethical issues have also caused concern for some. The concern lies in the fact that as the technology improves, robots become increasingly human-like, inevitably requiring people to form connections with them, regardless of whether or not people welcome this connection.
A member of the research team pointed out: “We do not want to make it look too much like a human. But on second thought, if we put wigs on it, apply lipstick, and add other accessories, it might really start to look more and more like a human.” He added: “I think this is an unstoppable trend; eventually, robots will continue to become more similar to humans, which will force us—whether we like it or not—to start forming connections with them.”
