Cars that read their drivers' emotions

We’ve all had that moment when we’ve been so angry or upset while driving that we just wish we could hand the wheel to someone else. It’s a frustrating feeling — one that semiconductor supplier Renesas Electronics has set out to solve.

The company has created a development kit for its R-Car system-on-chip (SoC) that takes advantage of the ‘emotion engine’, an artificial sensibility and intelligence technology pioneered by cocoro SB. The development kit enables cars with the sensibility to read the driver’s emotions and optimally respond to the driver’s needs based on their emotional state.

The first part of emotion engine is voice emotion recognition that performs an analysis of a voice signal. The second part is emotion generation technology that generates emotions for the device by forming a virtual hormone balance derived from various connected sensors. It thus becomes possible to perceive the speaker’s emotion by charting an ‘emotion map’ as both a colour — for example, yellow for happiness and red for sadness — and as a volume level from 1–10.

Utilising the emotion engine, the development kit recognises emotional states such as confidence or uncertainty based on the speech of the driver. The car’s response to the driver’s emotional state is displayed by a driver-attentive user interface (UI) implemented in the R-Car SoC. Since it is possible for the car to understand the driver’s words and emotional state, it can provide the appropriate response that ensures optimal driver safety. And as this technology is linked to AI-based machine learning, it is possible for the car to learn.

Renesas considers the driver’s emotional state, facial expression and eyesight direction as key information that combines with the driver’s vital signs to improve the car and driver interface, placing drivers closer to the era of self-driving cars. For example, if the car can recognise the driver is experiencing an uneasy emotional state, even if he or she has verbally accepted the switch to hands-free autonomous driving mode, it is possible for the car to ask “would you prefer to continue driving and not switch to autonomous driving mode for now?” Furthermore, understanding the driver’s emotions enables the car to control vehicle speed according to how the driver is feeling while driving at night in autonomous driving mode.

By providing carmakers and IT companies with a development kit that takes advantage of the emotion engine, Renesas hopes to expand the possibilities for this service model to the development of new interfaces between cars and drivers and other mobility markets that can take advantage of emotional state information. The company plans to release the development kit later this year.

Image credit: ©carlos_bcn/Dollar Photo Club