STMicroelectronics VG5761 Global-Shutter sensor for automotive in-cabin monitoring

Click2Contact

Semiconductor company STMicroelectronics has announced a cooperation with Eyeris, a provider of vision-based AI software and in-cabin sensor fusion technologies. The collaboration focuses on extending ST’s Global-Shutter sensor to in-cabin sensing applications with Eyeris’s portfolio of deep neural networks for a comprehensive visuospatial understanding of the entire vehicle interior. This supports safety and convenience features including driver monitoring systems (DMS), occupant monitoring systems (OMS), child presence detection, object recognition, gesture control and activity prediction.

ST’s 2.3 MP VG5761 Global-Shutter sensor features high linear dynamic range of up to 98 dB for sharp, crisp images. The sensor can capture images inside a vehicle under all lighting and environmental conditions. Combined with Eyeris’s in-cabin sensing technology, the sensor can be used for Eyeris’s DMS with eye-gaze tracking, especially from non-frontal camera locations such as rearview mirror, overhead console and centre stack areas. In parallel, the sensor also enables Eyeris’s OMS with tracking of body key-points, height, width, size, posture, movements and orientation. Furthermore, the sensor enables child presence detection and recognition of common vehicle interior objects under the widest range of lighting conditions. The technological integration from ST and Eyeris delivers in-cabin safety, comfort and convenience benefits for all occupants.

The image sensor offers two memory zones that allow double-image storage which, in addition to contributing to the high dynamic range, supports background removal without lag effects and additional processing by the host system. In its RGB-IR version, the image sensor uses high MTFs (modulation transfer functions) at near-infrared (IR) wavelengths as well as through the RGB channel to monitor the full in-cabin scene under a broad range of lighting environments while featuring the capability to analyse interior safety-critical regions of interest (ROIs) in great detail — like the driver’s eyes and the occupants’ body key-points — to detect early signs of driver drowsiness or distraction and occupants’ body posture. The sensor’s high-performance capability, combined with Eyeris’s AI-based algorithms, helps automakers personalise restraint controls and passive vehicle safety systems.