High-speed vision using single-photon sensors

Academic Institution: University of Edinburgh

Academic Supervisor: Dr Istvan Gyongy

Industry Partner: STMicroelectronics

PhD Student: Jack Maclean

Start Date: 1st September 2020

Abstract

3D image sensors based on Time-of-Flight (ToF) are integral to many robotic systems, and are also emerging as key sensors in driverless vehicles. However, effective frame rates are limited by bottleneck issues in the data readout and processing, and, for some sensors, the necessity for optical scanning to cover a sufficiently large field of view. These issues are compounded by environmental conditions such as a high ambient light level or fog, and can represent a significant performance limitation in applications where fast object tracking is required (e.g. for collision avoidance).

This project aims to build on recent architectural innovations in single-photon avalanche diode (SPAD) sensors arising from Edinburgh University and STMicroelectronics, which has led to array format sensors with on-chip data compression, and the capability of hundreds of frames per second when paired with flash-type illuminator. We propose the development of a high-speed vision system underpinned by optimised signal processing algorithms, which take full advantage of the high frame rates offered by the SPAD. More specifically, we are interested in robust, hardware-compatible object detection and tracking schemes, enabling real-time actuation in a range of environmental conditions.

The project brings together academics at the University of Edinburgh and Heriot-Watt University with track records in SPAD imaging and system-level design for ToF, and industrial partners STMicroelectronics, the world’s leading company for SPAD-based ToF modules. The algorithms developed in the project are expected to inform a next generation of SPAD sensors with embedded processing for low latency 3D perception. In addition to enhancing safety-critical applications in autonomous systems, the benefits would be broad, enabling, for example, fast 3D capture in AR/VR headsets for a seamless experience.