Software-defined sensors take advantage of computational photography concepts

Over the past decade, the traditional camera industry has been battered by consumers’ shift to using smartphones as their primary image capture device. The fact that a device that is usually less than 10mm thick can generate a pleasing image even from a distance is actually quite amazing. Some of the reflections that allowed phones to create what look like well-lit images in near-total darkness or portraits with simulated background blur are now being applied to vehicle sensors like lidar.

Over the past few years, there has been a lot of talk about moving to software-defined vehicles. Increasingly, almost everything in the vehicle is managed by software, and individual sensors, including lidar, are no exception. From controlling the parts that operate the sensor to processing the outgoing signals, software is critical to achieving the desired results. As lidar sensors begin to proliferate on new vehicles, engineers are beginning to apply some of the concepts of computational photography to improve perception capabilities.

Most if not all of the next wave of lidar sensors, including those made by companies like Aeye, Luminar and others, have the ability to update their on-board software if the vehicle has update capability. live day (OTA). While most lidar that have received attention so far have focused on mid- to long-range sensing capabilities to enable assisted and automated driving systems, a new class of sensors based on near-infrared light is emerging, near-field lidar (NFL).

Like most other lidars, NFL sensors use light in the near infrared spectrum with wavelengths around 905 nm. Light is emitted and the time it takes to reflect from objects is measured to calculate distance. The difference between other lidar and NFL is that there is usually no beam steering, they operate at lower output powers and have lower resolution. NFL sensors from companies such as PreAct Technologies and Valeo have a maximum range of around 15-20 meters, while Aeye’s high-end lidars have been shown to detect vehicle up to 1000m and Luminar sensors can go over 500m. These sensors also currently cost up to $1,000. PreAct’s T30P targets a high-volume production cost of $25 or less.

One of the ways Aeye, Luminar, Innoviz and others use the software is to control the mechanism that steers the laser beam across the field of view, putting more laser pulses on specific areas of interest and a sparser point cloud over areas of the field of view. periphery.

PreAct’s low-cost, low-power sensor is mechanically very simple with a 940 nm LED emitter, a 320 x 240 pixel photosensor to capture reflected photons, and a processing chip with integrated software. It is designed to fit into applications that currently use ultrasonic sensors or short-range radar such as park assist, curb detection, rear automatic emergency braking and impending collision detection.

Although it does not have a beam steering system to manage, the software can still adjust the operation in real time according to the current context. For example, while most of the assist functions listed earlier are intended for low speed operation where the short range is adequate, when the vehicle is moving at higher speeds, with more airflow for cooling , it could increase transmitter power for longer range. detection, but not as long as other more expensive sensors. It could also take advantage of some of the same computational photography techniques used on smartphones, such as stacking multiple images to get more detail than the sensor could capture in a single image. Closed-loop feedback from the driver assistance system’s perception software can be used to process the edges of detected objects.

Like other sensors, the PreAct sensor can be expected to improve over time with software updates. As new features are developed, automakers will likely try to find ways to charge customers for new features delivered via OTA updates without having to replace hardware.

PreAct claims to have won two designs from automakers for its near-field lidar sensors and they are expected to launch on vehicles in the middle of the decade. PreAct also offers the sensors for a variety of aftermarket applications, with the first product expected to launch before the end of 2022. PreAct doesn’t say exactly what type of product it will be, but CEO Paul Drysch acknowledged that systems like that blind spot monitors for trailers would be a potential application.

Stewart C. Hartline