New 3D smartphone technology could change photography,
Your next smartphone could have a camera that sees in three dimensions, opening up a whole new range of possibilities for apps ranging from fitness to photography.
Researchers at Stanford University have created a new approach that allows standard image sensors to see light in 3D. The technique would allow cameras to measure distance to objects and make three-dimensional imagery widely available in smartphones.
“Existing 3D cameras need specialized pixels, which are difficult to achieve in large formats and have smaller fill factors due to the complex electronics required to capture 3D in pixels,” Okan Atalar, PhD student in electrical engineering at Stanford University and the first author on a new paper which describes the new system, Digital Trends said in an interview. “Our approach converts standard and highly advanced 2D sensors into 3D sensors instead of building them from scratch.”
Measuring the distance between objects with light is currently only possible with specialized and expensive lidar systems – short for “light detection and ranging”. Lidar uses a laser that shoots at objects and measures the bouncing light. It can tell how far away the thing is, how fast it is moving, whether it is approaching or moving away, and whether the trajectories of two moving objects will intersect.
The Stanford researchers’ new approach could enable lidar at megapixel resolution – a level not possible today. Higher resolution would allow the lidar to identify targets at a greater distance.
One way to add 3D imaging to standard sensors is to add a light source and a modulator that turns the light on and off millions of times per second. By measuring variations in light, engineers can calculate distance. Existing modulators require impractical amounts of power.
The Stanford team solved the modulator problem using a phenomenon known as acoustic resonance. The researchers built an acoustic modulator using a wafer of lithium niobate – a transparent crystal much sought after for its electrical, auditory and optical properties – covered with two transparent electrodes.
It could become the basis for a new type of compact, low-cost, power-efficient lidar… that could find its way into drones, alien rovers and other applications.
The design of the new modulator is simple and fits into a proposed system that uses standard cameras, such as those found in cell phones and everyday DSLRs. Atalar and his adviser Amin Arbabianassociate professor of electrical engineering and lead author of the project, said it could become the basis for a new type of compact, low-cost, power-efficient lidar, the “standard CMOS lidar” as they call it, which could find its way into drones, alien rovers and other applications.
“Our approach could also work in the infrared regime,” Atalar said. “No IR image sensor can detect depth without requiring significant modifications.”
Apple includes lidar on its current iPhone 13 Pro and iPhone 13 Pro Max models. The company says the system provides better focus in low light and improves the effects of night portrait mode. The Stanford researchers said their lidar solution is less expensive to implement than that used by Apple and could be installed on a wider range of phones.
Lidar scanning devices are used to determine the depth of a photo, Hans Hansen, the CEO of Brand 3D, a 3D photography company, told Digital Trends. By moving the camera around the object, distances from multiple angles can be used to create a complete 3D model. There is also stereo photography where multiple cameras are placed apart (eg the three camera lens module on Apple iPhone Pro phones) and then use the information to create a spatial image of a scene or object.
“We’ve seen use cases of measuring the distance from the wall in your home and other spatial measurements,” Sukemasa Kabayama, the CEO of Uplift Labs, which uses 3D analytics to analyze human performance using smartphones, told Digital Trends. “While not specifically 3D cameras, these smartphone cameras have the power to capture valuable data and produce 3D visualizations using video and other applications.”
Recently, MIT researchers developed ultra low power radar which use ordinary radar technology to detect distances to moving objects. This technology could eventually be suitable for making a new type of camera that would not be sensitive to light problems, for example when scanning transparent objects.
The widespread use of 3D technology could radically change photography, experts say.
“With 3D cameras, you would be able to capture scenes and objects that people could experience from a distance as if they were physically in the room,” Hansen said. “It would be revolutionary for remote working, learning, and safe distancing during pandemics, as well as for diagnosis, treatment, and function repair in healthcare, technology, and manufacturing. »
Kabayama said 3D imagery collected by smartphones could provide detailed analyzes and improvements in various industries. One area where 3D technology could have an impact is sports, fitness and wellness.
“Whether you’re a CrossFit junkie, weekend golfer, or Peloton enthusiast, the risk of physical injury is present and for many, a constant battle,” Kabayama added. “Professional athletes have access to 3D technology that helps minimize performance-related injuries, but most of us everyday athletes don’t.”
Kabayama predicted that by making 3D cameras and analysis accessible via smartphones, athletes of all skill levels could track and analyze their movements for detailed biomechanical analyses.
“With most injuries from overuse, poor form or other poor body mechanics, 3D imaging can make identifying areas for improvement – whether it’s form or parts of the body to strengthen – a transparent task,” he added.
Having a 3D camera on your mobile device might also make it safer. After all, face-based security on phones is only as good as the camera behind it, Richard Careersenior vice president and general manager of CyberLink, which manufactures facial recognition technology, said in an interview.
Face ID, for example, which many people use to access sensitive accounts such as mobile bank accounts, work email and to pay via smartphone, runs on Apple’s TrueDepth camera system. More advanced 3D cameras coming to market are capable of capturing even more detailed depth readings, Carriere said.
“The improved accuracy not only reduces the number of times the technology fails to scan your face correctly, perhaps because you’re at an unusual angle, but it also critically protects against attempts to usurpation,” Carriere added.