Understanding The Application Of 3D Vision in The AGV/AMR Field

Key Takeaway:

3D vision enables mobile robots to see, understand, and interact with their environment. It's a multidisciplinary technology combining computer graphics, computer vision, and artificial intelligence. 3D vision technology captures the three-dimensional coordinates of every point within its field of view through 3D cameras, reconstructing a 3D image using algorithms. Compared to 2D imaging, 3D vision is more stable, resilient to environmental and lighting changes, and offers a better user experience and higher safety.

3D Vision Technology Pathways

3D sensors act as the "eyes" of 3D vision, using combinations of multiple cameras and depth sensors to gather data on the three-dimensional position and size of objects. The main 3D vision sensors currently available are binocular cameras, structured light cameras, and TOF (Time of Flight) cameras.

• 3D Structured Light Technology: This method uses infrared light, which is projected onto an object with a certain encoding. When the light reflects back, the pattern will deform depending on the object's distance. The image sensor captures the deformed pattern, and using triangulation, the deformation of each pixel is calculated to derive the corresponding disparity and further calculate the depth value.

• TOF (Time of Flight) Principle: This technique uses an infrared light source to emit high-frequency light pulses onto an object, then receives the reflected pulses and calculates the distance from the camera to the object by measuring the travel time of the light pulses. Currently, there are two mainstream TOF solutions in the market: dTOF and iTOF. Industry experts believe that dTOF will gradually replace iTOF because of its superior performance in key aspects such as resolution, accuracy, ultra-low power consumption, strong anti-interference capabilities, and simple calibration. However, dTOF has high technical barriers, high system integration, and limited supply chain resources.

• Binocular Stereo Vision Technology: This method simulates human vision by observing the same object from two viewpoints, obtaining images of the object from different perspectives. Using triangulation, the positional deviation (disparity) between pixels in the images is calculated to obtain a 3D image of the object. The hardware structure of binocular stereo vision typically uses two cameras as the visual signal acquisition devices. These cameras connect to a computer through a dual-input channel image acquisition card, and the analog signals collected by the cameras are sampled, filtered, enhanced, and converted to digital form, finally providing image data to the computer.

Applications of 3D Vision in Mobile Robots

As vision technology evolves from 2D to 3D, 3D vision sensors are becoming crucial in mobile robots, offering depth perception and enabling real-time sensing in three-dimensional spaces, accurate object recognition, multi-obstacle detection and avoidance, intelligent decision-making, and automated guidance. These capabilities are increasingly being applied in logistics, e-commerce, automation, manufacturing, industrial and service robots, commercial settings, and more, with expanding application boundaries.

In mobile robotics, 3D vision is mainly used for navigation, obstacle avoidance, and end-material recognition and docking.

• Navigation: Accurate environmental sensing is the primary task for mobile robots. The "environment" here includes various factors such as interference from different lighting conditions indoors and outdoors, obstacles in the path, whether the route is clear and flat, the types of objects in the environment, whether there are people who may cause the robot to slow down or stop, whether the pallet ahead is empty or full, where the insertion slots of a loaded pallet are, and how to plan the route for pickup. Simplified, the logic is that a vision-based mobile robot needs to accurately recognize its surroundings, avoid dynamic and static obstacles, approach the target object dynamically (navigation), and correctly interact with the target object (object detection and positioning recognition).

• Obstacle Avoidance: The market offers a variety of obstacle avoidance sensors, such as single-line LiDAR, ultrasound, and collision strips. Collision strips are usually the last line of defense for violent collision prevention; ultrasound obstacle avoidance often results in false positives; single-line LiDAR has significant blind spots (only detecting obstacles in a two-dimensional plane, unable to detect obstacles below or above the laser, posing a safety risk). 3D vision sensors can compensate for these shortcomings. The best current obstacle avoidance solution for mobile robots is a combination of 3D vision sensors and LiDAR, with 3D vision sensors providing precise short- and medium-range obstacle avoidance and LiDAR for long-range two-dimensional obstacle avoidance. Since TOF cameras virtually have no blind spots, they are currently the most widely used 3D vision cameras for AGV obstacle avoidance.

• End Recognition and Docking: In some warehouses, the placement of goods is complex, and manual or vehicle placement of pallets is often inaccurate. This inaccuracy makes it difficult for an unmanned forklift to accurately identify the pallet using traditional mechanical limits or monocular camera recognition, leading to frequent positioning errors during pallet docking and, consequently, low operational efficiency. Using 3D vision to capture pallet images, combined with appropriate image processing algorithms, the forklift can identify the pallet's position and posture coordinates, intelligently adjust the direction for insertion, and achieve unmanned intelligent pallet handling, resolving the issue of significant angular deviation during unmanned forklift pallet docking. Furthermore, AI algorithms can be used to strengthen and deeply learn pallet recognition models, further enhancing the accuracy of pallet recognition and tracking.
•  

Future Directions: Higher Resolution, Faster Frame Rates, Better Environmental Adaptability

As mobile robot applications continue to deepen, the demand for higher sensing capabilities has increased, pushing 3D vision technology development in this direction. However, the current application of 3D vision in mobile robots is still in its early stages. As mobile robots continue to evolve, with more diverse application environments, the requirements for 3D vision systems will become more stringent, driving further upgrades in 3D vision technology.

Please click on the link below to read more：

Introducing The Reeman Moon Knight Robot Chassis

Introducing The Flash Food Delivery Robot

Introducing The Nurse Hospital Delivery Robot