Visual Sensing and its Applications Integration of Laser Sensors to Industrial Robots provides comprehensive and up-to-date coverage of research and development on this robotic vision system. A laser-structured light is the main concern in discussions of visual sensing.Also addressed in this book are all components of the robotic vision system and an emphasis on how to increase the accuracy of the system using three levels of calibration. This includes calibration of the vision system (eye calibration), calibration of eye-to-hand configuration and calibration of robot kinematics (hand calibration). With the integration of the laser sensors to industrial robots numerous applications in the field of robotic welding, grinding, machining, inspection, and palletiz-ing are illustrated based on practical engineering projects in order to demonstrate how the visual sensing is performed. The book will serve as a valuable resource for researchers and engineers in the areas of robotics and machine vision.
Dr. Zhongxue Gan is a vice chairman and chief scientist of the ENN Group, China. He serves as a member of the National Energy Expert Consultation Committee of China and member of the National Coal Council of the USA. He is also a co-founder of Intersmart Robotic Systems Co. Ltd., China. He was a research fellow in flexible automation systems at ABB and a founding director of ABB Corporate Research Robot Laboratories, both in the USA and in China. Dr. Qing Tang is a co-thunder and CEO of Intersmart Robotic Systems Co. Ltd., China and an adjunct professor in Physics at Sichuan University, China. He was a principle consulting engineer and project manager at the ABB Corporate Research Robot Laboratory in the USA.
1 Introduction
1.1 3D Acquisition Techniques
1.1.1 2D Vision
1.1.2 Stereo Vision
1.1.3 Time of Flight
1.1.4 Laser Triangulation Sensor
1.2 Structure of Robot Visual Control System
1.2.1 Structured-Light Sensor Based Visual Control
1.2.2 Selection of Industrial Robots
1.2.3 Applications of Robot Visual Systems
1.2.4 Calibration of Robot Visual Systems
1.2.5 Laser Sensor Based Commercial Robot Visual Systems
1.3 Outline of Chapters
References
2 Characteristics of Laser Structured-Light Sensors
2.1 Formation of Laser Structured-Light Sensors
2.1.1 Light Source
2.1.2 Detector Types
2.1.3 Triangulation Measurement Principle
2.2 Accuracy Analysis
2.2.1 Effect of Laser Speckle Noise on the Measurement
Accuracy
2.2.2 Effect of the Environmental Factors on the Measurement
Accuracy
2.3 Commercial Systems
References
3 Laser Stripe Sensor Calibration
3.1 Modeling of Laser Stripe Sensor and Calibration Strategy
3.2 Camera Modeling
3.2.1 Pinhole Model of the Camera
3.2.2 Nonlinear Modeling with Lens Distortion
3.3 Calibration of Cameras
3.3.1 Calibration with Direct Linear Transform Method
3.3.2 Calibration with Tsai's RAC Based Algorithm
3.3.3 Calibration with Multiple View Algorithms
3.4 Calibration of Laser Stripe Sensor
3.4.1 Laser Stripe Plane Calibration with Two Known Planes
3.4.2 Laser Stripe Plane Calibration Based on Invariance of Cross Ratios
3.4.3 Laser Plane Calibration with a Planar Target
3.4.4 Calibration of Dual Laser Stripe Sensor
3.4.5 Calibration of the Rotation Table
3.4.6 Calibration of the Laser Stripe Sensor with Robot Alignment
3.4.7 Laser Scanner Calibration with Direct Coordinate Mapping
3.4.8 Calibration of Laser Stripe Sensor with Scheimpflug Configuration
3.5 Conclusion and Remarks
References
Calibration of a Robot Visual System
4.1 General Solution of Robot Tool Calibration
4.1.1 Calibration Target with Geometry Constraint: Point
4.1.2 Calibration Target with Geometry Constraint: Line
4.1.3 Calibration Target with Geometry Constraint: Sphere
4.1.4 Calibration Target with Geometry Constraint: Plane
4.2 TCP Calibration for a Point Laser
4.2.1 Algorithms
4.2.2 Calibration of Laser Beam Orientation (nx, ny, nz
4.2.3 Calibration of Laser Sensor Position (x0, Y0, z0
4.2.4 Experimental Results
4.3 TCP Calibration for Cameras
4.3.1 Camera Pose Calibration with Linear Equations
4.3.2 Camera Pose Calibration with Nonlinear Optimizations
4.4 TCP Calibration for 3D Laser Scanner
4.4.1 TCP Calibration with a Sphere
4.4.2 TCP Calibration with a Plane
4.4.3 TCP Calibration with a Structure Pattern
4.5 TCP Calibration with Direct Measurement
4.5.1 Calibration of Spindle
4.5.2 Calibration of Tools with Different Length
4.6 Relative Robot Workcell Calibration
4.6.1 Robot Workcell Calibration
4.6.2 Robot Error Compensation with Relative Measurement
4.7 Summary
References
5 Image Processing of Laser Structured-Light Based Vision System
5.1 Control Point Extraction from Pattern Images
5.1.1 Feature Extraction from Squared Control Points
5.1.2 Feature Extraction from Circle Control Points
5.2 Laser Stripe Sub-Pixel Positioning
5.2.1 Thinning and Pruning Algorithm
5.2.2 Gray Scale Gravity Center Algorithm
5.3 Range Image Registration with the ICP Algorithm
5.3.1 Determination of Corresponding Points
5.3.2 Calculation of Transformation Matrix
References
6 Robot Kinematic Calibration
6.1 Background
6.2 Model Function of Robots
6.3 Determination of Independent Error Parameters Using SVD Method
6.4 Error Budget Analysis
6.5 Solving the Error Parameters
6.6 Circle Fitting Based Calibration
6.7 TAU Parallel Robot Calibration
6.7.1 Kinematic Modeling
6.7.2 Jacobian Matrix of TAU Robot with All Error Parameters ..
6.7.3 Kinematic Modeling with all Error Parameters
6.7.4 Determination of Independent Design Variables
6.7.5 Error Budget Analysis
6.7.6 Simulation Results
6.7.7 Experimental Results
References
7 Visual Sensing and Control-Laser Sensor Based Robot Applications..
7.1 Automatic Inspection of Holes in 3D Space
7.1.1 Introduction
7.1.2 System Overview
7.1.3 System Calibrations
7.1.4 Inspection Procedure
7.1.5 Experimental Results and Conclusion
7.2 Robotic Grinding System of Free-Form Workpieces
7.2.1 Introduction
7.2.2 Offiine Programming
7.2.3 Workpiece Calibration
7.2.4 Robotic System Error Compensation
7.2.5 Experimental System
7.2.6 Conclusion and Remarks
7.3 Robot Remanufacturing of Blade Tip Refurbishment
7.3.1 Introduction
7.3.2 Profile Modeling Based Grinding
7.3.3 Experimental Setup
7.3.4 Conclusion and Future Work
7.4 Robotic Materials Handling System for Complex Parts
7.4.1 System Overview
7.4.2 Approximately Locatiog Workpieces
7.4.3 Precisely Locating Workpieces
7.4.4 Another Example
7.4.5 Summary and Remarks
7.5 Robot Machining System with Visual Feedback
7.5.1 Introduction
7.5.2 System Overview
7.5.3 Scanning and Edge Detection
7.5.4 Path Smoothing Based on the B-Spline
7.5.5 Other Examples
7.5.6 Summary and Remarks
7.6 Robotic Measurement and Inspection System for Quality Control
7.6.1 System Overview
7.6.2 Pick-up Error Compensation
7.6.3 Feature Based Workpiece Locationing
7.6.4 Point Cloud Comparison
7.6.5 Summay and Remarks
7.7 Robot Weld System with Seam Tracking Sensors
7.7.1 System Overview
7.7.2 Welding Joint Detection
7.7.3 Path Generation
7.7.4 Computer-Robot Communication
7.7.5 A Robotic Tube Panel Weld System
7.7.6 Summary and Remarks
7.8 Robotic Pick and Place System with Point Lasers
7.8.1 Robot Logs Pick and Center System
7.8.2 Robot Solar Panel Installation System
7.8.3 Summary and Remarks
References
Appendix
Index