The camera is fixed in place, looking downward to guide the robot's movement.

Detailed Explanation of Robot and Vision Calibration Theory

1.Nonlinear correction for the camera is performed using a calibration board for nonlinear correction. 2. For camera-to-robot calibration involving nine points, the robot can either poke nine points or grasp the workpiece and position it at nine different locations, thereby obtaining nine mechanical coordinates. Simultaneously, the camera captures nine corresponding pixel coordinates. Then, the calibration is carried out as follows: 3. Calculate the robot’s rotation center: The robot picks up the workpiece and rotates it around three different angles, positioning it within the camera’s field of view each time. The camera records three sets of coordinate values. By fitting a circle through these three sets of coordinates, the center of the circle—the rotation center—is determined. 4. The camera calculates the final output result using the following formula: (rx0, ry0) represents the rotation center; (x, y) denotes the point to be rotated; and (x0, y0) represents the rotated point. The formulas are as follows:   x0 = cos(a) * (x - rx0) – sin(a) * (y - ry0) + rx0   y0 = cos(a) * (y - ry0) + sin(a) * (x - rx0) + ry0

The camera is fixed and looking upward.

1.For nonlinear correction of the camera, a robot is used to pick up and position the calibration board for nonlinear correction. As for the 9-point calibration between the camera and the robot, physical calibration can be employed: The robot grasps the workpiece and positions it at nine different locations, obtaining nine sets of mechanical coordinates. Simultaneously, the camera captures nine sets of pixel coordinates. Then, the calibration is performed based on these data. Refer to the diagram above and look from top to bottom.

The camera is mounted on the robot, close to the center of rotation.

11. Nonlinear correction of the camera; 2. For camera-robot calibration using a 9-point method, physical calibration targets can be used. The robot picks up the workpiece and positions it at 9 different locations, obtaining 9 sets of mechanical coordinates. Simultaneously, the camera captures 9 sets of pixel coordinates. (Each time, the robot must return to a fixed position to take a photo.) Then, proceed with calibration: 3. Calculate the robot’s rotation center—by having the robot grasp the workpiece and rotate it through three different angles, positioning it within the camera’s field of view, the camera will capture three sets of coordinate values. By fitting a circle through these three coordinate sets, the center of the circle—the rotation center—can be determined. 4. Finally, the camera calculates the final output result using a specific formula.

The camera is mounted on the robot, far from the center of rotation.

11. Nonlinear correction of the camera; 2. Nine-point calibration between the camera and the robot; 3. Calculation of the robot’s rotation center; 4. The camera computes the final output result using a formula. Note: Since the selected center is located far from the field of view, the rotation center obtained through fitting typically has a relatively large error, which affects the positioning accuracy.

Separation shaft

There are many ways to design the separation axis—such as XY + θ, X + Y θ, X θ + Y..., and so on. The specific approach depends on the particular situation. The goal is to identify the rotation center and perform a 9-point calibration: 1. Nonlinear correction of the camera; 2. Perform a 9-point calibration between the camera and the robot; 3. Calculate the robot’s rotation center; 4. Use a formula to compute the final output result from the camera.

Rotational Center Calibration Instructions

The rotation center method is applicable to all robot-vision coordination scenarios: It calculates the actual displacement and rotation of the workpiece, performs secondary compensation based on the robot’s rotation center, and then sends the compensation values to the robot. After receiving these compensation values, the robot applies the corrections and proceeds with either gripping or placing the part.

Point Coordinate Rotation Method

Calculate the coordinates of a point after it has been rotated by a certain angle around another point, as shown in the figure:

Given that the position of point C(x0, y0) results from rotating point A(x, y) by an angle a around point B(rx0, ry0), the following relationship holds:   x0 = cos(a) * (x - rx0) – sin(a) * (y - ry0) + rx0   y0 = cos(a) * (y - ry0) + sin(a) * (x - rx0) + ry0

All subsequent calculations of rotations and offsets are based on the formula above.

Rotational Center Calibration Instructions

Calculation of the STD method:

CDx = cos(a) * (Cx0 - X0) – sin(a) * (Cy0 - Y0) + X1 – Cx0   CDy = cos(a) * (Cy0 - Y0) + sin(a) * (Cx0 - X0) + Y1 – Cy0   X’ = cos(a) * (X0 - Cx0) – sin(a) * (Y0 - Cy0) + Cx0   Y’ = cos(a) * (Y0 - Cy0) – sin(a) * (X0 - Cx0) + Cy0   CDx = X1 - X’   CDy = Y1 - Y’   Here’s how the robot works: Before picking up the material, it first adjusts its own orientation to match the current orientation of the material. It then moves to the position (X’, Y’). Next, it offsets its position by CDx and CDy so that it precisely overlaps with the material’s location, after which it picks up the material. This ensures that, after each pick-up, the material’s position relative to the robot remains consistent, allowing the robot to directly place the material at the target location. Since this method involves adjusting the robot’s position when picking up the material, it is particularly suitable for scenarios where the camera is fixed on the robotic arm or mounted above the material, looking downward. However, if the camera is fixed below the material, looking upward, this approach would no longer be appropriate, as the robot would have already completed the pick-up by that time.

Pay special attention.

The Cdx and Cdy values calculated here must be compensated into the robot’s pick-up position—not into the drop-off position. The deviation calculated using the rotation-center method can be directly compensated into the drop-off position. The reason is as follows:

Calculation of the rotation center method:

（(X, Y) represents the location of the positioned product, (GX, GY) is the location of the standard template, and (X’, Y’) is the new position after compensating for the angle. Then:   X’ = cos θ * (X - Xo) – sin θ * (Y - Yo) + Xo;   Y’ = cos θ * (Y - Yo) + sin θ * (X - Xo) + Yo;   Offset X = X’ - GX;   Offset Y = Y’ - GY;   Offset Theta = θ.

Method for finding the center of rotation

When the camera’s FOV is close to the rotation center, you can fit a circle using three or more rotated points to determine the circle’s center. However, when the camera’s FOV is far from the rotation center, using three or more rotated points to determine the circle’s center will result in poor accuracy.

Solution

（(X0, Y0) is the rotation center; (X1, Y1) and (X2, Y2) are the coordinates of the workpiece before and after it has been rotated by an angle a in the field of view; (Xt, Yt) is the extension of the line connecting these two points.   d = √[(X2 - X1)^2 + (Y2 - Y1)^2]   Angle aR = d/2/sin(a/2)   R/d = (Xt - X1)/(X2 - X1)   R/d = (Yt - Y1)/(Y2 - Y1)   Xt = (1 - R/d) * X1 + (R/d) * X2   Yt = (1 - R/d) * Y1 + (R/d) * Y2   X0 = cos(90° - a/2) * (Xt - X1) – sin(90° - a/2) * (Yt - Y1) + X1   Y0 = cos(90° - a/2) * (Yt - Y1) + sin(90° - a/2) * (Xt - X1) + Y1   Angle: X+ → Y+ is positive; otherwise, it is negative.

Rotational Center Calibration—Compensation Calculation

CDx Cdy calculation process, rotation angle a

The detailed explanation of robot and vision calibration theory:   CDx = cos(a) * (Cx0 - X0) – sin(a) * (Cy0 - Y0) + X1   Cx0 = (cos(a) - 1) * (Cx0 - X0) – sin(a) * (Cy0 - Y0) + MDx   MDx = (cos(a) - 1) * StDx - sin(a) * StDy + MDx   CDy = cos(a) * (Cy0 - Y0) + sin(a) * (Cx0 - X0) + Y1   Cy0 = (cos(a) - 1) * (Cy0 - Y0) + sin(a) * (Cx0 - X0) + Mdy   Mdy = (cos(a) - 1) * StDy + sin(a) * StDx + Mdy   From the formulas, we can see that StDx = Cx0 - X0 and StDy = Cy0 - Y0 are constants. MDx and Mdy represent the differences between the coordinates of the workpiece (Mark) at each photo capture and the standard reference coordinates of the workpiece (Mark).   How are StDx and StDy calculated? The blue expressions represent a system of two linear equations in two variables. By rotating the coordinate system by an appropriate angle, we can solve for StDx and StDy. Once StDx and StDy have been calculated, during runtime, we simply substitute these values into the above formulas to quickly compute CDx and CDy, thereby determining the robot’s offset without needing to calculate the exact coordinates of the rotation center.

How to solve for StDx and StDy

The calculation processes for StDx and StDy are as follows: First, the robot rotates by a certain angle a at the imaging point (this angle a must be provided by the robot). Next, we apply the detailed theory of robot-vision calibration: CDx = cos(a) * (Cx0 - X0) – sin(a) * (Cy0 - Y0) + X1 – Cx0   = (cos(a) - 1) * (Cx0 - X0) – sin(a) * (Cy0 - Y0) + MDx   MDx = (cos(a) - 1) * StDx - sin(a) * StDy + MDx   CDy = cos(a) * (Cy0 - Y0) + sin(a) * (Cx0 - X0) + Y1 – Cy0   = (cos(a) - 1) * (Cy0 - Y0) + sin(a) * (Cx0 - X0) + Mdy   Mdy = (cos(a) - 1) * StDy + sin(a) * StDx + Mdy   Next, we explain how to solve for StDx and StDy. After rotating by the angle a, MDx and Mdy become fixed values, and both cos(a) and sin(a) are also fixed values.   0 = (cos(a) - 1) * StDx - sin(a) * StDy + MDx   0 = (cos(a) - 1) * StDy + sin(a) * StDx + Mdy   StDx = -0.5 * [Mdx * (cos(a) - 1) + Mdy * sin(a)] / (1 - cos(a))   StDy = 0.5 * [Mdx * sin(a) - Mdy * (cos(a) - 1)] / (1 - cos(a))   Here, a is the standard rotation angle; Mdx = X1 - X0, which represents the difference between the Mark coordinates after rotation and the Mark coordinates in the training template.