Depth & Point Clouds, step 5
Where is the box?
Align a model of a box with its points (best-fit rotation by SVD, then ICP) to find its pose, and trace its top with the gripper.
Position isn't enough
To pick up a box by its short sides, or trace its edges, the robot needs its pose: the rotation and translation that carry the box's own frame (origin in the middle, along its long side) into the world. You know the box's shape, so box_model lists points over its top and sides in that frame. The job is to find the that lay those model points over the cloud.
Best fit for known pairs
Suppose you already knew which model point became which cloud point . The that minimise come from the Kabsch method:
- Centre both sets on their means .
- Form , a 3 × 3 matrix, and take its singular value decomposition .
- , where . Without , a flat or symmetric set can return a mirror image (), which no real object can turn into.
- .
ICP: guess the pairs, then improve
You don't know the pairs. Iterative closest point guesses them: move the model by the current pose, pair every cloud point with its nearest model point, solve for the best fit of those pairs, and repeat. Each round the pairs get better, and so does the pose. Pairing each cloud point (not each model point) matters: the camera can't see the box's back sides, so those model points mustn't drag the fit.
ICP only finds the nearest good answer, so it needs a rough start. The program supplies one: the middle of the box's voxels, turned along their longest spread (PCA again, in and ). A box looks the same turned by 180°, so either answer is right. Then it goes back to the full cloud around the box: voxel means sit inside an object's edges, and the edges are what pin the pose.
Your task
Write best_fit(A, B), returning (R, t) for paired arrays, and icp(scene, model, T, iters), returning the improved 4 × 4 pose. The program stores it as box_pose, overlays the model in orange and traces the box's top corners.