A.7 Vision-guided grasping in Space

ROTEX on board Space-Shuttle Orbiter Columbia, May 1993

The most challenging robot task was to visually determine the pose and motion relative to the robot hand. Cameras (Ki ) and the two fingers for grasping can be seen in the lower figure. The object to be grasped was a metal cube of originally 2 cm edge size, the corners of which had been cut off orthogonal to rays from the center such that just one point in the center of the original edges of the cube remained (see upper center of snapshot, left). The cube had been painted dark, the surfaces from the cut remained in metal color (bright triangles); thus, eight bright triangles and six dark squares represented the surface of the cube. After careful preparations in the laboratory on the ground, the flight experiment was successfully performed on May 3, 1993 while Space Shuttle Columbia passed over northern Africa.

The biggest challenge in this task was to compensate the variable delay times of up to more than six seconds in the closed perception – action loop encompassing the robot arm onboard Space-Shuttle Columbia in orbit and the ground station in Oberpfaffenhofen (near Munich) where the images were analyzed by computers and the proper control output was determined. The delay time resulted from signal transfer in both directions through several (changing) satellites in orbit and computer centers as well as cables on the ground.

The robot hand, containing two cameras (Ki in the sketch top right) and a laser range finder, was initially set by manual control such that the object to be grasped was visible for the cameras in the hand. Automatic control started at 4.8 seconds; the grasping act controlled by automatic vision lasted about 11 seconds while the Space Shuttle traveled almost 90 km over northern Africa on May 3, 1993. The curves labeled ‘estimated’ are the result of feedback of visual prediction errors in recursive estimation.

Video – ROTEX- GraspingInSpace 1993

H.2.6 Rotex German SpaceLab D2 onboard Space Shuttle Columbia