Camera Controlled Robot Arm - Adrian Bartoš

Parts Used

Arduino Uno
Servo Driver PCA9685
6x servos TD-8135MG
Mobile Phone for Camera detection

Assembely

Build robot arm controled by arduino
Write Urdf file for our robot
Set up Camera detection
Move robot acording to coordinates gathered from camera

Camera Detection

I chose to use OpenCVP's built-in AruCo markers for defining working space and desired object.

1. We need to generate markers

  * In code with name "GenereteArCo.py" we need to genere 4x 4x4 Aruco markers and 1x 6x6 marker for object detection. For working space AruCo's we need to generte id's 1-4. So we are only changing id variable. And then depending on camera distance and recording quality the tag size variable for size in pixels. So for 4x4 Aruco marker with id 1 and size of 100px we modify code like in example below. Same thing we do for object AruCo, we change form 4x4 to 6x6. I chose different AruCo type for object for easier recognition. When all markers are genereted. We need to print them.

aruco_type = "DICT_6X6_50"
id = 1

arucoDict = cv2.aruco.getPredefinedDictionary(ARUCO_DICT[aruco_type])

print("ArUCo type '{}' with ID '{}'".format(aruco_type, id))
tag_size = 100
tag = np.zeros((tag_size, tag_size, 1), dtype="uint8")
cv2.aruco.generateImageMarker(arucoDict, id, tag_size, tag, 1)

2. We need to know size of our working space. I printed on 4x 1cm grid paper that i glued together for easier coordination validation. We change in "Aruco_detection.py". I had to flip my x and y axis because i had already assemled the robot and prepared Arduino code so it was easier to flip axis then rewrite whole code and urdf file . In x_coordinate we change the last number that is multiplying by our X axis length. Form me it was hight of the papers. They were 37 cm high. Then y_coordinate. The width is 54 cm. Multiplication with 54 is our hight. And we are subtracting half of that so our y_coordinate goes from [-27,27] cm.

if keyboard.is_pressed('p'):
    x_coordinate = (1 - (centerCorner[0][1] / h)) * 37  # Convert pixels to cm
    y_coordinate = ((centerCorner[0][0] / w) * 54) - 27  
    print("Object Position (cm) relative to area:", round(x_coordinate, 2), ",", round(y_coordinate, 2))
    compute_ik_for_target([x_coordinate / 100, -y_coordinate / 100, 0.25])

3. We need to calibrate our camera. We use chessboard patter to get our camera data. I took 10 photos from different angles and position. Run "camera_calibration.py" script. That gives you your data in npz file.