I gave my OpenClaw agent a physical body

I recently gave it My OpenClaw Realistic robot arm to play with. The results almost blew my neural network away.

The AI ​​agent was able to configure the arm, use it to slowly see and pick up objects, and even train another AI model to pick up and place specific objects. And they say that AGI is still a few years away! (I’m just kidding, maybe it is).

The results have me convinced that we may be on the verge of a breakthrough in robotics. Training and controlling the robots required great skill. Today’s AI models can make it almost easy.

“AI-powered programming is very exciting because it has the potential to bridge the gap between traditional engineering methods, which are reliable but do not generalize, and contemporary models of vision, language and movement, which generalize but are not yet reliable,” says Ken Goldberg, a roboticist at the University of California, Berkeley, who is exploring this approach.

I asked OpenClaw to try moving his new arm and he came up with this little wave.

I purchased a pre-built arm called a Lirobot 101. It’s part of the HuggingFace open source project which makes it relatively cheap to get started building and experimenting with bots.

The LeRobot comes with two arms: a control arm that a person operates using a handle and actuator, and a slave arm equipped with a camera that replicates those movements. You can train an AI model by remotely operating a controller arm and having the model learn how to move the follower in response to what it sees on camera.

Building with OpenClaw

Before using OpenClaw, I spent several hours trying to connect and calibrate the robot, and at one point I almost broke the motors by applying the wrong settings, causing them to overheat.

Then, with the help of OpenClaw and Codex, I was able to program a simple program that closes the claw grip when it detects a red ball. At the station, Codex did the difficult work of establishing communications with the robot. Then, with my help, he calibrated his joint positions. She also wrote a Python script that uses several libraries to identify and catch the ball in question.

Then, with my help, the robot agent figured out how to recognize and catch the red ball.