MAKIINASDK

The ConsolesRobot ConsoleDrive from the cockpit

Drive from the cockpit

Arm the robot, move it with the jog, the IK handles and named poses, and watch its joints and cameras, with the API call behind every control.

The Control page is the cockpit. You arm the robot there, move it, and watch everything it measures. Each control is a call into the same makiina.robot API your scripts use, and this page puts the two side by side, so you can try a move in the console and then write it in code.

The Robot Console Control page in a live session with acme-robot-02: Safety and motion, Joint states, the 3D twin of both arms and the head, the camera pictures and Telemetry.

The page is laid out the same way for an arm on your PC and for a robot in your fleet:

  • Top bar: Exit, the robot's name and page menu, the connection readout, the ARM switch on the right, and next to it the layout button and, for a robot in your fleet, the record button.
  • Left: Safety and motion, Grippers, and the Joint states table.
  • Centre: the 3D twin of the robot, which mirrors the measured joints, with the Viewport card and view buttons beside it.
  • Right: the camera feeds (robots with a head).
  • Bottom: telemetry plots.

Arm the robot, and stop it

When a session starts, the robot reports but cannot move: the console sends no motion until you flip ARM on. The L and R chips next to it gate each arm on its own. A gated arm ignores the console's motion commands and holds where it is, which is useful when you want to work with one arm while the other stays put. Its gripper is the exception: it opens when you move the other gripper's slider (Open and close the grippers).

The top bar of the Control page.
Console

Flip ARM on to allow motion, off to send no more. Press Space or click STOP to end a Go neutral or Go zero move: the console sends no more steps, and the robot stops at the last one.

Python

In code there is no switch: the first motion command arms the session, and stop() disarms it again.

Python
robot.control.set_joints_relative({"J01-R": 0.05})   # arms the session
robot.control.stop()                                  # sends no new targets

Neither ARM off, STOP nor stop() halts a joint that already has its target, such as a jog step, a dragged handle or a target from a script: it goes on until it gets there, then holds. To stop the robot at once, click Torque off (the arms sag, so support them) or switch off its 24 V supply.

Turning ARM off does not release the motors. The robot keeps holding its last pose, energised, until you arm again or turn the torque off.

Set torque and speed limits

The Safety and motion card holds the two limits the robot applies on its own side to every target the console or a script sends.

Console

Torque fraction caps the current of every joint at a fraction of its limit. Max velocity caps how fast any joint turns, in rad/s at the joint. Torque off releases the motors; Torque on gives them their full current limit again.

The Safety and motion card: Torque on, Torque off, STOP, Go neutral, Go zero, Torque fraction 0.40, Max velocity 1.0 rad/s and Ramp 1.0 s.
Python
Python
robot.safety.set_torque_fraction(0.4)
robot.safety.set_max_joint_velocity(1.0)     # rad/s
robot.safety.torque_off()                    # the arm sags, hold it
robot.safety.torque_on()                     # full current limit, see below

A torque fraction of 0.3 to 0.5 and a velocity of 0.5 to 1.0 rad/s are good settings for bench work: strong enough to move the arm, slow enough to watch, weak enough that a collision stalls instead of damaging anything.

Set both at the start of every session, even when the sliders already show the values you want. The sliders do not read the robot: they show 0.80 and 3.0 or an earlier session's values, while a new session starts every motor at its full current and speed limits.

Go to neutral or to zero

Go neutral raises the arms into their ready pose. Go zero folds them down flat, every joint at 0, and turns the head to face forward. Both start from where the robot is measured to be and ease in and out over the time set by Ramp, in seconds. You need to arm first.

Console

Set Ramp to 2.0 s and click Go neutral.

Python

The console streams a smoothstep ramp at 50 Hz from the measured pose:

Python
import time

target = robot.model.named_poses()["neutral"]
start = robot.sensors.get_joints()
for k in range(1, 101):                       # 2 s at 50 Hz
    a = k / 100
    a = a * a * (3 - 2 * a)                   # smoothstep
    robot.control.set_joints_absolute(
        {j: start[j] + a * (target[j] - start[j]) for j in target})
    time.sleep(0.02)

Jog one joint

Click a row in Joint states to select that joint. The twin highlights it and the Jog card opens.

The Control page of acme-robot-02 with ARM on.
Console

Use the step buttons to move by 0.005 or 0.05 rad, or hold the stick and pull: the further you pull, the faster the joint turns, and it stops when you let go. The stick moves the target at up to 0.8 rad/s, so with Max velocity below that the joint falls behind and goes on after you let go, until it reaches the target. The head row jogs the head's yaw the same way.

The Jog card of the right arm's J2 shoulder at 1.100 rad, with its slider and steps of 0.05 and 0.005 rad.
Python

Every jog step is one absolute target for the selected joint:

Python
robot.control.set_joints_absolute({"J02-R": 1.15})   # a joint, in rad
robot.control.set_head(0.2)                          # head yaw, rad, + is right

Move a gripper in space

The handles on the twin let you place a gripper in space and let the kinematic twin work out the joints. In the Viewport card choose which handle to show (L, R or head), whether it moves or rotates (W and E switch between them), and whether it follows the gripper's own axes (local) or the robot's (world).

The 3D twin of both arms and the head with callouts on the joints.
Console

While disarmed, dragging a handle only previews: a copper ghost shows the joint angles that would reach it, and nothing moves. Once armed, dragging the handle moves the arm.

The Viewport card: targets follow robot, IK ghost, move or rotate, local or world, L, R or head, mirror smoothing and Sync targets.
Python
Python
import numpy as np

pose = robot.sensors.get_ee_poses()["right_arm"]     # 4x4, metres
pose[2, 3] += 0.03                                   # 3 cm up
q = robot.model.solve_ik({"right_arm": pose})        # preview only
robot.control.set_ee_poses_absolute({"right_arm": pose})

targets follow robot keeps the handles on the measured grippers while you are not dragging them. Sync targets snaps them back onto the twin if they drifted away. Move the gripper in space covers poses and frames in detail.

Open and close the grippers

Console

The Grippers card has one slider per gripper, from 0 (open) to 1 (closed). They are locked until you arm.

The Grippers card with Gripper R and Gripper L, both at 0.00.
Python
Python
robot.control.set_ee_triggers({"right": 0.6, "left": 0.0})

The robot maps the 0 to 1 range onto the gripper's own travel, so the same value means the same closure on every arm.

Read the joints

Joint states lists every joint with its measured position (rad), velocity (rad/s) and current (A). The robot sends its state 150 times a second; the console passes it to the page at about 60 Hz, the rate the table's header shows, and repaints the numbers about 12 times a second. The joints are grouped by part, and each part says whether it is safe (disarmed) or armed. The twin carries callouts with the live values of the joints you selected.

Console
The Joint states table: position, velocity, current and temperature of every joint of both arms and the head.
Python
Python
robot.sensors.get_joints()        # {"J01-R": 0.0, ...} rad
robot.sensors.get_velocities()    # rad/s
robot.sensors.get_currents()      # A
robot.sensors.get_head()          # head yaw, rad

Watch the cameras and steer the foveas

On a robot with a head, the right column shows the two head cameras (Context L and Context R), a fovea view and the two gripper cameras. The head cameras stream in one of two modes, and the readout at the top of the column says which one runs and at what bitrate:

  • full res, the default: each head camera sends its whole picture, 2304 x 1296 at 56 frames per second. The fovea view is a window that your PC cuts out of the full picture.
  • foveated: each head camera sends a 1152 x 648 context picture and a sharp fovea window that the robot cuts out of the full picture where you look. At the robot's starting bitrates it sends as much as full res, and a picture that a lost packet damages takes up to about 10 s to be whole again, against a few frames in full res.

Either way the fovea shows its part of the scene at the camera's full working detail. Click in either context view and the fovea moves there. Click the label of the fovea view to switch it between the left and the right eye.

Console
The Cameras panel in full res: Context L and Context R, the fovea card cut from the full picture, and Gripper R and Gripper L.
Python
Python
frames = robot.sensors.get_camera_frames()   # {stream: HxWx3 RGB array}
frames["stereo_cam_left"].shape              # (1296, 2304, 3) in full res
frames["stereo_cam_left_fovea"].shape        # (640, 640, 3)

robot.control.set_gaze([0.5, 0.5, 0.5, 0.5]) # right x, y, left x, y in 0..1

Change the camera settings

Click the readout at the top of the camera column to open the camera settings.

Console

Head video switches between full res and foveated. The camera streams restart, so they blink for a moment, and the line next to the switch says what the robot streams now. In full res, Full res bitrate (per camera) sets the rate of each camera's picture; in foveated, Context bitrate and Fovea bitrate set the two streams apart. Bitrates apply at once and the robot keeps them. Fovea picks a 640 px window (head 640, steered by head gaze) or a 320 px one (eye 320, for eye tracking). In foveated the robot restarts its camera pipelines for it; in full res it only changes the window your PC cuts out. Apply camera parameters sends auto exposure, exposure, gain and sharpness; the robot restarts its camera pipelines, so the streams blink for a moment, and it keeps the values.

The camera settings popover in full res: the bitrate per camera, Head video, the Fovea size and the camera parameters with Apply camera parameters.
Python
Python
robot.control.set_video_layout("head")                       # foveated, "full" for full res
robot.sensors.get_video_layout()                             # what the robot streams now
robot.control.set_video_bitrate(8000)                        # full res, kbit/s per camera
robot.control.set_video_bitrates(context=6000, fovea=2000)   # foveated, kbit/s
robot.control.set_gaze_method("HeadsetOrientation")          # 640 px fovea
robot.control.set_camera_params({"ae_enable": False,
                                 "exposure_time": 8000,       # us
                                 "analogue_gain": 4.0,
                                 "sharpness": 2.0})

See what the robot sees shows how to work with the frames and both modes in code.

Plot the telemetry

The telemetry band plots the last few seconds of whatever you selected: position, velocity, current, grip and head, or ping. With a joint selected it plots that joint; otherwise the position tab plots the right gripper's x, y and z.

The Telemetry plot of one joint's position over a 4 s window.

The window chip switches between 4, 8 and 12 seconds. auto range fits the plot to the data; turn it off to pin a minimum and maximum for that tab. The pause button freezes the plot while you read it.

Record a demonstration

On a fleet robot the record button in the top bar opens the recording card, and records every camera stream and every command of the session. Record demonstrations describes it together with the recording API.

Change the view

The buttons beside the twin switch between a perspective view and front (F), side (S) and top (T) views, and turn the floor grid on and off. The layout button in the top bar widens the twin and hides the cameras and telemetry. mirror smoothing adds 20 ms of display smoothing to the twin; the numbers in the table are never smoothed.

Keyboard shortcuts

KeyDoes
SpaceSTOP: end a Go neutral or Go zero move at its last step; a jog step or a dragged handle goes on to its target
W, Ehandle mode: move, rotate
Escclose the open card; cancel a session start
[, ]previous and next page of the console's whole page list, not this robot's menu (Move between pages)
Arrow keysnudge the focused slider

Shortcuts are ignored while you type in a field.