Conceptio › Archive › Google Patents

Google Patentsopen access

Methods and apparatus for controlling a gripper of a robotic device — Boston Dynamics, Inc. (US20260109038A1)

Boston Dynamics, Inc. · Google Patents

Google Patents · Patents · License: Open Access

Open Source ↗

bostondynamics

patent, google patents, intellectual property, US20260109038A1, Boston Dynamics, Inc., Neil Neville, en, 2026

ABSTRACT

Abstract

Methods and apparatus for controlling a robotic gripper of a robotic device are provided. The method includes activating a plurality of vacuum assemblies of the robotic gripper to grasp one or more objects, disabling one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold, assigning a score to each of the one or more disabled vacuum assemblies, reactivating the one or more disabled vacuum assemblies in an order based, at least in part, on the assigned scores, and grasping the one or more objects with the robotic gripper when a grasp quality of the robotic gripper is higher than a second threshold.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 18/545,148, filed Dec. 19, 2023, entitled, âMETHODS AND APPARATUS FOR CONTROLLING A GRIPPER OF A ROBOTIC DEVICE,â which claims the benefit under 35 U.S.C. Â§ 119(e) of U.S. Provisional Patent Application No. 63/434,507, filed Dec. 22, 2022, and titled, âMETHODS AND APPARATUS FOR CONTROLLING A GRIPPER OF A ROBOTIC DEVICE,â the entire contents of each of which is incorporated by reference herein.

FIELD OF THE INVENTION

This disclosure relates to techniques for controlling a gripper of a robotic device.

BACKGROUND

A robot is generally defined as a reprogrammable and multifunctional manipulator designed to move material, parts, tools, and/or specialized devices (e.g., via variable programmed motions) for performing tasks. Robots may include manipulators that are physically anchored (e.g., industrial robotic arms), mobile devices that move throughout an environment (e.g., using legs, wheels, or traction-based mechanisms), or some combination of one or more manipulators and one or more mobile devices. Robots are currently used in a variety of industries, including, for example, manufacturing, warehouse logistics, transportation, hazardous environments, exploration, and healthcare.

SUMMARY

During robotic pick-and-place tasks in logistics scenarios (e.g., depalletizing, truck unloading, order building, etc.), a firm grasp of the object being manipulated is often desirable. For a robotic manipulator with a vacuum-based gripper, grasp quality may be related to the number of vacuum assemblies of the gripper that are able to form a good seal with the object being manipulated. A high-quality grasp may be enabled by engaging a large number of vacuum assemblies with the object at certain times and/or deactivating vacuum assemblies that fail to make a good seal with the object at certain times. In some scenarios, a vacuum assembly may initially fail to make a good seal with the object and may be deactivated, but may later be capable of making a good seal (or better seal) if it were reactivated (e.g., due to changing conditions during a grasp). In some embodiments described herein individual vacuum assemblies of a vacuum-based gripper are controlled in an intelligent manner to facilitate obtaining a secure grasp on one or more objects prior to and/or during manipulation of the object.

In one aspect, the invention features a method. The method includes activating a plurality of vacuum assemblies of a robotic gripper to grasp one or more objects, disabling one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold, assigning a score to each of the one or more disabled vacuum assemblies, reactivating the one or more disabled vacuum assemblies in an order based, at least in part, on the assigned scores, and grasping the one or more objects with the robotic gripper when a grasp quality of the robotic gripper is higher than a second threshold.

In some embodiments, the method further includes receiving mask information associating different vacuum assemblies of the robotic gripper to different objects to be grasped, and activating the plurality of vacuum assemblies comprises activating the plurality of vacuum assemblies based, at least in part, on the mask information.

In some embodiments, the one or more objects includes a first object and a second object, and the mask information associates a first set of vacuum assemblies with the first object and a second set of vacuum assemblies with the second object. The method may further include releasing the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies, waiting for an amount of time, and releasing the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

In some embodiments, assigning a score to each of the one or more disabled vacuum assemblies comprises assigning a score based, at least in part, on a location of the vacuum assembly in the robotic gripper. In some embodiments, assigning a score to each of the one or more disabled vacuum assemblies comprises assigning a score based, at least in part, on a number of neighboring active vacuum assemblies. In some embodiments, assigning a score to each of the one or more disabled vacuum assemblies comprises assigning a score based, at least in part, on whether the vacuum assembly is located at an edge of the robotic gripper. In some embodiments, assigning a score to each of the one or more disabled vacuum assemblies comprises assigning a score based, at least in part, on a number of retry attempts for the vacuum assembly.

In some embodiments, the method further includes receiving force information associated with the robotic gripper, and releasing the one or more objects in response to determining that the force information is greater than a third threshold. In some embodiments, the force information includes radial force information experienced by a wrist assembly coupled to the robotic gripper. In some embodiments, the third threshold is 400 Newtons.

In some embodiments, the method further includes determining that a first set of vacuum assemblies are non-functional, wherein the first set includes one or more vacuum assemblies, and reactivating the one or more disabled vacuum assemblies is performed only for vacuum assemblies not included in the first set. In some embodiments, the method further includes displaying on a user interface, an indication of the vacuum assemblies included in the first set. In some embodiments, determining that a first set of vacuum assemblies are non-functional comprises activating a first vacuum assembly of the plurality of vacuum assemblies, measuring a pressure level within the first vacuum assembly when activated, and including the first vacuum assembly in the first set when the measured pressure level is less than a third threshold.

In one aspect the invention features a controller for a robotic gripper. The controller includes at least one computer processor programmed to activate a plurality of vacuum assemblies of a robotic gripper to grasp one or more objects, disable one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold, assign a score to each of the one or more disabled vacuum assemblies, reactivate the one or more disabled vacuum assemblies in an order based, at least in part, on the assigned scores, and grasp the one or more objects with the robotic gripper when a grasp quality of the robotic gripper is higher than a second threshold.

In some embodiments, the at least one computer processor is further programmed to receive mask information associating different vacuum assemblies of the robotic gripper to different objects to be grasped, and activating the plurality of vacuum assemblies comprises activating the plurality of vacuum assemblies based, at least in part, on the mask information.

In some embodiments, the one or more objects includes a first object and a second object, and the mask information associates a first set of vacuum assemblies with the first object and a second set of vacuum assemblies with the second object. The at least one computer processor may be further programmed to release the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies, wait for an amount of time, and release the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

In some embodiments, the at least one computer processor is further programmed to receive force information associated with the robotic gripper, and release the one or more objects in response to determining that the force information is greater than a third threshold. In some embodiments, the force information includes radial force information experienced by a wrist assembly coupled to the robotic gripper. In some embodiments, the third threshold is 400 Newtons.

In some embodiments, the at least one computer processor is further programmed to determine that a first set of vacuum assemblies are non-functional, wherein the first set includes one or more vacuum assemblies, and reactivating the one or more disabled vacuum assemblies is performed only for vacuum assemblies not included in the first set. In some embodiments, the at least one computer processor is further programmed to display on a user interface, an indication of the vacuum assemblies included in the first set. In some embodiments, determining that a first set of vacuum assemblies are non-functional comprises activating a first vacuum assembly of the plurality of vacuum assemblies, measuring a pressure level within the first vacuum assembly when activated, and including the first vacuum assembly in the first set when the measured pressure level is less than a third threshold.

In one aspect, the invention features a mobile robotic device. The mobile robotic device includes a robotic gripper comprising a plurality of vacuum assemblies and at least one pressure sensor associated with each vacuum assembly of the plurality of vacuum assemblies, and at least one computer processor. The at least one computer processor is programmed to activate the plurality of vacuum assemblies of the robotic gripper to grasp one or more objects, disable one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold, assign a score to each of the one or more disabled vacuum assemblies, reactivate the one or more disabled vacuum assemblies in an order based, at least in part, on the assigned scores, and grasp the one or more objects with the robotic gripper when a grasp quality of the robotic gripper is higher than a second threshold.

In some embodiments, the one or more objects includes a first object and a second object, and the mask information associates a first set of vacuum assemblies with the first object and a second set of vacuum assemblies with the second object. The at least one computer processor may be further programmed to release the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies, wait for an amount of time, and release the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

In some embodiments, the mobile robotic device further includes a force sensor coupled to the robotic gripper, and the at least one computer processor is further programmed to receive force information from the force sensor, and release the one or more objects in response to determining that the force information is greater than a third threshold. In some embodiments, the force information includes radial force information experienced by a wrist assembly coupled to the robotic gripper. In some embodiments, the third threshold is 400 Newtons.

In some embodiments, the at least one computer processor is further programmed to determine that a first set of vacuum assemblies are non-functional, wherein the first set includes one or more vacuum assemblies, and reactivating the one or more disabled vacuum assemblies is performed only for vacuum assemblies not included in the first set. In some embodiments, the mobile robotic device further includes a user interface, and the at least one computer processor is further programmed to display on the user interface, an indication of the vacuum assemblies included in the first set. In some embodiments, determining that a first set of vacuum assemblies are non-functional comprises activating a first vacuum assembly of the plurality of vacuum assemblies, determining, using the pressure sensor associated with the first vacuum assembly, a pressure level within the first vacuum assembly when activated, and including the first vacuum assembly in the first set when the measured pressure level is less than a third threshold.

BRIEF DESCRIPTION OF DRAWINGS

The advantages of the invention, together with further advantages, may be better understood by referring to the following description taken in conjunction with the accompanying drawings. The drawings are not necessarily to scale, and emphasis is instead generally placed upon illustrating the principles of the invention.

FIGS. 1 A and 1 B are perspective views of a robot, according to an illustrative embodiment of the invention.

FIG. 2 A depicts robots performing different tasks within a warehouse environment, according to an illustrative embodiment of the invention.

FIG. 2 B depicts a robot unloading boxes from a truck and placing them on a conveyor belt, according to an illustrative embodiment of the invention.

FIG. 2 C depicts a robot performing an order building task in which the robot places boxes onto a pallet, according to an illustrative embodiment of the invention.

FIG. 3 is a perspective view of a robot, according to an illustrative embodiment of the invention.

FIG. 4 depicts an end effecto

CROSS-REFERENCE TO RELATED APPLICATIONS

FIELD OF THE INVENTION

This disclosure relates to techniques for controlling a gripper of a robotic device.

BACKGROUND

SUMMARY

In some embodiments, the one or more objects includes a first object and a second object, and the mask information associates a first set of vacuum assemblies with the first object and a second set of vacuum assemblies with the second object. The at least one computer processor may be further programmed to release the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies, wait for an amount of time, and release the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

In some embodiments, the at least one computer processor is further programmed to determine that a first set of vacuum assemblies are non-functional, wherein the first set includes one or more vacuum assemblies, and reactivating the one or more disabled vacuum assemblies is performed only for vacuum assemblies not included in the first set. In some embodiments, the mobile robotic device further includes a user interface, and the at least one computer processor is further programmed to display on the user interface, an indication of the vacuum assemblies included in the first set. In some embodiments, determining that a first set of vacuum assemblies are non-functional comprises activating a first vacuum assembly of the plurality of vacuum assemblies, determining, using the pressure sensor associated with the first vacuum assembly, a pressure level within the first vacuum assembly when activated, and including the first vacuum assembly in the first set when the measured pressure level is less than a third threshold.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1 A and 1 B are perspective views of a robot, according to an illustrative embodiment of the invention.

FIG. 2 A depicts robots performing different tasks within a warehouse environment, according to an illustrative embodiment of the invention.

FIG. 2 B depicts a robot unloading boxes from a truck and placing them on a conveyor belt, according to an illustrative embodiment of the invention.

FIG. 2 C depicts a robot performing an order building task in which the robot places boxes onto a pallet, according to an illustrative embodiment of the invention.

FIG. 3 is a perspective view of a robot, according to an illustrative embodiment of the invention.

FIG. 4 depicts an end effector of a robot, according to an illustrative embodiment of the invention.

FIG. 5 schematically illustrates a controller architecture configured to control individual cups of a vacuum-based gripper, according to an illustrative embodiment of the invention.

FIG. 6 schematically illustrates functions of processor components of the controller architecture shown in FIG. 5 .

FIG. 7 is a flowchart of a process for controlling individual cups of a vacuum-based gripper of a robot to grasp an object with sufficient grasp quality to move the object, according to an illustrative embodiment of the invention.

FIG. 8 is a flowchart of a process for determining a reactivation strategy for retrying activation of a plurality of disabled cups of a gripper, according to an illustrative embodiment of the invention.

FIG. 9 schematically illustrates how different cups on a gripper may be associated with different priorities for reactivation, according to an illustrative embodiment of the invention.

FIG. 10 is a visualization of a layout of cups of a vacuum-based gripper that may be used to convey activation information for each of the plurality of cups, according to an illustrative embodiment of the invention.

FIG. 11 is a flowchart of a process for determining whether to release a grasped object based on one or more force measurements associated with an end effector of a robot, according to an illustrative embodiment of the invention.

FIG. 12 illustrates an example configuration of a robotic device, according to an illustrative embodiment of the invention.

DETAILED DESCRIPTION

Suction-based grippers for robotic devices often include a large number of vacuum assemblies designed to engage with an object to grasp the object. Independent control of the state of individual vacuum assemblies of the gripper provides for a highly configurable grasping system in which various numbers, locations and/or type of vacuum assemblies in the gripper can be activated or deactivated to grasp objects having a wide range of shapes, sizes and weights. Some embodiments of the present disclosure relate to techniques for controlling activation, reactivation, and/or deactivation of individual vacuum assemblies of a robotic gripper to optimally grasp or release one or more objects using the gripper.

Robots can be configured to perform a number of tasks in an environment in which they are placed. Exemplary tasks may include interacting with objects and/or elements of the environment. Notably, robots are becoming popular in warehouse and logistics operations. Before robots were introduced to such spaces, many operations were performed manually. For example, a person might manually unload boxes from a truck onto one end of a conveyor belt, and a second person at the opposite end of the conveyor belt might organize those boxes onto a pallet. The pallet might then be picked up by a forklift operated by a third person, who might drive to a storage area of the warehouse and drop the pallet for a fourth person to remove the individual boxes from the pallet and place them on shelves in a storage area. Some robotic solutions have been developed to automate many of these functions. Such robots may either be specialist robots (i.e., designed to perform a single task or a small number of related tasks) or generalist robots (i.e., designed to perform a wide variety of tasks). To date, both specialist and generalist warehouse robots have been associated with significant limitations.

For example, because a specialist robot may be designed to perform a single task (e.g., unloading boxes from a truck onto a conveyor belt), while such specialized robots may be efficient at performing their designated task, they may be unable to perform other related tasks. As a result, either a person or a separate robot (e.g., another specialist robot designed for a different task) may be needed to perform the next task(s) in the sequence. As such, a warehouse may need to invest in multiple specialized robots to perform a sequence of tasks, or may need to rely on a hybrid operation in which there are frequent robot-to-human or human-to-robot handoffs of objects.

In contrast, while a generalist robot may be designed to perform a wide variety of tasks (e.g., unloading, palletizing, transporting, depalletizing, and/or storing), such generalist robots may be unable to perform individual tasks with high enough efficiency or accuracy to warrant introduction into a highly streamlined warehouse operation. For example, while mounting an off-the-shelf robotic manipulator onto an off-the-shelf mobile robot might yield a system that could, in theory, accomplish many warehouse tasks, such a loosely integrated system may be incapable of performing complex or dynamic motions that require coordination between the manipulator and the mobile base, resulting in a combined system that is inefficient and inflexible.

Typical operation of such a system within a warehouse environment may include the mobile base and the manipulator operating sequentially and (partially or entirely) independently of each other. For example, the mobile base may first drive toward a stack of boxes with the manipulator powered down. Upon reaching the stack of boxes, the mobile base may come to a stop, and the manipulator may power up and begin manipulating the boxes as the base remains stationary. After the manipulation task is completed, the manipulator may again power down, and the mobile base may drive to another destination to perform the next task.

In such systems, the mobile base and the manipulator may be regarded as effectively two separate robots that have been joined together. Accordingly, a controller associated with the manipulator may not be configured to share information with, pass commands to, or receive commands from a separate controller associated with the mobile base. As such, such a poorly integrated mobile manipulator robot may be forced to operate both its manipulator and its base at suboptimal speeds or through suboptimal trajectories, as the two separate controllers struggle to work together. Additionally, while certain limitations arise from an engineering perspective, additional limitations must be imposed to comply with safety regulations. For example, if a safety regulation requires that a mobile manipulator must be able to be completely shut down within a certain period of time when a human enters a region within a certain distance of the robot, a loosely integrated mobile manipulator robot may not be able to act sufficiently quickly to ensure that both the manipulator and the mobile base (individually and in aggregate) do not threaten the human. To ensure that such loosely integrated systems operate within required safety constraints, such systems are forced to operate at even slower speeds or to execute even more conservative trajectories than those limited speeds and trajectories as already imposed by the engineering problem. As such, the speed and efficiency of generalist robots performing tasks in warehouse environments to date have been limited.

In view of the above, a highly integrated mobile manipulator robot with system-level mechanical design and holistic control strategies between the manipulator and the mobile base may provide certain benefits in warehouse and/or logistics operations. Such an integrated mobile manipulator robot may be able to perform complex and/or dynamic motions that are unable to be achieved by conventional, loosely integrated mobile manipulator systems. As a result, this type of robot may be well suited to perform a variety of different tasks (e.g., within a warehouse environment) with speed, agility, and efficiency.

Example Robot Overview

In this section, an overview of some components of one embodiment of a highly integrated mobile manipulator robot configured to perform a variety of tasks is provided to explain the interactions and interdependencies of various subsystems of the robot. Each of the various subsystems, as well as control strategies for operating the subsystems, are described in further detail in the following sections.

FIGS. 1 A and 1 B are perspective views of a robot 100 , according to an illustrative embodiment of the invention. The robot 100 includes a mobile base 110 and a robotic arm 130 . The mobile base 110 includes an omnidirectional drive system that enables the mobile base to translate in any direction within a horizontal plane as well as rotate about a vertical axis perpendicular to the plane. Each wheel 112 of the mobile base 110 is independently steerable and independently drivable. The mobile base 110 additionally includes a number of distance sensors 116 that assist the robot 100 in safely moving about its environment. The robotic arm 130 is a 6 degree of freedom (6-DOF) robotic arm including three pitch joints and a 3-DOF wrist. An end effector 150 is disposed at the distal end of the robotic arm 130 . The robotic arm 130 is operatively coupled to the mobile base 110 via a turntable 120 , which is configured to rotate relative to the mobile base 110 . In addition to the robotic arm 130 , a perception mast 140 is also coupled to the turntable 120 , such that rotation of the turntable 120 relative to the mobile base 110 rotates both the robotic arm 130 and the perception mast 140 . The robotic arm 130 is kinematically constrained to avoid collision with the perception mast 140 . The perception mast 140 is additionally configured to rotate relative to the turntable 120 , and includes a number of perception modules 142 configured to gather information about one or more objects in the robot's environment. The integrated structure and system-level design of the robot 100 enable fast and efficient operation in a number of different applications, some of which are provided below as examples.

FIG. 2 A depicts robots 10 a , 10 b , and 10 c performing different tasks within a warehouse environment. A first robot 10 a is inside a truck (or a container), moving boxes 11 from a stack within the truck onto a conveyor belt 12 (this particular task will be discussed in greater detail below in reference to FIG. 2 B ). At the opposite end of the conveyor belt 12 , a second robot 10 b organizes the boxes 11 onto a pallet 13 . In a separate area of the warehouse, a third robot 10 c picks boxes from shelving to build an order on a pallet (this particular task will be discussed in greater detail below in reference to FIG. 2 C ). The robots 10 a , 10 b , and 10 c can be different instances of the same robot or similar robots. Accordingly, the robots described herein may be understood as specialized multi-purpose robots, in that they are designed to perform specific tasks accurately and efficiently, but are not limited to only one or a small number of tasks.

FIG. 2 B depicts a robot 20 a unloading boxes 21 from a truck 29 and placing them on a conveyor belt 22 . In this box picking application (as well as in other box picking applications), the robot 20 a repetitiously picks a box, rotates, places the box, and rotates back to pick the next box. Although robot 20 a of FIG. 2 B is a different embodiment from robot 100 of FIGS. 1 A and 1 B , referring to the components of robot 100 identified in FIGS. 1 A and 1 B will case explanation of the operation of the robot 20 a in FIG. 2 B .

During operation, the perception mast of robot 20 a (analogous to the perception mast 140 of robot 100 of FIGS. 1 A and 1 B ) may be configured to rotate independently of rotation of the turntable (analogous to the turntable 120 ) on which it is mounted to enable the perception modules (akin to perception modules 142 ) mounted on the perception mast to capture images of the environment that enable the robot 20 a to plan its next movement while simultaneously executing a current movement. For example, while the robot 20 a is picking a first box from the stack of boxes in the truck 29 , the perception modules on the perception mast may point at and gather information about the location where the first box is to be placed (e.g., the conveyor belt 22 ). Then, after the turntable rotates and while the robot 20 a is placing the first box on the conveyor belt, the perception mast may rotate (relative to the turntable) such that the perception modules on the perception mast point at the stack of boxes and gather information about the stack of boxes, which is used to determine the second box to be picked. As the turntable rotates back to allow the robot to pick the second box, the perception mast may gather updated information about the area surrounding the conveyor belt. In this way, the robot 20 a may parallelize tasks which may otherwise have been performed sequentially, thus enabling faster and more efficient operation.

Also of note in FIG. 2 B is that the robot 20 a is working alongside humans (e.g., workers 27 a and 27 b ). Given that the robot 20 a is configured to perform many tasks that have traditionally been performed by humans, the robot 20 a is designed to have a small footprint, both to enable access to areas designed to be accessed by humans, and to minimize the size of a safety field around the robot (e.g., into which humans are prevented from entering and/or which are associated with other safety controls, as explained in greater detail below).

FIG. 2 C depicts a robot 30 a performing an order building task, in which the robot 30 a places boxes 31 onto a pallet 33 . In FIG. 2 C , the pallet 33 is disposed on top of an autonomous mobile robot (AMR) 34 , but it should be appreciated that the capabilities of the robot 30 a described in this example apply to building pallets not associated with an AMR. In this task, the robot 30 a picks boxes 31 disposed above, below, or within shelving 35 of the warehouse and places the boxes on the pallet 33 . Certain box positions and orientations relative to the shelving may suggest different box picking strategies. For example, a box located on a low shelf may simply be picked by the robot by grasping a top surface of the box with the end effector of the robotic arm (thereby executing a âtop pickâ). However, if the box to be picked is on top of a stack of boxes, and there is limited clearance between the top of the box and the bottom of a horizontal divider of the shelving, the robot may opt to pick the box by grasping a side surface (thereby executing a âface pickâ).

To pick some boxes within a constrained environment, the robot may need to carefully adjust the orientation of its arm to avoid contacting other boxes or the surrounding shelving. For example, in a typical âkeyhole problemâ, the robot may only be able to access a target box by navigating its arm through a small space or confined area (akin to a keyhole) defined by other boxes or the surrounding shelving. In such scenarios, coordination between the mobile base and the arm of the robot may be beneficial. For instance, being able to translate the base in any direction allows the robot to position itself as close as possible to the shelving, effectively extending the length of its arm (compared to conventional robots without omnidirectional drive which may be unable to navigate arbitrarily close to the shelving). Additionally, being able to translate the base backwards allows the robot to withdraw its arm from the shelving after picking the box without having to adjust joint angles (or minimizing the degree to which joint angles are adjusted), thereby enabling a simple solution to many keyhole problems.

The tasks depicted in FIGS. 2 A- 2 C are only a few examples of applications in which an integrated mobile manipulator robot may be used, and the present disclosure is not limited to robots configured to perform only these specific tasks. For example, the robots described herein may be suited to perform tasks including, but not limited to: removing objects from a truck or container; placing objects on a conveyor belt; removing objects from a conveyor belt; organizing objects into a stack; organizing objects on a pallet; placing objects on a shelf; organizing objects on a shelf; removing objects from a shelf; picking objects from the top (e.g., performing a âtop pickâ); picking objects from a side (e.g., performing a âface pickâ); coordinating with other mobile manipulator robots; coordinating with other warehouse robots (e.g., coordinating with AMRs); coordinating with humans; and many other tasks.

Example Robotic Arm

FIG. 3 is a perspective view of a robot 400 , according to an illustrative embodiment of the invention. The robot 400 includes a mobile base 410 and a turntable 420 rotatably coupled to the mobile base. A robotic arm 430 is operatively coupled to the turntable 420 , as is a perception mast 440 . The perception mast 440 includes an actuator 444 configured to enable rotation of the perception mast 440 relative to the turntable 420 and/or the mobile base 410 , so that a direction of the perception modules 442 of the perception mast may be independently controlled.

The robotic arm 430 of FIG. 3 is a 6-DOF robotic arm. When considered in conjunction with the turntable 420 (which is configured to yaw relative to the mobile base about a vertical axis parallel to the Z axis), the arm/turntable system may be considered a 7-DOF system. The 6-DOF robotic arm 430 includes three pitch joints 432 , 434 , and 436 , and a 3-DOF wrist 438 which, in some embodiments, may be a spherical 3-DOF wrist.

Starting at the turntable 420 , the robotic arm 430 includes a turntable offset 422 , which is fixed relative to the turntable 420 . A distal portion of the turntable offset 422 is rotatably coupled to a proximal portion of a first link 433 at a first joint 432 . A distal portion of the first link 433 is rotatably coupled to a proximal portion of a second link 435 at a second joint 434 . A distal portion of the second link 435 is rotatably coupled to a proximal portion of a third link 437 at a third joint 436 . The first, second, and third joints 432 , 434 , and 436 are associated with first, second, and third axes 432 a , 434 a , and 436 a , respectively.

The first, second, and third joints 432 , 434 , and 436 are additionally associated with first, second, and third actuators (not labeled) which are configured to rotate a link about an axis. Generally, the nth actuator is configured to rotate the nth link about the nth axis associated with the nth joint. Specifically, the first actuator is configured to rotate the first link 433 about the first axis 432 a associated with the first joint 432 , the second actuator is configured to rotate the second link 435 about the second axis 434 a associated with the second joint 434 , and the third actuator is configured to rotate the third link 437 about the third axis 436 a associated with the third joint 436 . In the embodiment shown in FIG. 3 , the first, second, and third axes 432 a , 434 a , and 436 a are parallel (and, in this case, are all parallel to the X axis). In the embodiment shown in FIG. 3 , the first, second, and third joints 432 , 434 , and 436 are all pitch joints.

In some embodiments, a robotic arm of a highly integrated mobile manipulator robot may include a different number of degrees of freedom than the robotic arms discussed above. Additionally, a robotic arm need not be limited to a robotic arm with three pitch joints and a 3-DOF wrist. A robotic arm of a highly integrated mobile manipulator robot may include any suitable number of joints of any suitable type, whether revolute or prismatic. Revolute joints need not be oriented as pitch joints, but rather may be pitch, roll, yaw, or any other suitable type of joint.

Returning to FIG. 3 , the robotic arm 430 includes a wrist 438 . As noted above, the wrist 438 is a 3-DOF wrist, and in some embodiments may be a spherical 3-DOF wrist. The wrist 438 is coupled to a distal portion of the third link 437 . The wrist 438 includes three actuators configured to rotate an end effector 450 coupled to a distal portion of the wrist 438 about three mutually perpendicular axes. Specifically, the wrist may include a first wrist actuator configured to rotate the end effector relative to a distal link of the arm (e.g., the third link 437 ) about a first wrist axis, a second wrist actuator configured to rotate the end effector relative to the distal link about a second wrist axis, and a third wrist actuator configured to rotate the end effector relative to the distal link about a third wrist axis. The first, second, and third wrist axes may be mutually perpendicular. In embodiments in which the wrist is a spherical wrist, the first, second, and third wrist axes may intersect.

In some embodiments, an end effector may be associated with one or more sensors. For example, a force/torque sensor may measure forces and/or torques (e.g., wrenches) applied to the end effector. Alternatively or additionally, a sensor may measure wrenches applied to a wrist of the robotic arm by the end effector (and, for example, an object grasped by the end effector) as the object is manipulated. Signals from these (or other) sensors may be used during mass estimation and/or path planning operations. In some embodiments, sensors associated with an end effector may include an integrated force/torque sensor, such as a 6-axis force/torque sensor. In some embodiments, separate sensors (e.g., separate force and torque sensors) may be employed. Some embodiments may include only force sensors (e.g., uniaxial force sensors, or multi-axis force sensors), and some embodiments may include only torque sensors. In some embodiments, an end effector may be associated with a custom sensing arrangement. For example, one or more sensors (e.g., one or more uniaxial sensors) may be arranged to enable sensing of forces and/or torques along multiple axes. An end effector (or another portion of the robotic arm) may additionally include any appropriate number or configuration of cameras, distance sensors, pressure sensors, light sensors, or any other suitable sensors, whether related to sensing characteristics of the payload or otherwise, as the disclosure is not limited in this regard.

FIG. 4 illustrates an example of a robotic component 401 , which corresponds to an end-effector portion of a mobile manipulator robot. Robotic component 401 includes a wrist assembly 402 coupled to an arm of the robot, and a gripper 403 coupled to the wrist assembly 402 . Gripper 403 may be configured as a vacuum-based gripper in which suction is applied through a plurality of suction cup assemblies 425 (also referred to herein as âvacuum assembliesâ) configured to grasp an object (e.g., a box) when suction is applied through them. Each of the suction cup assemblies 425 may include a pressure sensor configured to sense a pressure within the suction cup assembly when suction is applied through the assembly. Pressure signals sensed by the pressure sensors and/or information based on such pressure signals may be provided to a controller to detect suction cup assemblies with poor seals (e.g., leaks), which in turn, may be used by the controller to deactivate and possibly retry activation of the suction cup assemblies in an attempt to improve a grasp quality of the gripper 403 , as described herein.

FIG. 5 schematically illustrates a computing architecture 500 that may be used to implement one or more of the techniques described herein. As shown, computing architecture 500 includes a first computing device 510 and a second computing device 520 . In some embodiments, first computing device 510 may be implemented as a portion of a controller associated with a mobile robot. The controller may be configured to control the operation of various systems in the robot by, for example, providing control instructions to one or more actuators associated with an arm and/or wrist of the robot to control their movement. In some embodiments, second computing device 520 may be implemented as firmware that translates commands received from the first computing device 510 into control actions on a robot hardware component. In the example system described herein, the second computing device 520 may be implemented as firmware that directly controls the valve states (i.e., activated or deactivated) of individual vacuum assemblies within a gripper of the robot. Second computing device 520 may also receive feedback about the state of the individual vacuum assemblies in the gripper and provide the feedback to the first computing device 510 . For instance, pressure sensor signals sensed by a pressure sensor within each of the vacuum assemblies may be provided by second computing device 520 to first computing device 510 to determine vacuum assemblies with poor seal qualities (e.g., due to leaks).

In some embodiments, the first computing device 510 may include multiple components including one or more hardware computing processors programmed to carry out one or more of the methods described herein. In the example shown in FIG. 5 , first computing device 510 includes a supervisor processor, configured to control activation/deactivation of the individual cups in the gripper and to process feedback from the gripper. First computing device 510 may also include a server interface component configured to receive information from the supervisor processor and provide control instructions to, and receive feedback information (e.g., vacuum assembly pressures) from, second computing device 520 .

FIG. 6 schematically illustrates further details regarding the supervisor processor and server component included in the example first computing device 510 shown in FIG. 5 . As shown in FIG. 6 , supervisor processor 610 may be configured to receive mask information from a grasp planning process. A gripper of a robot may be configured to grasp multiple objects (e.g., boxes) at the same time by, for example, selectively activating groups of vacuum assemblies, each of the groups being associated with one of the objects. In some embodiments, when a grasp planning process has determined to grasp multiple objects at the same time (also referred to as a âmulti-pickâ operation), the grasp planning process may provide mask information to the supervisor processor 610 . The mask information may describe which suction cups of the gripper should be used to grasp each of the objects. Alternatively, the mask information may describe the size of the multiple objects to be grasped, and the supervisor processor 610 may determine which vacuum assemblies to activate based on the received mask information. By providing the supervisor processor 610 with mask information, the supervisor processor 610 is enabled to make decisions on how to individually control the grasping and releasing of the multiple objects in accordance with the mask information. For instance, as described in more detail below, by controlling sets of vacuum assemblies as a unit based on the mask information, multiple objects simultaneously grasped by a gripper may be released at separate times, which may be advantageous in situations where, for example, boxes picked from a stack in a container such as a truck are being placed by the robot on a conveyor.

As shown in FIG. 6 , supervisor processor 610 may be configured to determine grasp operations including, but not limited to, a start grasp operation, a drop object operation, or a release all (e.g., drop all objects) operation. Supervisor processor 610 may further be configured to determine operations to modify a grasp once attempted including, but not limited to, turning off cups with a poor seal quality, identifying and ignoring non-functional cups (e.g., those with broken solenoids or valves), and cup retrying to reactivate cups that were previously deactivated due to poor seal quality. Upon making decisions about how to activate the vacuum assemblies of the gripper, supervisor processor 610 may provide information to server interface 620 , which may translate those decisions into control instructions that may be sent to processing software associated with the vacuum assembly hardware (e.g., second computing device 520 shown in FIG. 5 ). Server interface 620 may also be configured to receive feedback, such as pressure sensor information from the gripper, which may be provided to supervisor processor 610 to facilitate decision making about cup control as described herein.

FIG. 7 is a flowchart of a process 700 for controlling operation of a gripper of a mobile robot in accordance with some embodiments of the present disclosure. In some embodiments, process 700 may be implemented using the computer architecture 500 illustrated and described in connection with FIGS. 5 and 6 . Process 700 begins in act 710 , where mask information is received, for example, from a grasp planning process as described above. It should be appreciated that mask information may not be received in all embodiments. For instance, if the robot is not configured to perform multi-picking of objects or if the current grasp is not a multi-pick operation, mask information may not be received and used to determine a cup control strategy. Process 700 then proceeds to act 712 , where a cup control strategy is determined. When mask information is received, determining a cup control strategy may involve assigning different vacuum assemblies in the gripper to the different mask layouts specified in the mask information such that grasping/releasing of multiple objects with the gripper can be controlled independently. In embodiments in which mask information is not received, determining a cup control strategy may involve determining to sequentially or simultaneously activate all or a portion of the vacuum assemblies of the gripper to grasp an object in accordance with grasp strategy information provided as input to the cup control strategy process.

Process 700 then proceeds to act 714 , where a grasp of one or more objects is attempted based, at least in part, on the cup control strategy determined in act 712 . For instance, the cup control strategy may be translated into control commands that are sent to firmware associated with the gripper to activate valves of particular vacuum assemblies of the gripper. Process 700 then proceeds to act 716 , where one or more vacuum assemblies having a poor seal quality are disabled or deactivated. By disabling vacuum assemblies with poor seal qualities, the amount of vacuum provided to the other assemblies with good seal quality may be increased to provide a more secure grasp. In some embodiments, the decision about which cups to disable in act 716 may be made in part, based on pressure information received from pressure sensors within individual vacuum assemblies.

Process 700 then proceeds to act 718 , where it is determined whether the overall grasp quality is sufficient to move the object(s) with a planned trajectory. If it is determined in act 718 that the grasp quality is not sufficient, process 700 proceeds to act 720 , where a strategy for reactivating one or more of the disabled vacuum assemblies is determined. In some embodiments, the reactivation of disabled vacuum assemblies is performed in accordance with one or more rules based on a likelihood that reactivation of particular assemblies is likely to improve the overall grasp quality. An example process for determining a strategy for reactivating disabled vacuum assemblies in accordance with some embodiments is described in connection with FIG. 8 . After a strategy for reactivating one or more disabled vacuum assemblies is determined in act 720 , process 700 returns to act 714 where the grasp of the gripper is modified by carrying out the reactivation strategy determined in act 720 . Acts 716 - 720 may be repeated until it is determined in act 718 that the grasp quality is sufficient to move the object(s) with the planned trajectory.

When it is determined in act 718 that the grasp quality is sufficient, process 700 proceeds to act 722 , where the robot is controlled to move the grasped object from a first location (e.g., a stack of boxes in a truck container) to a second location (e.g., above a conveyor). One or more of the cups of the gripper may lose their quality seal with an object as the moved. In such instances, one or more of the cups with a poor seal may be reactivated mid-motion in an attempt to improve the grasp of the object. Additionally, in some embodiments, information about the grasp quality and/or information about forces on the gripper, wrist or arm of the robot may be used to determine how to move the object through the trajectory from the first location to the second location. For instance, if the weight of a face-picked object is estimated to be 50 lb and it is known that the robot can safely move face-picked objects up to 55 lb, the arm of the robot may be controlled to move the object along the trajectory in act 722 using a slow speed. If however, the weight of the face-picked object is estimated to be 5 lb, the robot may be controlled to move the object along the trajectory in act 722 using a faster (e.g., safe maximum) speed. After the object(s) have bee

CLAIMS

Claims ( 22 )

1 . A method, comprising:

activating a plurality of vacuum assemblies of a robotic gripper to grasp one or more objects; disabling one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold; and reactivating the one or more disabled vacuum assemblies in an order based, at least in part, on a proximity of the disabled vacuum assembly to at least one activated vacuum assembly of the plurality of vacuum assemblies having a seal quality with the one or more objects that is greater than the first threshold.

2 . The method of claim 1 , further comprising:

receiving mask information associating different vacuum assemblies of the robotic gripper to different objects to be grasped, wherein activating the plurality of vacuum assemblies comprises activating the plurality of vacuum assemblies based, at least in part, on the mask information.

3 . The method of claim 2 , wherein

the one or more objects includes a first object and a second object, and the mask information associates a first set of vacuum assemblies with the first object and a second set of vacuum assemblies with the second object, the method further comprising:

releasing the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies;

waiting for an amount of time; and

releasing the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

4 - 7 . (canceled)

8 . The method of claim 1 , further comprising:

receiving force information associated with the robotic gripper; and releasing the one or more objects in response to determining that the force information is greater than a second threshold.

9 . The method of claim 8 , wherein the force information includes radial force information experienced by a wrist assembly coupled to the robotic gripper.

10 . The method of claim 1 , further comprising:

determining that a first set of vacuum assemblies are non-functional, wherein the first set includes one or more vacuum assemblies, wherein reactivating the one or more disabled vacuum assemblies is performed only for vacuum assemblies not included in the first set.

11 . The method of claim 10 , further comprising:

displaying on a user interface, an indication of the vacuum assemblies included in the first set.

12 . The method of claim 10 , wherein determining that a first set of vacuum assemblies are non-functional comprises:

activating a first vacuum assembly of the plurality of vacuum assemblies; measuring a pressure level within the first vacuum assembly when activated; and including the first vacuum assembly in the first set when the measured pressure level is less than a second threshold.

13 . A controller for a robotic gripper, the controller comprising:

at least one computer processor programmed to:

activate a plurality of vacuum assemblies of a robotic gripper to grasp one or more objects;

disable one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold; and

reactivate the one or more disabled vacuum assemblies in an order based, at least in part, on a proximity of the disabled vacuum assembly to at least one activated vacuum assembly of the plurality of vacuum assemblies having a seal quality with the one or more objects that is greater than the first threshold.

14 . The controller of claim 13 , wherein the at least one computer processor is further programmed to:

receive mask information associating different vacuum assemblies of the robotic gripper to different objects to be grasped, wherein activating the plurality of vacuum assemblies comprises activating the plurality of vacuum assemblies based, at least in part, on the mask information.

15 . The controller of claim 14 , wherein

the one or more objects includes a first object and a second object, the mask information associates a first set of vacuum assemblies with the first object and a second set of vacuum assemblies with the second object, and the at least one computer processor is further programmed to:

release the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies;

wait for an amount of time; and

release the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

16 . (canceled)

17 . The controller of claim 13 , wherein the at least one computer processor is further programmed to:

receive force information associated with the robotic gripper; and release the one or more objects in response to determining that the force information is greater than a second threshold.

18 . The controller of claim 13 , wherein the at least one computer processor is further programmed to:

determine that a first set of vacuum assemblies are non-functional, wherein the first set includes one or more vacuum assemblies, wherein reactivating the one or more disabled vacuum assemblies is performed only for vacuum assemblies not included in the first set.

19 . The controller of claim 18 , wherein determining that a first set of vacuum assemblies are non-functional comprises:

20 . A mobile robotic device, comprising:

a robotic gripper comprising a plurality of vacuum assemblies and at least one pressure sensor associated with each vacuum assembly of the plurality of vacuum assemblies; and at least one computer processor programmed to:

activate the plurality of vacuum assemblies of the robotic gripper to grasp one or more objects;

disable one or more of the plurality of vacuum assemblies having a seal quality with the one or more objects that is less than a first threshold; and

21 . The mobile robotic device of claim 20 , wherein the at least one computer processor is further programmed to:

receive mask information associating different vacuum assemblies of the robotic gripper to different objects to be grasped; and activate the plurality of vacuum assemblies based, at least in part, on the mask information.

22 . The mobile robotic device of claim 21 , wherein

release the first object by selectively deactivating the vacuum assemblies in the first set of vacuum assemblies;

wait for an amount of time; and

release the second object by selectively deactivating the vacuum assemblies in the second set of vacuum assemblies.

23 . The mobile robotic device of claim 20 , wherein the at least one computer processor is further programmed to:

receive force information associated with the robotic gripper; and release the one or more objects in response to determining that the force information is greater than a second threshold.

24 . The mobile robotic device of claim 20 , wherein the at least one computer processor is further programmed to:

25 . The mobile robotic device of claim 24 , wherein determining that a first set of vacuum assemblies are non-functional comprises:

US19/333,516

2022-12-22

2025-09-19

Methods and apparatus for controlling a gripper of a robotic device

Pending

US20260109038A1

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US19/333,516

US20260109038A1

( en )

2022-12-22

2025-09-19

Methods and apparatus for controlling a gripper of a robotic device

Applications Claiming Priority (3)

Application Number

Priority Date

Filing Date

Title

US202263434507P

2022-12-22

US18/545,148

US12447620B2

( en )

2022-12-22

2023-12-19

Methods and apparatus for controlling a gripper of a robotic device

US19/333,516

US20260109038A1

( en )

2022-12-22

2025-09-19

Methods and apparatus for controlling a gripper of a robotic device

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US18/545,148

Continuation

US12447620B2

( en )

2022-12-22

2023-12-19

Methods and apparatus for controlling a gripper of a robotic device

Publications (1)

Publication Number

Publication Date

US20260109038A1

true

US20260109038A1 ( en )

2026-04-23

Family

ID=89768547

Family Applications (2)

Application Number

Title

Priority Date

Filing Date

US18/545,148

Active

2044-04-21

US12447620B2

( en )

2022-12-22

2023-12-19

Methods and apparatus for controlling a gripper of a robotic device

US19/333,516

Pending

US20260109038A1

( en )

2022-12-22

2025-09-19

Methods and apparatus for controlling a gripper of a robotic device

Family Applications Before (1)

Application Number

Title

Priority Date

Filing Date

US18/545,148

Active

2044-04-21

US12447620B2

( en )

2022-12-22

2023-12-19

Methods and apparatus for controlling a gripper of a robotic device

Country Status (3)

Country

Link

( 2 )

US12447620B2

( en )

( 1 )

EP4638066A1

( en )

( 1 )

WO2024137781A1

( en )

Families Citing this family (10)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12306033B2

( en )

2022-04-15

2025-05-20

Invia Robotics, Inc.

Systems and methods for deriving weight and quantity based on activations of robot actuators used for object retrieval

US12447620B2

( en )

2022-12-22

2025-10-21

Boston Dynamics, Inc.

Methods and apparatus for controlling a gripper of a robotic device

US12403611B2

( en )

2023-04-17

2025-09-02

Figure Ai Inc.

Head and neck assembly for a humanoid robot

US12365094B2

( en )

2023-04-17

2025-07-22

Figure Ai Inc.

Head and neck assembly for a humanoid robot

US12539618B1

( en )

2023-04-17

2026-02-03

Figure Ai Inc.

Head and neck assembly of a humanoid robot

US12420434B1

( en )

2024-01-04

2025-09-23

Figure Ai Inc.

Kinematics of a mechanical end effector

US12605824B2

( en )

2024-02-26

2026-04-21

Figure Ai Inc.

Humanoid robot

US12578733B2

( en )

2024-09-04

2026-03-17

Figure Ai Inc.

Bipedal action model for humanoid robot

US12611767B2

( en )

2024-09-06

2026-04-28

Figure Ai Inc.

System and method for efficient control of a humanoid robot

US12611766B2

( en )

2024-09-13

2026-04-28

Figure Ai Inc.

Humanoid robot with advanced kinematics

Family Cites Families (8)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20060182607A1

( en )

2005-01-18

2006-08-17

Clark Jason A

Method and apparatus for depalletizing bagged products

DE202011108836U1

( en )

2011-12-08

2011-12-29

Grenzebach Maschinenbau Gmbh

Apparatus for the industrial production of photovoltaic concentrator modules

US9205558B1

( en )

2014-07-16

2015-12-08

Google Inc.

Multiple suction cup control

US10766149B2

( en )

2018-03-23

2020-09-08

Amazon Technologies, Inc.

Optimization-based spring lattice deformation model for soft materials

US10919151B1

( en )

2018-03-23

2021-02-16

Amazon Technologies, Inc.

Robotic device control optimization using spring lattice deformation model

JP7660368B2

( en )

2019-12-17

2025-04-11

ãã¹ãã³ ãã¤ããã¯ã¹ï¼ã¤ã³ã³ã¼ãã¬ã¤ããã

Intelligent gripper with individual cup control

AU2022245984A1

( en )

2021-03-26

2023-10-12

Boston Dynamics, Inc.

An integrated mobile manipulator robot

US12447620B2

( en )

2022-12-22

2025-10-21

Boston Dynamics, Inc.

Methods and apparatus for controlling a gripper of a robotic device

2023

2023-12-19

US18/545,148

patent/US12447620B2/en

active

Active

2023-12-20

PCT/US2023/085076

patent/WO2024137781A1/en

not_active

Ceased

2023-12-20

EP23848082.6A

patent/EP4638066A1/en

active

Pending

2025

2025-09-19

US19/333,516

patent/US20260109038A1/en

active

Pending

Also Published As

Publication number

Publication date

US20240217104A1

( en )

2024-07-04

EP4638066A1

( en )

2025-10-29

US12447620B2

( en )

2025-10-21

WO2024137781A1

( en )

2024-06-27

Methods and apparatus for controlling a gripper of a robotic device — Boston Dynamics, Inc. (US20260109038A1)

Related documents