Conceptio › Archive › Google Patents

Google Patentsopen access

System and method for instructing a robot — Cognibotics Ab (US11305431B2)

Cognibotics Ab · Google Patents

Google Patents · Patents · License: Open Access

patent, google patents, intellectual property, US11305431B2, Cognibotics Ab, M. Mahdi Ghazaei Ardakani, en, 2022

ABSTRACT

Abstract

The disclosure relates to a system (1) and method for instructing a robot. The system (1) comprising an immersive haptic interface, such that operator interaction with a master robot arm (2) is reflected by a slave robot arm (3) arranged for interaction with a workpiece (4). The interaction of the slave robot arm (3) is reflected back to the master robot arm (2) as haptic feedback to the operator. The dynamic system is continually simulated forward and new commands are calculated for the master robot arm and the slave robot arm.

Description

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

CROSS-REFERENCE TO RELATED APPLICATION

This application is the National Phase, under 35 U.S.C. Â§ 371(c), of International Application No. PCT/EP2017/078558, filed Nov. 8, 2017, which claims priority from EP 16198208.7, filed Nov. 10, 2016. The disclosures of all of the referenced applications are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to technology for robots, and in particular to a method and a system for instructing a robot in an intuitive way by force interaction. An operator may use intuitive commands for defining a task of the robot, such as a robot moving an end-effector in contact with a workpiece, in an immersive manner based on haptics and control of multiple arms.

BACKGROUND

It is a most common case that robot motions are to be carried out in physical contact with the environment, with a well-defined interaction force. In specific configurations it is well known how to specify and control such motions. However, needs for flexibility and productivity also call for motion definitions to be portable with respect to physical arrangements and choice of equipment, and there are needs for intuitive commands thus permitting non-expert operators, without qualified engineering of each specific application.

A robot is here defined as a programmable manipulator with one or several arms, each arm optionally being equipped with one or several end-effectors, and the joints of the one or several arms being controlled by a controller. Robots have found wide application in many areas of industry. Some industrial areas involve labour dangerous to human health or labour performed under conditions not possible for humans to withstand. Other areas of industry involve repetitive tasks which can be performed much more efficiently and precisely by a robot.

Robots can be programmed on a system level by the robot provider, using some computer programming language. Normally, one part of the system or controller is an interpreter of user programs, which are written in some robot programming language that is defined by that interpreter. These end-user programs are referred to as robot programs. A robot program is formed by instructions that can be interpreted. Equivalently for the following, robot programs can be compiled, and different combinations of system level and user level programs can be utilized.

A robot program can be written by hand in plain text, they can be defined in graphical 3D environments, they may be generated from CAD/CAM systems, they can be instructed using manual guidance of a manipulator, or by some combination of these and any other possible technique. Combinations are appropriate since certain types of instructions may be easier to define with a specific technique. Instructions that upon execution result is physical effects such as end-effector motions are of particular interest for efficient usage of robots.

For force interaction, such as in assembly applications, the desired motion often depends on unmodeled physical behaviour that is best observed/experienced by the production engineer who is operating the robot. How to transfer the operator experience-based intent to the corresponding set of robot instructions has for some time been subject to extensive research. Ideally, robot programs define the intended task to be performed by the robot. Practically today, a robot performing programmed instructions result in calls to functions of the controller that controls the joints of the robot such that the end-effector performs an actual motion as part of the task, but the program needs ad-hoc modifications since the available instruction set of the controller mainly provides means of secondary effects.

Such a secondary effect can be an implicitly programmed position deviation resulting in a desired end-effector force, rather than directly performing a force-controlled motion with instruction parameters expressing the intent of the operator. One reason for this unfortunate situation is that robot programming and control functions are traditionally based on position-controlled motions, which operate in free space or with the end-effector being in contact with the workpiece. That is, even in the case of a desired contact force between the robot and its environment, the motion definition is based on programmed positions, with the resulting force being a secondary effect from positions that bring objects in contact with each other. The motions being position based, either on user or on system level of the control, has the advantage that control functions are easier to provide and support as packaged control features that can be verified without specifics of the environment such as the stiffness of the force contact. The disadvantage can be that the forces as secondary effects can be more difficult to specify and to control with reasonable bandwidth.

To accomplish appropriate control of the contact forces, a variety of control schemes have been developed. Force sensors and force estimation techniques are becoming commonplace technologies to facilitate such control. Special attention has been paid to the transition between contact and noncontact situations. Also the dependency on kinematic structure of the robot manipulator has been an obstacle for industrial applicability, but during the last decade several products capable of up to 6D force/torque-controlled end-effector motions have appeared on the market.

One common approach to robot task definition is to program a virtual robot of the same type as the physical one. This is typically done in some programming tool with 3D graphics. Interaction-forces are, however, not handled in such programming environments, so programming of robots involving physical interactions with the environment is time consuming and requires hours of engineering, comprising repeated tests on the physical system.

Another approach is to equip a single-arm robot with another force-sensing device that is used to command the motions, both for position control and for force control. However, there are many situations where usual single-arm lead-through programming is impractical or undesired:

Due to safety restrictions (grinding tools, press machines, etc.) it may not be possible to stay close to the robot. The workspace may be cluttered such that a human cannot enter. The coupling between the operator and the manipulator may change the nature of the task.

In âA constraint-based programming approach to physical human-robot interactionâ by Gianni Borghesan et al, Robotics and Automation (ICRA), 2012 IEEE International conference 14-18 May 2012, a constraint based approach to the design of teleoperation system has been considered. That article represents state-of-the-art, the approach being purely kinematic, while force interplay in most applications are more related to dynamics. Thus, there is still need for improvements to also reflect dynamics of the system.

SUMMARY

It is an object of the present disclosure to alleviate at least some of the limitations of existing technologies. It is a further object of the disclosure to enable a practical and efficient teaching of robot motions that are to exhibit a well-defined force interaction with workpieces. For such efficient teaching, the robot instructor/programmer should not be distracted by system restrictions that are not part of the task at hand or that are hard to understand from appearance of the equipment/robot that is used. For such a system to be maintained for widespread industrial usage, the implementation should be model-based and allow incorporation of new mechanics without re-programming.

These objects and others are at least partly achieved by the system and method according to the independent claims, and by the embodiments according to the dependent claims.

According to one aspect, the disclosure proposes a system for instructing a robot. The system comprises a master robot arm arranged for being influenced by operator interaction and a slave robot arm arranged for interaction with a workpiece. The system also comprises a control unit configured to determine master external force data indicating a force interplay between the operator and the master robot arm, and to determine slave external force data indicating a force interplay between the slave robot arm and the workpiece. The control unit further comprises a haptic interface module comprising a constraint submodule defining a coupling, namely a kinematic coupling, between a designated master coupling frame of the master robot arm and a designated slave coupling frame of the slave robot arm such that a velocity of the master coupling frame and a velocity of the slave coupling frame are interrelated. The haptic interface module also comprises a calculation submodule, e.g. a solver submodule, configured to calculate a joint movement command for the master robot arm and a joint movement command for the slave robot arm, based on the master external force data, the slave external force data, a model, i.e. a dynamic model, of the master robot arm, a model, i.e. a dynamic model, of the slave robot arm, and a relation between the dynamic models including forces/torques for accomplishing the kinematic coupling, while constraints imposed by the dynamic models are respected. The system is further configured to control the master robot arm and the slave robot arm according to the joint movement commands. The operator will then receive haptic feedback from the force interplays reflecting dynamics of the system. A command may relate to an instruction, but it may also be a time-series of set-points provided as reference to the servo control according to a trajectory generator or any other computations. The system is further configured to record resulting movement of a frame related to the slave robot arm, and to record resulting/applied force to the workpiece in the same frame. This frame may be a Tool Centre Point (TCP) frame, an end flange frame, or a frame of an elbow of the slave robot arm.

The system supports an operator in intuitive commanding of the slave robot arm via the master robot arm; the operator can then sense and see how the master robot arm reacts using the combination of haptic and visual feedback from the application at hand. As part of the robot tasks, the motion instructions can be defined in an immersive manner, and thereby facilitating the operator to stay focused on the actual task. Immersive here means that the desired end-effector operation (for instance with a for the task suitable force acting on the workpiece) results from a natural/physical operator interaction, with the operator totally focused on the intended motion while spending a minimum cognitive effort on any surrounding artificial/dynamic constraints such as dynamic effects of proximity to singular configurations that are not physically obvious. One goal is thus to maintain the kinematic coupling despite such dynamic effects.

The system gives flexibility in instructing the robot, as the physical arrangement of the robotic (and possibly human) workplace can vary between instructing the robot and the robot performing the task. By physically and virtually maintaining the relationship between the end-effector and the workpiece, with forces determined from the operator-robot interaction, in combination with a mapping that in a flexible way copes with any artificial limitation, changes such as type of robot and arm placement are permitted without the complex tuning as in applications today.

The system makes it possible for the operator to teach the robot in an intuitive way, which is industrially valuable since operators knowing the manufacturing processes (but with very little experience of programming and today's complex tuning of indirectly acting robot instructions) can work closer to the physical process at hand. Still, the operator must cope with practical limitations such as robot workspace boundaries and limited gripper dexterity, but since such limits are apparent to the operator and since the commanding is dealt with by a controller that can be configured to monitor and map the involved physical quantities to whatever internal representation that is needed, the complexity of the control is encapsulated such that instructing the robot is perceived as being intuitive.

As mentioned, force interacting tasks are particularly difficult to program. A major problem with prior technologies is that the operator is not able to feel the same motion and forces as those acting between an interaction point of the robot and a workpiece. The disclosed system solves this problem. Previously the kinematic and dynamic effects of the robot motions disturbed the demonstration, but with the present innovative system such effects are taken care of. Thereby, the system is useful for speeding up the process of programming tasks involving force interaction. Since both force and position are taken into account, reasonable variations in position of a workpieces can be coped with easily.

The well-defined force interaction in combination with the fact that system restrictions are often related to dynamic limitation such as limited torques and inertial forces, the model-based approach means that the dynamic models impose constraints on the permitted motions, which in an example d industrial implementation are to be easily incorporated by some type of solver instead of advanced/inflexible re-programming.

To facilitate efficient definition of robot tasks, the system provides a solution in which a master robot arm conveys instructions to a slave robot in an efficient and intuitive manner that fits with existing industrial robot control systems. The master robot arm is e.g. a physical robot arm, which can be a standard haptic interface or a manipulator with force sensing and force control capabilities. The slave robot arm can be the real robot to be used in the application or its virtual representation with an adequate model of the environment. Multiple master arms and multiple slave arms are obvious extensions. For example, one dual-arm robot may act as a master device for an operator using both hands for separate motions, influencing another dual arm robot that performs instructions as a slave dual-handed device. Modern industrial controllers permit such connections over a real-time communication network. For brevity, a single master and a single slave is described in the sequel.

Although the master and slave roles of arms may change dynamically during operation, and the desired coupling between master and slave is beyond functionality of existing robot products, robots available on the market can still be used as modules by means of the included mapping between the intended operation in virtual terms and the physical command/control signals to existing interfaces. Thus, desired forces for haptic feedback can be defined in accordance with industrial practices and performance needs.

According to some embodiments, the constraint submodule comprises a plurality of motion constraints, and wherein the calculation submodule is configured to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm based on the plurality of motion constraints such that a corresponding constrained motion is accomplished, while constraints imposed by the dynamic models are respected. In other words, the calculation submodule is configured to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm such that additional motion constraints are respected. An additional motion constraint, thus one of the plurality of motion constraints, may include a joint limit, a distance between a pair of frames, coupling of a pair of joints or a motion constraint for a task including motion along a surface with a certain force. In one implementation the configuration of the calculation submodule is done automatically based on declarative symbolic dynamic models and declarative symbolic constraints that are included in the constraint module, thereby avoiding the need for re-programming the system for new applications/equipment. Thus, the proposed incorporation of a plurality of constraints is a feature of the present invention according to one embodiment. It should here be mentioned that mentioned kinematic coupling defined by the constraint submodule is also a motion constraint. The motion constraints are defined by the user according to the application setup and represented in the constraint submodule.

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

CROSS-REFERENCE TO RELATED APPLICATION

TECHNICAL FIELD

BACKGROUND

SUMMARY

These objects and others are at least partly achieved by the system and method according to the independent claims, and by the embodiments according to the dependent claims.

According to some embodiments, the calculation submodule is configured to determine a solution to a system of differential-algebraic equations defining a relation between dynamics of the system and the forces and/or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected, and to use the solution to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm. The system of differential-algebraic equations will be explained in greater detail in the following.

According to some embodiments, the calculation submodule is configured to calculate the solution to comprise the forces and/or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected. These forces and/or torques may be referred to as virtual/artificial/dynamic forces and/or torques. This is because they are not real forces and/or torques acting against the system, but forces and/or torques calculated by the system to reflect the kinematic and dynamic constraints of the system, and which are then applied by the system such that the operator can perceive the constraints of the system etc. via haptic feedback. In other words, the calculation submodule, e.g. the solver submodule, is configured to determine at least one virtual force and/or at least one virtual torque needed to maintain motion constraints on the master robot arm and/or on the slave robot arm, and wherein the calculation submodule further is configured to determine the joint movement command for the master robot arm, and the joint movement command for the slave robot arm also based on the at least one virtual force. Thus, the conceptual virtual force is here being explicitly represented.

According to some embodiments, the calculation submodule, such as the solver submodule, is configured to determine the at least one virtual force and/or at least one virtual torque needed to maintain motion constraints in any kinematic configuration. Thus, the calculation submodule is here configured to calculate the virtual force and/or virtual torque in order to enforce dynamic constraints, for any kinematic configuration of the slave robot arm and/or master robot arm.

According to some embodiment, the master robot arm and the slave robot arm have dissimilar kinematics and/or have dissimilar degrees of freedom, DOF. Thus, there is no need to have the same kinematics or the same number of DOFs of the master robot arm and the slave robot arm in the proposed system. According to some embodiment, the control unit is configured to calculate a joint movement command for another master robot arm and/or a joint movement command for another slave robot arm without re-programing the control unit. With re-programming here means change of system software or robot programs to cope with unforeseen combinations of master and slave arm properties. Such a change, as necessary with any prior-art solution, would be problematic since it would require competencies that are normally not available at user sites. Instead, according to the present invention, application constraints are expressed in a declarative manner in a constraint module, which via a solver results in the necessary computations.

According to some embodiments, the system is configured to record the determined joint movement command and the slave external force data. According to some embodiments, the system is configured to determine a robot program or robot instructions based on the recorded resulting movement expressed in the frame related to the slave robot arm, and based on the recorded resulting/applied force to the workpiece in the same frame. According to some embodiments, the robot program comprises at least one set of instructions.

According to some embodiments, the kinematic coupling between the master coupling frame and the slave coupling frame is defined in task space. Task space (also known as operational space) refers to the space where motion and forces of a Cartesian frame relevant to the robot task is specified. This frame can for example be attached to the end effector, an object held by the end effector, or a fixed or moving object that the robot works on.

According to some embodiments, the dynamic models are non-linear dynamic models, for instance in terms of mechanical multi-body effects and in terms of non-linear feedback for proper dynamic behaviour. It should here be mentioned that a linear model, as found e.g. in prior art, is a special case of non-linear models. The present invention thus deals with a more generic case of real-world dynamics as resulting from the physical properties of a system, which differs from the dynamics obtained using standard closed loop control that by actuation is permitted to more freely modify the dynamics.

According to some embodiments, the master external force data is determined based on joint motion data and/or joint position data and/or motor signal data of at least one joint of the master robot arm, and wherein the slave external force data is determined based on joint motion data and/or joint position data and/or motor signal data of at least one joint of the slave robot arm. According to some embodiments, the external force data is obtained from one or several torque sensors and/or joint force data and/or motor signal data of at least one joint of the master robot arm or of the slave robot arm, respectively.

According to some embodiments, the control unit is configured to accomplish a bidirectional transfer of force and torque between the master robot arm and the slave robot arm such that haptic feedback is complied with according to any restricted and singular configuration of the slave robot arm and/or master robot arm. Bidirectional here means a practically simultaneous transfer of control signals such that the force interplay agrees with the models that may be non-causal (DAEs expressed in a declarative manner).

According to some embodiments, the master robot arm is mechanically separated from the slave robot arm. Thus, the slave robot arm and the master robot arm may be separate arms, for example belong to different robots or belong to the same robot that has two physical arms.

According to some embodiments, the slave robot arm is a virtual slave robot arm. The slave robot arm is then virtual as part of a virtual environment. According to some other embodiments, the slave robot arm and the master robot arm are the same arm but arranged with additional force sensing means in order to determine both master and slave forces, i.e. master external force data and slave external force data. The operated slave robot arm is then physical, but the same arm as the master robot arm.

According to some embodiments, the haptic interface module is configured to map at least one restriction on the slave robot arm into at least one intuitive force reaction that the control unit is configured to reproduce as haptic feedback to the master robot arm.

According to some embodiments, the intuitive force reactions correspond to a direct kinematic coupling over a common workpiece that is common to the master robot arm and the slave robot arm.

According to a second aspect, the disclosure propose a method for instructing a robot. The method comprising: determining master external force data indicating a force interplay between an operator and a master robot arm and determining slave external force data indicating a force interplay between a slave robot arm and a workpiece. The method further comprises calculating a joint movement command for the master robot arm and a joint movement command for the slave robot arm, based on the master external force data, the slave external force data, a model, i.e. a dynamic model, of the master robot arm, a model, i.e. a dynamic model, of the slave robot arm, and a defined coupling, i.e. a kinematic coupling, between a designated master coupling frame of the master robot arm and a designated slave coupling frame of the slave robot arm, and by enforcing a relationship between the dynamic models including forces/torques for accomplishing the kinematic coupling, while respecting constraints imposed by the dynamic models resulting in that a velocity of the master coupling frame and a velocity of the slave coupling frame are interrelated. The method further comprises controlling the master robot arm and the slave robot arm according to the joint movement commands. The operator then receives haptic feedback from the force interplays reflecting dynamics of the master robot arm and the slave robot arm. The dynamics of the master robot arm and the slave robot arm results from their physical properties and their individual (or common) control systems/control units. The haptic feedback from the force interplays may reflect further dynamics of the system, as explained in connection with the system and as will be further explained in the following.

The same positive effects as with the system may be accomplished with the method, and reference is made to the corresponding sections above.

According to some embodiment, the calculating comprises calculating the joint movement command for the master robot arm and the joint movement command for the slave robot arm based on the plurality of motion constraints such that a corresponding constrained motion is accomplished, while constraints imposed by the dynamic models are respected. With other words, according to some embodiments, the calculating comprises calculating the joint movement command for the master robot arm and the joint movement command for the slave robot arm such that additional motion constraints are respected. An additional motion constraint, thus one of the plurality of motion constraints, includes e.g. a joint limit, a distance between a pair of frames, a coupling of a pair of joints or a motion constraint for a task e.g. moving along a surface with a certain force. It should here be mentioned that mentioned kinematic coupling is also a motion constraint. The motion constraints may be predefined by the user.

According to some embodiments, the calculating comprises determining a solution to a system of differential-algebraic equations defining a relation between dynamics of the system and the forces and/or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected, and using the solution to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm. The system of differential-algebraic equations will be explained in greater detail in the following.

According to some embodiments, the calculating comprises calculating the solution comprising the forces and/or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected. The forces and/or torques may be referred to as virtual forces and/or torques. This because they are not real forces and/or torques acting against the master robot arm and/or slave robot arm, but forces and/or torques calculated by the method to reflect the constraints of the dynamic models, the master robot arm/slave robot arm etc., and which are then applied by the method such that the operator can perceive the constraints via haptic feedback. In other words, in some embodiments, the calculating comprises determining at least one virtual force needed to maintain motion constraints on the master robot arm and/or on the slave robot arm, and determining the joint movement command for the master robot arm, and the joint movement command for the slave robot arm also based on the at least one virtual force. Thus, the conceptual virtual force can be explicitly calculated.

According to some embodiments, the calculating comprises determining the at least on virtual force and/or at least on virtual torque needed to maintain motion constraints in any kinematic configuration of the master robot arm and/or the slave robot arm. Thus, any kinematic configuration of the master robot arm and/or the slave robot arm is respected and thus allowed.

According to some embodiments, the method comprises recording the determined joint movement command and the slave external force data. According to some embodiments, the method comprises determining a robot program or robot instruction based on the recorded joint movement command and the slave external force data. According to some embodiments, the method comprises determining a robot program or robot instruction based on the recorded resulting movement expressed in the frame related to the slave robot arm, and based on the recorded resulting/applied force to the workpiece in the same frame. Thus, the joint movement command or commands and the external forces of the slave arm, the resulting movement of the frame related to the slave robot arm, the recorded resulting/applied force to the workpiece can be monitored, determined and/or recorded to form instructions, e.g. motion instructions. According to some embodiments, the robot program comprises at least one set of instructions. Thus, it is possible to form a robot program from at least one set of instructions.

According to some embodiments, the method comprising determining the master external force data based on joint motion data and/or joint position data and/or motor signal data of at least one joint of the master robot arm, and determining the slave external force data based on joint motion data and/or joint position data and/or motor signal data of at least one joint of the slave robot arm. According to some embodiments, the method comprising obtaining the external force data from one or several torque sensors and/or joint force data and/or motor signal data of at least one joint of the master robot arm or of the slave robot arm. Thus, explicit force sensing, or observations from motor torques, can be used to determine external forces.

According to some embodiments, the calculating comprising accomplishing a bidirectional transfer of force and torque between the master robot arm and the slave robot arm such that haptic feedback is complied with, or adjusted, according to any restricted and/or singular configuration of the slave robot arm and/or master robot arm.

According to some embodiments, the master robot arm is mechanically separated from the slave robot arm.

According to some embodiments, the slave robot arm is a virtual slave robot arm.

According to some embodiments, the calculating comprises mapping at least one restriction to the slave robot arm into at least one intuitive force reaction and reproducing the intuitive force reaction as haptic feedback to the master robot arm. According to some embodiments, the intuitive force reaction corresponds to a direct kinematic coupling over a common workpiece that is common to the master robot arm and the slave robot arm.

According to a third aspect, the disclosure relates to a computer program comprising instructions which, when the program is executed by a control unit or a computer connected to the computer, cause the control unit to carry out the method according to any of the method steps as disclosed herein.

According to a fourth aspect, the disclosure relates to a computer-readable medium comprising instructions which, when executed by a control unit or a computer connected to the control unit, cause the control unit to carry out the method according to any of the embodiments as described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a two-armed collaboration robot according to some embodiments.

FIG. 2 illustrates a control architecture according to some embodiments.

FIG. 3 illustrates some of the control architecture of FIG. 2 in more detail.

FIG. 4 illustrates a flow chart of a method according to some embodiments.

DETAILED DESCRIPTION

Definitions Master robot arm: A robot arm, or any kinematic device or linkage, that continually or temporarily is arranged for being influenced by physical operator interaction via at least one interaction point such as an end effector or tool centre point, TCP. The influence via any such interaction point can be any combination of forces and torques in one to six dimensions. Simultaneous use of multiple interaction points permits more than six dimensions, e.g., for influencing both an end-effector and a kinematic configuration of a redundant (slave) robot arm.

Slave robot arm: A robot arm, or any kinematic device or linkage, that continually or temporarily is arranged for (virtual or physical) motions that are intended for an effect on the (virtual or physical) environment via at least one interaction point such as an end-effector or TCP.

Constraint module: A control system module that defines and stores motion constrains, such as the coupling between a pair of coupling frames. A motion constraint may be defined e.g. in terms of relative position and/or orientation, distance coupling of a pair of frames, coupling of a pair of joints as well as other motion constraints such as joint limits and task constraints. The coupling between a pair of coupling frames may include a mapping between a master robot arm and a slave robot arm. The constraint module may include a plurality of couplings between pairs of coupling frames. A preferred constraint module implementation is generic with respect to the robot configuration (kinematics, end-effectors, location, etc.) and to the task at hand.

Constraint: The system is under influence of different types of constraints. Constraints that results from the physical properties of the system such as the construction of the mechanical system, or the environment, may be referred to as actual, intrinsic or natural constraints. Constraints that result from the control system/control unit may be referred to as calculated constraints (or enforced constraints). The calculated constraints give rise to calculated forces/torques. The constraints of the system may be divided in motion constraints and dynamic constraints. Motion constrains include the kinematic coupling and geometrical constraints. The dynamic constrains arise from the dynamics of the system. Virtual force (VF) or Virtual Torque (VT): Calculated force respective calculated torque to maintain a coupling between the master robot arm and the slave robot arm and respect other motion constraints (joint limits, etc.). Motion constraints, e.g. kinematic constraints, are imposed by virtual constraints, which result in virtual forces and/or virtual torques. Thus, as a response to a constraint, either a constraint that is defined in beforehand or a constraint that is inherent in the system, one or several forces/torques is calculated for each arm and applied in the system. The system may be under influence of several different constraints at the same time, and the calculated force/torque is then a response to all the several different constraints.

Solver: A Differential-Algebraic Equation (DAE) solver, which uses the constraints in the constraint module, a model of the master robot arm and a model of the slave robot arm, to compute the virtual force/virtual torque over time. A task can be accomplished by means of a declarative symbolic description of a state-transition system in combination with a DAE system, where the latter requires the solver for determining the actual control outputs such as motion commands or servo setpoints. By also expressing the constraints in a declarative way, typically according to some symbolic math software package, new robot tasks and new equipment/robots can be incorporated without re-programming.

Immersive: When an operator is enabled, e.g. by a system, to stay totally focused on a task including force interplay with a workpiece, such that the operator can intuitively perform the task, where the system supports the operator to stay totally focused on the task by transferring movements and forces from the operator to the robot and vice versa while respecting constraints. The operator then does not have to spend cognitive effort on respecting any constraints induced by the system itself, as these are transferred to the operator in an intuitive way. The system supports the operator by reflecting the intrinsic or desired constraints to the operator as haptic feedback while transferring movements and forces from the operator to the robot and vice versa. The operator thus experiences the real world through the robot.

Immersive interface: An operator interface, here for kinaesthetic teaching that engages the operator in an immersive manner.

Haptic feedback: Propagation of forces by physical contact with the operator such that a sense of touch is perceived.

Immersive feedback: Haptic feedback to the operator via a control system or control unit such that an immersive interface is accomplished together with direct visual feedback. Force interplay: Bidirectional force interaction between master robot arm and operator, and between slave robot arm and workpiece.

Interaction point on master robot arm: A point, or more generally a handle, on the master arm designated to provide force interaction with the operator, normally in six dimensions for both force and torque. However, jointed handles or special devices may limit the influence to fewer dimensions. Multiple points (for example one per finger) may be present, but in the following one equivalent point is assumed (for example representing an equivalent handle with the same effect as multiple finger forces).

Interaction point on slave robot arm: A point, or multiple of points, on the slave robot arm, for example an effector attached to the end point of the slave robot arm as an end effector or an effector mounted to an elbow of the slave robot arm.

Coupling frame: A coordinate frame firmly attached to a point on the master robot arm, and a corresponding frame on the slave robot arm, where the master and slave coordinate frames are coupled by means of the control as per the present invention. There is at least one coupling frame for each of the master and slave arms. A pair of coupling frames comprises a coupling frame of a master robot arm and a corresponding coupling frame of a slave robot arm. Thus, a pair of coupling frames includes a master coupling frame and a slave coupling frame.

The disclosure concerns an immersive haptic interface for task demonstration, where the operator can sense and act through the robot. This is achieved by coupling two robotic arms or systems on a dynamical level. Limitations caused by singular configurations or the reach of either of the arms or robots may be naturally reflected to the other as haptic feedback.

First an introduction to the topic will be described. Thereafter follows a description of an exemplary robot, and other examples, where the immersive haptic interface can be used, and description of the immersive haptic interface itself. Then, methods for instructing a robot using the immersive haptic interface will be described.

A robot can be taught to handle a multitude of tasks. Many tasks are however still carried out manually, since it is overly difficult to program the robot to achieve a similar performance. Typically, these tasks involve interaction with an object or the environment, where the success of the task largely relies on the skills of the human. To program a robot, these skills need to be transferred to the robot. The most natural way for a human to do this is via demonstration. For teaching by demonstration, if the robot does not have a similar kinematic structure to the operator doing the demonstration, the mapping between robot motion and human motion will not be trivial. By using the robot as part of the demonstration of a task, this problem could be entirely avoided. Therefore, this approach has been widely used in human skill acquisitions. Despite this, the interface between humans and robots can be inconvenient and difficult for accurately transferring motions because of mechanical properties of robots such as inertia and friction. Although compliant motion control could be employed to reduce inertial/friction forces, direct teaching of industrial robots is still limited. Moreover, the mechanical coupling between the operator, the robot, and the workpiece makes it impossible to record faithfully the required force values for a task. Hence, an interface for demonstrating a task can contribute by allowing the operator to perceive the differences and limitations of the robotic system.

A physical robot, if equipped with force control capabilities, can be used as a haptic device. However, for proper definition of force-interacting motions to be performed with the end-effector that is available to the robot, the force control should act via the robot for motion commands to later be useful in the actual physical environment. In the case of accurate modelling of the equipment, including the robot and its end-effector, and environment including workpiece and peripherals, the robot performing the task does it virtually and only one physical robot acting as a haptic device is needed.

Previously existing implementations of haptic interaction are not industrially applicable. A main reason is that in any realistic setup there are motion constraints that also need to be managed. Neither full awareness nor classification of these constraints and their impact on the application has previously existed, nor proper handling of them such that an immersive operator experience can be created. The transparency is further limited by the structures of the master or the slave devices. Allowing the arms to have different configurations, increases substantially the flexibility in demonstration. For example, the demonstration of a task can be performed in a part of the workspace that is more convenient for the operator. However, when two robotic systems are employed in a master-slave configuration, their workspace is limited to the points reachable by both systems simultaneously. This defines a common workspace for the robots. At the boundary of that common workspace, one or both of the systems are typically in a singular configuration with reduced manipulability.

As has been previously mentioned, previously existing technologies are not considered to define interaction forces efficiently in a robot programming context. A way forward is enabled by recent industrial developments of dual-arm robots, as one arm is readily available to act as a haptic interface. Motivations for such robots comprise safe levels of maximum power per arm in combination with short cycle times, improved flexibility by avoiding fixtures, compatibility with human workplaces, and manipulability in general. Furthermore, most modern robot controllers have so called multi-move functionality, which means that two or more robots can be connected over a real-time communication link and programmed together as a dual arm or multi-arm robot with a plurality of arms.

In the following an inventive system 1 and a method for instructing a robot will be described, with reference to the figures. The robot is generally a common robot that is not specifically designed as a haptic device. The disclosure thus proposes a generic approach to provide an immersive user experience. Utilizing both visual and haptic feedback from a robot or a model of it, an operator can ideally feel and perceive a task from the robot's perspective, hence enabling accurate demonstration including force specification.

Initially, each arm can move freely using lead-through setup. Upon activation of the system, the arms move in synchrony respecting the initial fixed offset between the end-effectors. This constitutes an immersive haptic interface, where the operator can impact the environment while receiving the haptic feedback.

As previously explained, a robot is here defined as a programmable manipulator with one or several arms, each arm optionally being equipped with one or several end-effectors and the joints of the one or several arms being controlled by a controller. FIG. 1 illustrates a robot 10 that may be included in the system 1 . The depicted robot 10 is a two-armed, or dual-arm, collaborative robot that is arranged to work on a workpiece 4 according to a program with computer instructions. The robot 10 comprises a first arm 2 and a second arm 3 . The first arm 2 comprises a plurality of links and joints, and the second arm 3 comprises a plurality of links and joints. A link and a joint are commonly referred to as an axis. Each axis is driven by an actuator, i.e. a servo-controlled motor, via a transmission. The first arm 2 is arranged with an end effector 11 in the shape of a teaching handle. The second arm 3 is arranged with an end effector 17 in the shape of a gripper. The gripper here holds the workpiece 4 . Each arm

2 , 3 has got seven axes. Some axes that may be important for the application at hand are explicitly referred to in the figure. The end effector 11 is attached to the seventh axis 2 a of the first arm 2 . The seventh axis 2 a comprises a first end flange defining a first end flange frame, the end effector 11 is attached to the first end flange. The end effector 11 defines a first tool centre point, TCP 1 , being the centre point of the end effector 11 . The end effector 11 further defines a first TCP frame, which originates in the TCP 1 . The forth axis 2 b , also referred to as the elbow, of the first arm 2 , defines an elbow frame of the first arm 2 . The elbow frame of the first arm 1 is thus defined in relation to the elbow of the first arm 2 . Turning to the second arm 3 , the end effector 17 is attached to the seventh axis 3 a of the second arm 3 . The seventh axis 3 a comprises a second end flange defining a second end flange frame, the end effector 4 is attached to the second end flange. The end effector 4 defines a second tool centre point, TCP 2 , being the centre point of the end effector 4 . The end effector 4 defines a second TCP frame, which originates in the TCP 2 . The forth axis 3 b , also referred to as the elbow, of the second arm 3 , defines an elbow frame of the second arm 3 . The elbow frame of the second arm 3 is thus defined in relation to the elbow of the second arm 3 .

The <figure-callout id="1

CLAIMS

Claims ( 25 )

The invention claimed is:

1. A system, comprising:

a robot having a master robot arm configured for being operably influenced by interaction with an operator; and a slave robot arm, configured for interaction with a workpiece; and

a control unit configured to determine master external-force data indicating a force interplay between the operator and the master robot arm, and to determine slave external-force data indicating a force interplay between the slave robot arm and the workpiece;

wherein the control unit further comprises a haptic interface module, comprising:

a constraint submodule defining a plurality of motion constraints including a kinematic coupling in a task space between a designated master coupling frame of the master robot arm and a designated slave coupling frame of the slave robot arm, such that a velocity of the master coupling frame and a velocity of the slave coupling frame are interrelated in the task space; and

a calculation submodule configured to calculate a joint movement command for the master robot arm and a joint movement command for the slave robot arm, based on the master external-force data, the slave external-force data, a non-linear dynamic model of the master robot arm, a non-linear dynamic model of the slave robot arm, a relationship between the non-linear dynamic model of the master robot arm and the non-linear dynamic model of the slave robot arm including virtual forces or torques at the designated master coupling frame and the designated slave coupling frame for the kinematic coupling in the task space, and based on at least one virtual force or virtual torque needed to enforce the plurality of motion constraint, while preserving constraints imposed by the non-linear dynamic models of the master robot arm and the slave robot arm;

wherein the system is configured to:

control the master robot arm and the slave robot arm according to the joint movement commands, wherein during instruction of the robot, haptic feedback from the force interplay between the operator and the master robot arm and the force interplay between the slave robot arm and workpiece reflecting dynamics of the system is provided to the operator.

2. The system according to claim 1 , wherein the system is configured to record resulting movement of a frame related to the slave robot arm, and to record a resulting or applied force onto the workpiece in the same frame.

3. The system according to claim 2 , wherein the system is configured to determine one of a robot program and a robot instruction based on the recorded resulting movement expressed in the frame related to the slave robot arm, and based on the recorded resulting or applied force to the workpiece in the same frame.

4. The system according to claim 1 , wherein the calculation submodule is configured to determine a solution to a system of differential-algebraic equations defining a relation between dynamics of the system and forces or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected, and to use the solution to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm.

5. The system according to claim 1 , wherein the master robot arm and the slave robot arm have dissimilar kinematics.

6. The system according to claim 1 , wherein the master robot arm and the slave robot arm have dissimilar degrees of freedom.

7. The system according to claim 1 , wherein the control unit is configured to calculate a joint movement command for at least one of another master robot arm and another slave robot arm without re-programming the control unit.

8. The system according to claim 1 , wherein the master external-force data is determined based on at least one of joint motion data, joint position data, and motor signal data of at least one joint of the master robot arm, and wherein the slave external-force data is determined based on at least one of joint motion data, joint position data, and motor signal data of at least one joint of the slave robot arm.

9. The system according to claim 1 , wherein the external-force data is obtained from at least one torque sensor.

10. The system according to claim 1 , wherein the master external-force data is obtained from one of joint force data and motor signal data of at least one joint of the master robot arm.

11. The system according to claim 1 , wherein the slave external-force data is obtained from one of joint force data and motor signal data of at least one joint of the slave robot arm.

12. The system according to claim 1 , wherein the master robot arm is mechanically separated from the slave robot arm.

13. The system according to claim 1 , wherein the slave robot arm is a virtual slave robot arm.

14. The system according to claim 1 , wherein the haptic interface module is configured to map at least one restriction on the slave robot arm into at least one intuitive force reaction that the control unit is configured to reproduce as haptic feedback to the master robot arm.

15. The system according to claim 14 , wherein the at least one intuitive force reaction corresponds to a direct kinematic coupling over a common workpiece that is common to the master robot arm and the slave robot arm.

16. A method for instructing a robot comprising:

determining master external-force data indicating a force interplay between an operator and a master robot arm of a robot;

determining slave external-force data indicating a force interplay between a slave robot arm of the robot and a workpiece;

calculating a joint movement command for the master robot arm and a joint movement command for the slave robot arm based on (i) the master external-force data, the slave external-force data, a first non-linear dynamic model of the master robot arm, a second non-linear dynamic model of the slave robot arm, and a plurality of motion constraints including a defined kinematic coupling in a task space between a designated master coupling frame of the master robot arm and a designated slave coupling frame of the slave robot arm, by enforcing a relationship between the first and second non-linear dynamic models, including virtual forces or torques at the designated frames for accomplishing the kinematic coupling in the task space, while respecting constraints imposed by the first and second non-linear dynamic models, resulting in that a velocity of the master coupling frame and a velocity of the slave coupling frame are interrelated in the task space, and (ii) at least one virtual force or virtual torque needed for enforcing the plurality of motion constraints; and

controlling the master robot arm and the slave robot arm according to the joint movement commands, wherein during instruction of the robot, the operator receives haptic feedback from the force interplay between the operator and the master robot arm and the force interplay between the slave robot arm and workpiece reflecting dynamics of at least one of the master robot arm and the slave robot arm.

17. The method according to claim 16 , wherein the method further comprises recording (a) resulting movement of a frame relative to the slave robot arm, and (b) resulting or applied force to the workpiece in the same frame.

18. The method according to claim 16 , wherein the calculating comprises determining a solution to a system of differential-algebraic equations defining a relation between dynamics of the system and the forces or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected, and using the solution to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm.

19. The method according to claim 16 , wherein the method further comprises accomplishing a bidirectional transfer of force and torque between the master robot arm and the slave robot arm such that haptic feedback is complied with or adjusted according to any restricted and/or singular configuration of at least one of the master robot arm and the slave robot arm.

20. The method according to claim 16 , wherein the calculating comprises:

mapping at least one restriction to the slave robot arm into at least one intuitive force reaction; and

reproducing the intuitive force reaction as haptic feedback to the master robot arm.

21. A non-transitory, computer-readable medium comprising instructions which, when executed by a control unit or a computer operatively connected to the control unit, cause the control unit to instruct a robot in accordance with a method comprising:

determining master external-force data indicating a force interplay between an operator and a master robot arm of a robot;

determining slave external-force data indicating a force interplay between a slave robot arm of the robot and a workpiece;

22. The non-transitory, computer readable medium according to claim 21 , wherein the method further comprises recording (a) resulting movement of a frame relative to the slave robot arm, and (b) resulting or applied force to the workpiece in the same frame.

23. The non-transitory, computer readable medium according to claim 21 , wherein the calculating comprises determining a solution to a system of differential-algebraic equations defining a relation between dynamics of the system and the forces or torques needed to accomplish the kinematic coupling while the constraints imposed by the dynamic models are respected, and using the solution to calculate the joint movement command for the master robot arm and the joint movement command for the slave robot arm.

24. The non-transitory, computer readable medium according to claim 21 , wherein the method further comprises accomplishing a bidirectional transfer of force and torque between the master robot arm and the slave robot arm such that haptic feedback is complied with or adjusted according to any restricted and/or singular configuration of at least one of the master robot arm and the slave robot arm.

25. The non-transitory, computer readable medium according to claim 21 , wherein the calculating comprises:

mapping at least one restriction to the slave robot arm into at least one intuitive force reaction; and

reproducing the intuitive force reaction as haptic feedback to the master robot arm.

US16/348,137

2016-11-10

2017-11-08

System and method for instructing a robot

Active

2038-09-08

US11305431B2

( en )

Applications Claiming Priority (4)

Application Number

Priority Date

Filing Date

Title

EP16198208

2016-11-10

EP16198208

2016-11-10

EP16198208.7

2016-11-10

PCT/EP2017/078558

WO2018087123A2

( en )

2016-11-10

2017-11-08

System and method for instructing a robot

Publications (2)

Publication Number

Publication Date

US20190358817A1

US20190358817A1 ( en )

2019-11-28

US11305431B2

true

US11305431B2 ( en )

2022-04-19

Family

ID=57288196

Family Applications (1)

Application Number

Title

Priority Date

Filing Date

US16/348,137

Active

2038-09-08

US11305431B2

( en )

2016-11-10

2017-11-08

System and method for instructing a robot

Country Status (6)

Country

Link

( 1 )

US11305431B2

( en )

( 1 )

EP3538328B1

( en )

( 1 )

KR102353178B1

( en )

( 1 )

CN110035871B

( en )

( 1 )

DK3538328T5

( en )

( 1 )

WO2018087123A2

( en )

Cited By (5)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12214975B2

( en )

2021-06-16

2025-02-04

Dexterity, Inc.

State estimation using geometric data and vision system for palletizing

US12258227B2

( en )

2021-06-16

2025-03-25

Dexterity, Inc.

Simulated box placement for algorithm evaluation and refinement

US12390928B2

( en )

2021-06-16

2025-08-19

Dexterity, Inc.

Workflow for using tree search-based approach for placing boxes on pallet with limited knowledge of future sequence

US12485544B2

( en )

2021-06-16

2025-12-02

Dexterity, Inc.

Workflow for using learning based approach for placing boxes on pallets

US12486125B2

( en )

2021-06-16

2025-12-02

Dexterity, Inc.

Collision avoidance based on vision data, geometric data and physics engine evaluation

Families Citing this family (41)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

EP3342562A4

( en )

2015-08-25

2019-06-26

Kawasaki Jukogyo Kabushiki Kaisha

REMOTE CONTROL ROBOT SYSTEM

JP6706489B2

( en )

2015-11-24

2020-06-10

å·å´éå·¥æ¥æ ªå¼ä¼ç¤¾

Robot direct teaching method

JP6960242B2

( en )

2017-05-19

2021-11-05

å·å´éå·¥æ¥æ ªå¼ä¼ç¤¾

Robot system and robot system control method

JP6959762B2

( en )

2017-05-19

2021-11-05

å·å´éå·¥æ¥æ ªå¼ä¼ç¤¾

Remote control robot system

US11103994B2

( en )

2018-07-02

2021-08-31

Teradyne, Inc.

System and method for natural tasking of one or more robots

CN109333497B

( en )

2018-11-30

2021-11-19

è¥¿åå¤§å¦

Control method of teleoperation system with virtual constraint force

US12011834B2

( en )

2018-12-21

2024-06-18

Kawasaki Jukogyo Kabushiki Kaisha

Robot system and method of controlling the robot system

US11762369B2

( en )

2019-02-06

2023-09-19

Sensory Robotics, Inc.

Robotic control via a virtual world simulation

DE102019117217B3

( en )

2019-06-26

2020-08-20

Franka Emika Gmbh

Method for specifying an input value on a robot manipulator

US11420331B2

( en )

2019-07-03

2022-08-23

Honda Motor Co., Ltd.

Motion retargeting control for human-robot interaction

CN110757454B

( en )

2019-10-12

2022-08-16

å¹¿å·ä¸å½ç§å¦é¢åè¿ææ¯ç ç©¶æ

Path planning method and device for cooperative rotation of double robots

CN112894794B

( en )

2019-11-19

2022-08-05

æ·±å³å¸ä¼å¿éç§æè¡ä»½æéå¬å¸

Human body arm action simulation method and device, terminal equipment and storage medium

CN111136633B

( en )

2020-01-13

2021-04-09

çå±±å¤§å¦

Full-state control method for flexible master-slave robot system with time-varying delay

DE102020111179B3

( en )

2020-04-24

2021-08-12

Deutsches Zentrum fÃ¼r Luft- und Raumfahrt e.V.

Method for controlling a robot and system

KR102865509B1

( en )

2020-04-27

2025-09-26

ì¦ì¨ì´ ì¡

Autonomous machines and methods for controlling them

US11518042B2

( en )

2020-05-09

2022-12-06

Ubtech Robotics Corp Ltd

Robotic assistant

US11305424B2

( en )

2020-05-22

2022-04-19

Honda Motor Co., Ltd.

Generalized admittance control for robotic manipulators

US12269173B2

( en )

2020-05-25

2025-04-08

Fanuc Corporation

Robot system

US11548150B2

( en )

2020-05-29

2023-01-10

Mitsubishi Electric Research Laboratories, Inc.

Apparatus and method for planning contact-interaction trajectories

DE102020121561B3

( en )

2020-08-17

2021-11-25

Deutsches Zentrum fÃ¼r Luft- und Raumfahrt e.V.

Control method for a bilateral teleoperation system, controller, teleoperation system and use of the controller and / or the teleoperation system

CN112207801B

( en )

2020-08-27

2022-04-26

æ·±æç§æï¼æ·±å³ï¼æéå¬å¸

Control device, control system and control method of robot

US20220111518A1

( en )

2020-10-13

2022-04-14

Autodesk, Inc.

Techniques for robot control based on generated robot simulations

WO2022145150A1

( en )

2020-12-28

2022-07-07

ã½ãã¼ã°ã«ã¼ãæ ªå¼ä¼ç¤¾

Control system and control method

CN112947439B

( en )

2021-02-05

2024-07-23

æ·±å³å¸ä¼å¿éç§æè¡ä»½æéå¬å¸

Position adjustment method, device, terminal equipment and readable storage medium

DE102022107130B4

( en )

2021-03-26

2023-02-23

Deutsches Zentrum fÃ¼r Luft- und Raumfahrt e.V.

Method of stabilizing a haptic teleoperation system with a custom haptic feedback controller

US20220371186A1

( en )

2021-05-19

2022-11-24

Guangzhou Institute of advanced technology

Dual-robot position/force multivariate-data-driven method using reinforcement learning

US20230047860A1

( en )

2021-08-13

2023-02-16

Board Of Regents Of The University Of Nebraska

Systems, methods, and media for fault localization in declarative specification languages

IT202100024899A1

( en )

2021-09-29

2023-03-29

Gaiotto Automation S P A

METHOD FOR THE SELF-LEARNING CONTROL OF A CLOSED ROBOTIC SYSTEM AND RELATED PROCESSING PLANT

JP7722891B2

( en )

2021-10-05

2025-08-13

å·å´éå·¥æ¥æ ªå¼ä¼ç¤¾

Control device, robot system, robot control method, and robot control program

CN118401346A

( en )

2021-10-11

2024-07-26

Abbçå£«è¡ä»½æéå¬å¸

Responsive teaching interface for programming an industrial robot

CN114179080B

( en )

2021-12-01

2024-12-17

ä¸æµ·ç¾çéä¿¡ç§ææéå¬å¸

Motion control method, motion control device, mechanical arm, interaction system and storage medium

CN114237054B

( en )

2021-12-18

2024-08-09

ç¦å·å¤§å¦

6D interaction control method of aerial robot

CN116038685B

( en )

2022-09-09

2026-04-17

æ·±å³å¸è¶çç§æè¡ä»½æéå¬å¸

Teleoperation system, teleoperation method and robotic arm

CN115625711B

( en )

2022-11-08

2024-04-30

æ²é³å·¥ä¸å¤§å¦

A collaborative control method for dual-arm robots considering end force

CN116968016B

( en )

2022-11-16

2024-08-06

éåºå¤§å¦

Construction method of hydraulic arm tail end speed feasible space and visual interaction system

CN116214510B

( en )

2023-02-09

2025-11-07

æå·çµè¥¿æºå¨äººæºè½ç§ææéå¬å¸

Mechanical arm admittance control method and system

CN116079733A

( en )

2023-02-10

2023-05-09

ä¸æµ·çµæ°éå¢è¡ä»½æéå¬å¸

Master-slave robot manpower feedback control method, system, equipment and medium

CN116141329B

( en )

2023-03-06

2025-08-08

åäº¬çå·¥å¤§å¦

A spatial closed-loop dual-arm robot motion control method based on self-correcting control

US20240424663A1

( en )

2023-06-26

2024-12-26

Honda Motor Co., Ltd.

Robot-mediated physical human-human interaction

JP7827227B1

( en )

2024-09-13

2026-03-10

æ ªå¼ä¼ç¤¾æé»è

Track restraint device, track restraint method

CN119338981B

( en )

2024-09-14

2025-10-31

åæµå¤§å¦

Object three-dimensional information reconstruction method based on humanoid robot body perception

Citations (6)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US6424885B1

( en )

1999-04-07

2002-07-23

Intuitive Surgical, Inc.

Camera referenced control in a minimally invasive surgical apparatus

CN101610721A

( en )

2006-09-25

2009-12-23

Haptic feedback medical scanning and system

US20150066051A1

( en )

2013-09-04

2015-03-05

Samsung Electronics Co., Ltd.

Surgical robot and control method thereof

CN104758055A

( en )

2005-05-19

2015-07-08

ç´è§å¤ç§ææ¯æä½å¬å¸

Software center and highly configurable robotic systems for surgery and other uses

CN105101903A

( en )

2013-02-04

2015-11-25

å¿ç«¥å½å®¶å»çä¸å¿

Hybrid Control Surgical Robotic System

US9855653B2

( en )

2013-11-07

2018-01-02

Muscle Corporation

Master-slave system

2017

2017-11-08

EP17800780.3A

patent/EP3538328B1/en

active

Active

2017-11-08

PCT/EP2017/078558

patent/WO2018087123A2/en

not_active

Ceased

2017-11-08

DK17800780.3T

patent/DK3538328T5/en

active

2017-11-08

CN201780074062.9A

patent/CN110035871B/en

active

Active

2017-11-08

US16/348,137

patent/US11305431B2/en

active

Active

2017-11-08

KR1020197014701A

patent/KR102353178B1/en

active

Active

Patent Citations (6)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US6424885B1

( en )

1999-04-07

2002-07-23

Intuitive Surgical, Inc.

Camera referenced control in a minimally invasive surgical apparatus

CN104758055A

( en )

2005-05-19

2015-07-08

ç´è§å¤ç§ææ¯æä½å¬å¸

Software center and highly configurable robotic systems for surgery and other uses

CN101610721A

( en )

2006-09-25

2009-12-23

Haptic feedback medical scanning and system

CN105101903A

( en )

2013-02-04

2015-11-25

å¿ç«¥å½å®¶å»çä¸å¿

Hybrid Control Surgical Robotic System

US20150066051A1

( en )

2013-09-04

2015-03-05

Samsung Electronics Co., Ltd.

Surgical robot and control method thereof

US9855653B2

( en )

2013-11-07

2018-01-02

Muscle Corporation

Master-slave system

Non-Patent Citations (12)

* Cited by examiner, â Cited by third party

Title

Alcocer et al.; " Force Estimation and Control in Robot Manipulators "; Elsevier IFAC Publications; Proc. 7th IFAC Symposium on Robot Control 2003 (SYROCO'03), Wroclaw, Poland; vol. 1, p. 55; Sep. 1-3, 2004.

Billard et al.; " Springer Handbook of Robotics", 2nd ed., Chapter "Learning from Humans "; pp. 1995-2014, 2016.

Borghesan et al.; " A constraint-based programming approach to physical human-robot interaction "; Robotics and Automation (ICRA), 2012 IEEE International conference May 14-18, 2012.

Daly et al.; " Bilateral teleoperation using unknown input observers for force estimation "; American Control Conference, 2009; ACC '09, IEEE; Piscataway, NJ, USA; p. 89-95; Jun. 10, 2009.

Hacksel et al.; " Estimation of environment forces and rigid-body velocities using observers "; from Proc. IEEE Int. Conf. Robotics and Automation (ICRA); San Diego, CA; pp. 931-936; May 8-13, 1994.

International Search Report on corresponding PCT application (PCT/EP2017/078558) from International Searching Authority (EPO) dated Jul. 6, 2018.

Li et al., " Bilateral Teleoperation of Holonomic Constrained Robotic Systems With Time-Varying Delays "; IEEE Transactions on Instrumentation and Measurement; IEEE Service Center; Piscataway, NJ, US; vol. 62, No. 4; p. 752-765; Apr. 1, 2013.

Turro et al., " Haptically Augmented Teleoperation "; Proceedings of the 2001 IEEE International Conference on Robotics and Automation; ICRA 2001; Seoul, Korea; p. 386-392; May 21-26, 2001.

Van Damme et al.; " Estimating robot end-effector force from noisy actuator torque measurements "; IEEE International Conference Robotics and Automation (ICRA); Shanghai, China; pp. 1108-1113; May 9-11, 2011.

Written Opinion on corresponding PCT application (PCT/EP2017/078558) from International Searching Authority (EPO) dated Jul. 6, 2018.

Yokokohji et al.; " Bilateral Control of Master-Slave Manipulators for Ideal Kinesthetic Coupling-Formulation and Experiment "; IEEE Transactions on Robotics and Automation; IEEE Inc.; New York, US; vol. 10, No. 5; p. 605-619; Oct. 1, 1994.

Yokokohji, " Bilateral Control of Master-Slave Manipulators for Ideal Kinesthetic Coupling-Formulation and Experiment ", 1994 (Year: 1994).

Cited By (5)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12214975B2

( en )

2021-06-16

2025-02-04

Dexterity, Inc.

State estimation using geometric data and vision system for palletizing

US12258227B2

( en )

2021-06-16

2025-03-25

Dexterity, Inc.

Simulated box placement for algorithm evaluation and refinement

US12390928B2

( en )

2021-06-16

2025-08-19

Dexterity, Inc.

Workflow for using tree search-based approach for placing boxes on pallet with limited knowledge of future sequence

US12485544B2

( en )

2021-06-16

2025-12-02

Dexterity, Inc.

Workflow for using learning based approach for placing boxes on pallets

US12486125B2

( en )

2021-06-16

2025-12-02

Dexterity, Inc.

Collision avoidance based on vision data, geometric data and physics engine evaluation

Also Published As

Publication number

Publication date

DK3538328T5

( en )

2020-09-28

US20190358817A1

( en )

2019-11-28

EP3538328A2

( en )

2019-09-18

KR20190075098A

( en )

2019-06-28

WO2018087123A3

( en )

2018-08-09

CN110035871A

( en )

2019-07-19

DK3538328T3

( en )

2020-09-07

CN110035871B

( en )

2022-09-23

EP3538328B1

( en )

2020-06-10

WO2018087123A2

( en )

2018-05-17

KR102353178B1

( en )

2022-01-20

System and method for instructing a robot — Cognibotics Ab (US11305431B2)

Related documents