ABSTRACT
Abstract
A method to enable autonomous and tele-operation of tele-operated robots for maintenance of a property around known and unknown obstacles may include using an unmanned aerial vehicle for obtaining additional data relating to the property and obstacles within the property and plan a path around the obstacles using data from sensors on-board the tele-operated robot and the aerial image. A method may also provide optimization of total time needed for performing the property maintenance and the labor costs in situations where manual intervention is needed for navigating the tele-operated robot around obstacles on the property or for removing obstacles on the property. Embodiments further include systems and devices practicing the method.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
This application in a continuation of International Application No. PCT/US20/26846, filed on Apr. 6, 2020 which claims the benefit of priority to U.S. Provisional Application No. 62/830,447, filed on Apr. 6, 2019, the disclosures of both are incorporated herein by reference in their entirety.
TECHNICAL FIELD
The systems, devices and methods disclosed herein are directed to tele-operated robots, and in particular to tele-operated robots used for outdoor property maintenance.
BACKGROUND
Tele-operated robots have generally been used in hazardous situations such as bomb defusal, inspection of nuclear or chemical plants, or other situations where an operator may be exposed to hazardous environment. Another application of tele-operated robots is in robot assisted surgeries where the robots augment the capabilities of a surgeon and reduce the size of incision.
One untapped application for tele-operated robots is in various processes occurring during outdoor property maintenance such as navigation, lawn maintenance, landscaping, fumigation, pest control, spraying of surfaces, etc.
SUMMARY
In an aspect of the present disclosure, a method for operating a tele-operated robot for maintenance of a property includes obtaining an aerial image of the property using an unmanned aerial vehicle (UAV). The aerial image is transmitted to a control center communicatively coupled to the UAV and the tele-operated robot. The control center includes a processor. An area of interest within the property where property maintenance is to be performed based on the aerial image is determined at the control center. Based on the aerial image, the area of interest is classified at the control center as a first area that is autonomously navigable by the tele-operated robot and a second area that is not autonomously navigable by the tele-operated robot. A schedule of operation of the tele-operated robot for performing the property maintenance in the area of interest is determined at the control center based on a relative size of the first area and the second area. The schedule is configured to minimize labor hours and total time spent in performing the property maintenance.
In another aspect of the present disclosure, a method for operating a tele-operated robot for maintenance of a property includes obtaining, during an operation of the tele-operated robot for maintenance of the property, data relating to an obstacle in an operating path of the tele-operated robot using a sensor of the tele-operated robot. Upon detection of the obstacle, it is determined, at a control center, whether a path avoiding the obstacle can be estimated based on the data relating to the obstacle. The control center includes a processor and is communicatively coupled to the tele-operated robot. Upon a determination that a path avoiding the obstacle cannot be estimated, a flight of an UAV is initiated at by the control center. The flight path of the UAV is configured to obtain an aerial image of an area surrounding the obstacle for enabling estimation of the path avoiding the obstacle while minimizing deviation from the operating path of the tele-operated robot.
In yet another aspect of the present disclosure, a method for operating a tele-operated robot for maintenance of a property includes obtaining, during an autonomous operation of the tele-operated robot for maintenance of the property, data relating to an obstacle in an operating path of the tele-operated robot using a sensor of the tele-operated robot. At a control center, it is determined whether one or both of a position and a classification of the obstacle is previously known. The control center includes a processor communicatively coupled to the tele-operated robot. At the control center, upon a determination that the position and the classification of the obstacle is not previously known, it is determined whether an alternate operating path that preserves an unmaintained area of the property while avoiding the obstacle can be estimated based on the data relating to the obstacle. At the control center, upon a determination that the alternate operating path can be estimated, the alternate operating path is estimated. The autonomous operation of the tele-operated robot is continued using the alternate operating path.
BRIEF DESCRIPTION OF THE DRAWINGS
The disclosed aspects will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed aspects, wherein like designations denote like elements.
FIG. 1 shows a block diagram of a tele-operated robotic system, in accordance with some embodiments of the present disclosure.
FIG. 2 illustrates a flow chart for a method for obstacle avoidance in accordance with an embodiment of the present disclosure.
FIGS. 3A, 3B and 3C show flow charts for a method of operating a tele-operated robot for property maintenance, in accordance with an embodiment of the present disclosure.
FIG. 4 shows a flow chart for a method of operating a tele-operated robot for maintenance of a property in accordance with an embodiment of the present disclosure.
DETAILED DESCRIPTION
Human-in-the-loop based supervision is the process of using humans as a backstop to partially automated robots or tools equipped with electromechanical actuators. This is known more commonly as conditional automation. In automotive parlance, this is also called Level 3 automation. In this the tasks performed run the gamut of tasks that need next to no human supervision, to tasks that need a material percentage of human supervision.
The system involves providing the remote operator with data captured from sensors (force, temperature, pressure etc.) and video cameras installed on the on-site robot and surrounding implements, and providing them with an interface to relay commands back to the actuators on the on-site robot via a telemetry data link.
Property maintenance involves several tasks that are perceptually complex but relatively low on the scale of dexterity necessary. In this scenario the maximum value addition of human labor comes from the ability to provide a layer of remote management over several machines, when dealing with unforeseen situations, or situations in which there is not yet high confidence in the robot's decision making capabilities. Moreover, it is relatively easier to automate a machine given that we have a backstop for dealing with unforeseen contingencies rather than climb the asymptote of ever rarer cases for which specific decision paths need to be applied.
For many tasks, this is the only possible way to achieve cost efficiencyâpair a robot with some fractional amount of human oversight, often in a cheaper remote location with lower costs of labor. This allows lots of industries in property maintenance (included but not limited to lawn care, tree care, neighborhood security, parcel delivery, home cleaning etc).
FIG. 1 shows a block diagram of a tele-operated robotic system, in accordance with some embodiments of the present disclosure. In an implementation, the tele-operated robotic system 100 includes a tele-operated robot 110 and a control center 130 . In some implementations, the tele-operated robotic system 100 may additionally include an unmanned aerial vehicle (UAV, interchangeably referred to herein as a drone).
A. Tele-Operated Robot
The term âtele-operated robotâ as used herein refers to a robot capable of being operated or supervised locally or remotely by a human user as well as being capable of functioning autonomously. It will be understood that functioning autonomously does not necessarily mean functioning fully autonomously without any human supervision or support. In other words, functioning autonomously as used herein does not refer to automation similar to automotive Level 5 automation. Thus, a tele-operated robot in accordance with an implementation of the present disclosure can function autonomously; however, a human user can override the autonomous control of the tele-operated robot and control it locally or remotely.
Referring to FIG. 1 , a tele-operated robot 110 may include a toolkit 112 , a processor 114 , a sensor module 116 and a communication module 118 . While not shown in the Figures, in some embodiments, the tele-operated robot may optionally include augmented reality (AR) or virtual reality (VR) markers, shapes or contour lines, or paint to enable it to be easily identified by a UAV flying overhead using basic machine vision approaches, e.g., to enable the UAV to distinguish the tele-operated robot 110 from low lying obstacles, or ground terrain during operation.
The toolkit 112 may include one or more tools or actuators enabling the tele-operated robot 110 to perform its functions. For example, the toolkit of a tele-operated robotic lawn mower may include one or more set of blades structured and positioned for cutting grass on the ground within a property being maintained. The toolkit of a tele-operated robotic lawn mower may also include a suction motor for sucking up cut grass and other debris, a container or a basket for collecting the sucked up grass and other debris, a hose for connecting the suction motor to the container or basket, as well as other tools generally suitable for a cutting and/or shaping grass on the ground.
In the context of tele-operated robots for property maintenance, the tele-operated robots may include, but are not limited to, lawn mower; hedge trimmer; string trimmer; tiller; cultivator; weed puller; pole saw; leaf blower; chain saw; hedge shears; pressure washer wand for washing surfaces; fumigator for spraying pesticides, insecticides, herbicides, anti-viral sprays, anti-bacterial sprays; a UV wand for disinfecting surfaces and frequently touched surfaces; or any other tools suitable for landscaping and/or property maintenance. The toolkit for the tele-operated robots may, therefore, vary depending on the primary function of the robot. In some embodiments, some or all of the tools within the toolkit for the tele-operated robot may be replaceable by a different tool such that the primary function of the tele-operated robot is changed.
The processor 114 may include one or more processors such as one or more motion processing units (MPUs), digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. The term âprocessor,â as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured as described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of an MPU and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with an MPU core, or any other such configuration.
In some embodiments, the processor 114 processes data received from the sensor module 116 and/or the communication module 118 to navigate and operate the tele-operated robot 110 . In some embodiments, the processor 114 processes the commands and controls received from communication module 118 to navigate and operate the tele-operated robot 110 .
In some embodiments, the processor 114 may control the operation of the one or more actuators. For example, in some embodiments, the processor 114 may change the operating speed at which the actuators perform the task based on latency of communication (e.g., navigation or operational commands) being received from a control center. In some embodiments, the processor 114 may calculate the minimum reaction time needed to tele-supervise/tele-operate in a given obstacle density (and a given view), and calculate a minimum stopping distance (both for motion/any kind of dextrous manipulation) under the constraints of this reaction time.
The sensor module 116 may include one or more sensors that enable autonomous operation of the tele-operated robot or enable the local or remote operator or supervisor of the tele-operated robot to sense an environment surroun
CROSS REFERENCE TO RELATED APPLICATIONS
This application in a continuation of International Application No. PCT/US20/26846, filed on Apr. 6, 2020 which claims the benefit of priority to U.S. Provisional Application No. 62/830,447, filed on Apr. 6, 2019, the disclosures of both are incorporated herein by reference in their entirety.
TECHNICAL FIELD
The systems, devices and methods disclosed herein are directed to tele-operated robots, and in particular to tele-operated robots used for outdoor property maintenance.
BACKGROUND
Tele-operated robots have generally been used in hazardous situations such as bomb defusal, inspection of nuclear or chemical plants, or other situations where an operator may be exposed to hazardous environment. Another application of tele-operated robots is in robot assisted surgeries where the robots augment the capabilities of a surgeon and reduce the size of incision.
One untapped application for tele-operated robots is in various processes occurring during outdoor property maintenance such as navigation, lawn maintenance, landscaping, fumigation, pest control, spraying of surfaces, etc.
SUMMARY
In an aspect of the present disclosure, a method for operating a tele-operated robot for maintenance of a property includes obtaining an aerial image of the property using an unmanned aerial vehicle (UAV). The aerial image is transmitted to a control center communicatively coupled to the UAV and the tele-operated robot. The control center includes a processor. An area of interest within the property where property maintenance is to be performed based on the aerial image is determined at the control center. Based on the aerial image, the area of interest is classified at the control center as a first area that is autonomously navigable by the tele-operated robot and a second area that is not autonomously navigable by the tele-operated robot. A schedule of operation of the tele-operated robot for performing the property maintenance in the area of interest is determined at the control center based on a relative size of the first area and the second area. The schedule is configured to minimize labor hours and total time spent in performing the property maintenance.
In another aspect of the present disclosure, a method for operating a tele-operated robot for maintenance of a property includes obtaining, during an operation of the tele-operated robot for maintenance of the property, data relating to an obstacle in an operating path of the tele-operated robot using a sensor of the tele-operated robot. Upon detection of the obstacle, it is determined, at a control center, whether a path avoiding the obstacle can be estimated based on the data relating to the obstacle. The control center includes a processor and is communicatively coupled to the tele-operated robot. Upon a determination that a path avoiding the obstacle cannot be estimated, a flight of an UAV is initiated at by the control center. The flight path of the UAV is configured to obtain an aerial image of an area surrounding the obstacle for enabling estimation of the path avoiding the obstacle while minimizing deviation from the operating path of the tele-operated robot.
In yet another aspect of the present disclosure, a method for operating a tele-operated robot for maintenance of a property includes obtaining, during an autonomous operation of the tele-operated robot for maintenance of the property, data relating to an obstacle in an operating path of the tele-operated robot using a sensor of the tele-operated robot. At a control center, it is determined whether one or both of a position and a classification of the obstacle is previously known. The control center includes a processor communicatively coupled to the tele-operated robot. At the control center, upon a determination that the position and the classification of the obstacle is not previously known, it is determined whether an alternate operating path that preserves an unmaintained area of the property while avoiding the obstacle can be estimated based on the data relating to the obstacle. At the control center, upon a determination that the alternate operating path can be estimated, the alternate operating path is estimated. The autonomous operation of the tele-operated robot is continued using the alternate operating path.
BRIEF DESCRIPTION OF THE DRAWINGS
The disclosed aspects will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed aspects, wherein like designations denote like elements.
FIG. 1 shows a block diagram of a tele-operated robotic system, in accordance with some embodiments of the present disclosure.
FIG. 2 illustrates a flow chart for a method for obstacle avoidance in accordance with an embodiment of the present disclosure.
FIGS. 3A, 3B and 3C show flow charts for a method of operating a tele-operated robot for property maintenance, in accordance with an embodiment of the present disclosure.
FIG. 4 shows a flow chart for a method of operating a tele-operated robot for maintenance of a property in accordance with an embodiment of the present disclosure.
DETAILED DESCRIPTION
Human-in-the-loop based supervision is the process of using humans as a backstop to partially automated robots or tools equipped with electromechanical actuators. This is known more commonly as conditional automation. In automotive parlance, this is also called Level 3 automation. In this the tasks performed run the gamut of tasks that need next to no human supervision, to tasks that need a material percentage of human supervision.
The system involves providing the remote operator with data captured from sensors (force, temperature, pressure etc.) and video cameras installed on the on-site robot and surrounding implements, and providing them with an interface to relay commands back to the actuators on the on-site robot via a telemetry data link.
Property maintenance involves several tasks that are perceptually complex but relatively low on the scale of dexterity necessary. In this scenario the maximum value addition of human labor comes from the ability to provide a layer of remote management over several machines, when dealing with unforeseen situations, or situations in which there is not yet high confidence in the robot's decision making capabilities. Moreover, it is relatively easier to automate a machine given that we have a backstop for dealing with unforeseen contingencies rather than climb the asymptote of ever rarer cases for which specific decision paths need to be applied.
For many tasks, this is the only possible way to achieve cost efficiencyâpair a robot with some fractional amount of human oversight, often in a cheaper remote location with lower costs of labor. This allows lots of industries in property maintenance (included but not limited to lawn care, tree care, neighborhood security, parcel delivery, home cleaning etc).
FIG. 1 shows a block diagram of a tele-operated robotic system, in accordance with some embodiments of the present disclosure. In an implementation, the tele-operated robotic system 100 includes a tele-operated robot 110 and a control center 130 . In some implementations, the tele-operated robotic system 100 may additionally include an unmanned aerial vehicle (UAV, interchangeably referred to herein as a drone).
A. Tele-Operated Robot
The term âtele-operated robotâ as used herein refers to a robot capable of being operated or supervised locally or remotely by a human user as well as being capable of functioning autonomously. It will be understood that functioning autonomously does not necessarily mean functioning fully autonomously without any human supervision or support. In other words, functioning autonomously as used herein does not refer to automation similar to automotive Level 5 automation. Thus, a tele-operated robot in accordance with an implementation of the present disclosure can function autonomously; however, a human user can override the autonomous control of the tele-operated robot and control it locally or remotely.
Referring to FIG. 1 , a tele-operated robot 110 may include a toolkit 112 , a processor 114 , a sensor module 116 and a communication module 118 . While not shown in the Figures, in some embodiments, the tele-operated robot may optionally include augmented reality (AR) or virtual reality (VR) markers, shapes or contour lines, or paint to enable it to be easily identified by a UAV flying overhead using basic machine vision approaches, e.g., to enable the UAV to distinguish the tele-operated robot 110 from low lying obstacles, or ground terrain during operation.
The toolkit 112 may include one or more tools or actuators enabling the tele-operated robot 110 to perform its functions. For example, the toolkit of a tele-operated robotic lawn mower may include one or more set of blades structured and positioned for cutting grass on the ground within a property being maintained. The toolkit of a tele-operated robotic lawn mower may also include a suction motor for sucking up cut grass and other debris, a container or a basket for collecting the sucked up grass and other debris, a hose for connecting the suction motor to the container or basket, as well as other tools generally suitable for a cutting and/or shaping grass on the ground.
In the context of tele-operated robots for property maintenance, the tele-operated robots may include, but are not limited to, lawn mower; hedge trimmer; string trimmer; tiller; cultivator; weed puller; pole saw; leaf blower; chain saw; hedge shears; pressure washer wand for washing surfaces; fumigator for spraying pesticides, insecticides, herbicides, anti-viral sprays, anti-bacterial sprays; a UV wand for disinfecting surfaces and frequently touched surfaces; or any other tools suitable for landscaping and/or property maintenance. The toolkit for the tele-operated robots may, therefore, vary depending on the primary function of the robot. In some embodiments, some or all of the tools within the toolkit for the tele-operated robot may be replaceable by a different tool such that the primary function of the tele-operated robot is changed.
The processor 114 may include one or more processors such as one or more motion processing units (MPUs), digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. The term âprocessor,â as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured as described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of an MPU and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with an MPU core, or any other such configuration.
In some embodiments, the processor 114 processes data received from the sensor module 116 and/or the communication module 118 to navigate and operate the tele-operated robot 110 . In some embodiments, the processor 114 processes the commands and controls received from communication module 118 to navigate and operate the tele-operated robot 110 .
In some embodiments, the processor 114 may control the operation of the one or more actuators. For example, in some embodiments, the processor 114 may change the operating speed at which the actuators perform the task based on latency of communication (e.g., navigation or operational commands) being received from a control center. In some embodiments, the processor 114 may calculate the minimum reaction time needed to tele-supervise/tele-operate in a given obstacle density (and a given view), and calculate a minimum stopping distance (both for motion/any kind of dextrous manipulation) under the constraints of this reaction time.
The sensor module 116 may include one or more sensors that enable autonomous operation of the tele-operated robot or enable the local or remote operator or supervisor of the tele-operated robot to sense an environment surrounding the tele-operated robot. Examples of various sensors that may be included in a tele-operated robot for property maintenance include, but are not limited to, camera, stereo camera LIDAR, RADAR, ultrasound sensors, GPS positioning system, IR sensors, spectral sensors covering various portions of the light spectrum, and gyroscope(s). In addition, the sensor module 116 may also include sensors that enable the tele-operated robot to detect its performance. For example, a lawn mower may include a sensor for detecting a height of the grass to be cut as well as height of the grass that has been cut. In some embodiments, the lawn mower may include a sensor for detecting the quality of grass using, for example, a spectral analysis of the grass. As another example, a sensor may detect coverage area and spread of a gas/liquid that has been sprayed over a given surface.
The sensor module 116 may obtain data relating to an environment surrounding the tele-operated robot 110 , e.g., in the form of optical images, LIDAR data, and/or ultrasound data. In some embodiments, the sensor module 116 may additionally or alternately obtain information about the (absolute or relative) position or orientation of the tele-operated robot 110 .
In some embodiments, the data obtained by the sensor module 116 is used by the processor 114 to operate the tele-operated robot 110 autonomously. In some embodiments, the data obtained by the sensor module 116 is used by the processor 114 to augment commands received by the tele-operated robot 110 from a human operator. In some embodiments, the data obtained by the sensor module 116 is used by the processor 114 to additionally or alternately provide a feedback the human operator about the environment and/or operation of the tele-operated robot 110 .
The communication module 118 may include a receiver and a transmitter configured to wirelessly communicate with other components of the system 100 . The transmitter and/or the receiver of the communication module 118 may utilize any presently available communication protocols such as, for example, 4G, 5G, WiMax, WiFi, or a combination thereof to communicate with the other components of the system 100 through a network, e.g., Internet. In some embodiments, the communication module 118 may be configured to utilize multiple communication protocols for improving reliability of communication. For example, in an embodiment, the tele-operated robot 110 may communicate with the UAV 150 using WiFi while communicating with the control center 130 using 4G or 5G communication protocols. It will be understood that the communication protocols used by the communication module 118 are not limited to those presently available, and as communication technology advances, other protocols may be used in the future. Thus, the scope of the present disclosure is not limited to presently available communication protocols, but also includes any communication protocols that may be available in the future.
For example, the communication module 118 enables the tele-operated robot 110 to communicate with the control center 130 or the UAV 150 . In some embodiments, the communication module 118 can transmit sensor data from the tele-operated robot 110 to the control center 130 , and receive command and control data from the control center 130 . In some embodiments, the communication module 118 may additionally or alternately receive aerial image information from the UAV 150 .
B. Control Center
The term âcontrol centerâ as used herein refers to a component of the tele-operated robotic system that provides control and commands for the tele-operated robot. Thus, the control center may provide navigation and/or operational commands provided by a human operator or supervisor to the tele-operated robot. In some embodiments, the control center may additionally or alternately provide navigation and/or operational commands based on analysis of the data received by the control center (either from the tele-operated robot or from other components of the system such as the UAV).
Referring to FIG. 1 , the control center 130 may include a command module 132 , a processor 134 , a communication module 136 , and optionally a virtual control room 138 .
The command module 132 may include an input terminal such as, for example, a laptop, a console, a desktop, a tablet, a mobile computing device, or a mobile phone. The input terminal may be provided with input devices such as, for example, a key board, a joystick, a mouse, a microphone, a game controller or a combination thereof to enable a human user to input commands to the input terminal.
The commands input by the human user are processed by the input terminal and transmitted through the control center 130 to the tele-operated robot 110 . The commands sent from the control center 130 to the tele-operated robot 110 may cause the tele-operated robot 110 to move around an area of interest along a certain path, change a mode of operation, change of speed of movement, change a speed of operation, In some embodiments, the tele-operated robot 110 may continue to operate autonomously if no command is sent through the control center 130 .
The processor 134 may include one or more processors such as one or more motion processing units (MPUs), digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. The term âprocessor,â as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured as described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of an MPU and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with an MPU core, or any other such configuration.
In some embodiments, the processor 134 may be part of the input terminal of the command module 132 . In some embodiments, the processor 134 may be separate from the input terminal of the command module 132 , and dedicated for processing the data received by the control center 130 from other components of the system 100 such as, for example, the tele-operated robot 110 and/or the UAV 150 .
In some embodiments, the control center 130 is implemented such that the processor 134 resides onboard the tele-operated robot 110 while the command module 132 is provided at a remote site. In some embodiments, the processor resides onboard a UAV 150 while the command module 132 is provided at a remote site. In some embodiments, the processor 134 and the command module 132 are provided onboard the tele-operated robot 110 and an additional identical (or different) command module 132 is provided at a remote site. Other permutations are contemplated within the scope of the present disclosure.
In some embodiments, the processor 134 may process the data received from the tele-operated robot 110 and/or the UAV 150 and generate commands for navigation and/or operation of the tele-operated robot 110 . In some embodiments, the commands generated by the processor 134 may further include commands issued by a human user through the command module 132 .
The communication module 136 may include a receiver and a transmitter configured to wirelessly communicate with other components of the system 100 . The transmitter and/or the receiver of the communication module 136 may utilize any presently available communication protocols such as, for example, LTE, 5G, WiMax, WiFi, etc. to communicate with the other components of the system 100 through a network, e.g., Internet. It will be understood that the communication protocols used by the communication module 136 are not limited to those presently available, and as communication technology advances, other protocols may be used in the future. Thus, the scope of the present disclosure is not limited to presently available communication protocols, but also includes any communication protocols that may be available in the future.
For example, the communication module 136 enables the control center 130 to communicate with the tele-operated robot 110 or the UAV 150 . In some embodiments, the communication module 136 can transmit commands and controls from the control center 130 to the tele-operated robot 110 , and receive data from the tele-operated robot 110 and/or other components of the system 100 such as the UAV 150 .
The virtual control room 138 may include output devices such as, for example, a display and a speaker to enable the human user to visualize and/or listen to the environment surrounding the tele-operated robot 110 being operated or supervised through the control center 130 .
In some embodiments, a map of the property may be loaded into a virtual space where the human supervisor/operator may enter the commands for the tele-operated robot. These commands may include but are not limited to, actuation instructions, navigation instructions, etc. These commands may generate the necessary changes in the virtual space and may be iteratively or continuously altered till the desired output is created. Then, the sequence is transmitted to the tele-operated robot in the real world where it then creates the desired task. In other words, the sequence of commands provided in the virtual space act as a blueprint for actions performed by the tele-operated robot. Advantageously, such a blueprint may be helpful in reducing latency or the need for real-time transfer of high-bandwidth data. For example, in some embodiments, rather than transmitting data relating to an entire scene, only data relating to a change in the scene may be transmitted, which can then be rendered appropriately in the virtual space.
In some embodiments, the virtual control room 138 may include a dome screen display for a more immersive experience for the human supervisor/operator. For example, the human supervisor/operator may be mounted on a chair in front of a hemispherical dome when direct 1:1 tele-operation is needed. In some embodiments, the position of the human supervisor/operator is calculated such that the perspective is intuitive and aids in creating perspective. In some embodiments, the chair is provided with a force feedback and a joystick for situations where a tactile feedback relayed to the tele-operated robot is important for efficacy. In some binaural audio is provided to the human supervisor/operator in order to replay the aspects of auditory feedback.
It will be understood that while the discussion that follows may use a tele-operated lawn mower, interchangeably referred to herein as a robotic lawn mower, as an example of how various methods according to the present disclosure operate, the discussion is not meant limit the scope of the present disclosure. One of ordinary skill in the art, upon an understanding of the present disclosure, would find the discussion to be equally applicable and or readily modifiable for other types of tele-operated robots.
C. Unmanned Aerial Vehicle
Referring back to FIG. 1 , the system 100 may optionally include an unmanned aerial vehicle (also interchangeably referred to herein as a UAV or a drone) 150 . In some embodiments, the UAV 150 may include a flight control module 152 , a sensor module 154 and a communication module 156 .
The UAV 150 may be any unmanned aerial vehicle that is capable of autonomous flight, semi-autonomous flight or remotely controlled flight. A UAV may include, but is not limited to a drone, a helicopter, a multi-copter, a hexa-copter, a octocopter, a balloon, a blimp, or the like or a mixture of a drone and an airplane, capable of vertical take-off and landing (VTOL).
In some embodiments, the UAV may be equipped with a GPS (global positioning system) controller in order to determine its current location and plan a path to a target location. In some embodiments, the UAV may be operated with electrical power, rechargeable batteries and/or with a combustion engine. In embodiments in which the UAV is operated with electrical power, it may be generated using fuel cells.
The flight control module 152 is primarily responsible for controlling the flight of the UAV including, for example, landing, take-off, navigation, aerial obstacle avoidance, and so forth. The flight control module 152 may include a controller including a processor.
In some embodiments, the processor may additionally or optionally process data obtained by the sensors of the sensor module 154 before transmitting to the control center 130 and/or the tele-operated robot 110 .
The sensor module 154 may include one or more sensors that enable autonomous operation of the UAV. In addition, the sensor module 154 includes one or more sensors that enable collection of data relating to the property and augment the data available to the tele-operated robot 110 from the sensor module 116 . For example, the sensor module 154 may include, without limitation, a depth camera, video camera, stereo camera, LIDAR, RADAR, ultrasound sensors, GPS positioning system, IR sensors, spectral sensors covering various portions of the light spectrum, and gyroscope(s).
In some embodiments, sensor parameters such as position (of the UAV), zoom, or area of focus in real-time to obtain a clearer view of the environment around the tele-operated robot 110 for the human supervisor/operator or the autonomous navigation processor at the tele-operated robot 110 or the control center 130 . For example, if the tele-operated robot were to go under a tree canopy, the UAV would trail behind at an angle, e.g., at a 45° angle, or adjust its hovering height to be just above the tele-operated robot 110 while avoiding the trees. Alternately, or additionally, the zoom level, e.g., of the optical sensors, can be increased or decreased to obtain a clearer image of a localized part of the environment surrounding the tele-operated robot 110 to enhance tele-operation or autonomous navigation.
Thus, in some embodiments, the sensors of the sensor module 154 are selected and configured to augment or substitute the sensors onboard the tele-operated robot 110 that enable navigation of the tele-operated robot 110 .
In some embodiments, the sensor module 154 may include sensors for detecting specific structures, artifacts or defects in the environment surrounding the tele-operated robot 110 . For example, the sensor module 154 may include a spectral sensor configured to detect specific wavelengths emitted by weeds to enable the tele-operated robot 110 to detect weeds and plan an optimal operating path for weed control. As another example, the spectral sensor may be configured to detect patches of uncut or differently treated grass using images taken at different portions of the light spectrum.
The communication module 156 may include a wireless transmitter and receiver to communicate with the control center 130 and/or the tele-operated robot 110 or any other components of system 100 . Via the communication module 156 , the UAV 150 may receive instructions from the control center 130 including, for example, coordinates of the property being maintained and a flight path for an overflight over the property.
D. Method for Obstacle Avoidance
As discussed herein, tele-operated robots, such as the tele-operated robot 110 , may be used for performing property maintenance. For example, in an embodiment, a tele-operated robot such as a tele-operated lawn mower may be used for mowing grass in a stretch of land, e.g., of a golf course. Such a stretch of land may include known and unknown objects which may be obstacles for the tele-operated lawn mower when mowing the lawn. Disclosed herein are methods and systems for avoiding the obstacles that may obstruct a motion of a tele-operated robot.
FIG. 2 illustrates a flow chart for a method for obstacle avoidance in accordance with an embodiment of the present disclosure. In an implementation, a method for operating a tele-operated robot for maintenance of a property such as a robotic lawn mower includes obtaining, at S 202 , during an autonomous operation of the tele-operated robot for maintenance of the property, data relating to an object in an operating path of the tele-operated robot using a sensor of the tele-operated robot. In some embodiments, the operating path may be predetermined based on a prior survey of the property. In some embodiments, the operating path may be determined based on real-time analysis of an environment surrounding the tele-operated robot. The real-time analysis may be performed based on data obtained from sensors onboard the tele-operated robot in some embodiments.
At S 204 , based on the data relating to the object, at a control center, it is determined whether a probability that the object is an obstacle is greater than a threshold. The determination of whether the probability that the object is an obstacle is based on factors such as, for example, a detected shape and size of the object. The control center includes a processor communicatively coupled to the tele-operated robot. For example, in some embodiments, the control center may be the control center 130 of the system 100 . In some embodiments, the control center may be included within the tele-operated robot.
If it is determined that the probability that the object is an obstacle is not greater than the threshold, at S 206 , the control center causes the tele-operated robot to continue operating in the operating path.
If it is determined that t
CLAIMS
Claims ( 22 )
What is claimed is:
1. A method for operating a tele-operated robot for maintenance of a property, the method comprising:
receiving, at a control center, an aerial image of the property obtained using an unmanned aerial vehicle (UAV), the control center being communicatively coupled to the UAV and the tele-operated robot, and comprising a processor;
determining, at the control center, an area of interest within the property
where property maintenance is to be performed based on the aerial image;
classifying, at the control center, based on the aerial image, the area of interest as a first area that is autonomously navigable by the tele-operated robot and a second area that is not autonomously navigable by the tele-operated robot;
determining, at the control center, a schedule of operation of the tele-operated robot for performing the property maintenance in the area of interest based on a relative size of the first area and the second area, the schedule being configured to minimize labor hours and total time spent in performing the property maintenance; and
performing the property maintenance in accordance with the determined schedule of operation.
2. The method of claim 1 , wherein the second area is further classified into a third area that is navigable by remote operation of the tele-operated robot and a fourth area that requires on-site intervention by a ground-crew member.
3. The method of claim 2 , wherein the determining the schedule is further configured to minimize interference with a normal work schedule of the ground-crew member.
4. The method of claim 2 , wherein the on-site intervention by a ground-crew member comprises modifying an environment in the fourth area to render the fourth area into a first area or a third area by:
removing an object from the fourth area, smoothing of a terrain in a portion of the fourth area, moving an object within the fourth area to a different location, spraying a ultra-violet visible paint in portions of the fourth area, broadening an area from where an obstacle is removed from within the fourth area, broadening a margin of operation near a known obstacle within the fourth area, flagging a portion of the fourth area for repair, flagging a portion of the fourth area for accessory tasks, or a combination thereof.
5. The method of claim 1 , further comprising:
transmitting, from the control center, coordinates of the first area to the tele-operated robot to enable the tele-operated robot to autonomously perform the property maintenance in the first area.
6. The method of claim 1 , wherein the aerial image comprises at least one of an optical image, LIDAR data, or ultrasound sensor data.
7. The method of claim 1 , wherein classifying the area of interest comprises:
determining an operating path for the tele-operated robot within the area of interest; and
determining one or more of:
a number of obstacles inhibiting an operation of the tele-operated robot within the area of interest, a density of the obstacles per unit area within the area of interest, size of the obstacles within the area of interest, a type of the obstacles, and location of the obstacles within the area of interest relative to each other based on the aerial image.
8. The method of claim 1 , further comprising determining, at the control center, based on the aerial image, a first operating path for autonomously performing the maintenance of the property in the first area, and a second operating path for performing the maintenance of the property in the second area by remote operation of the tele-operated robot by a human operator.
9. A tele-operated robot for maintenance of a property, comprising:
a toolkit including one or more actuators configured to perform property maintenance;
a sensor module including sensors configured to sense an environment surrounding the tele-operated robot; and
a non-transitory memory coupled to a processor, the non-transitory memory having instructions thereon, the instructions causing the processor to:
receive, during an operation of the tele-operated robot for maintenance of the property, data relating to an obstacle detected in an operating path of the tele-operated robot using a sensor of the tele-operated robot, the control center comprising a processor communicatively coupled to the tele-operated robot; determine whether a path avoiding the obstacle can be estimated based on the data relating to the obstacle; and
initiate, upon a determination that a path avoiding the obstacle cannot be estimated, a flight of an unmanned aerial vehicle (UAV), a flight path of the UAV being configured to obtain an aerial image of an area surrounding the obstacle for enabling estimation of the path avoiding the obstacle while minimizing deviation from the operating path of the tele-operated robot.
10. The tele-operated robot of claim 9 , wherein the aerial image comprises at least one of an optical image, LIDAR data, or ultrasound sensor data.
11. The tele-operated robot of claim 9 , determining whether a path avoiding the obstacle can be estimated comprises:
determining whether one or both of a position and a classification of the obstacle is previously known;
determining upon a determination that the position and the classification of the obstacle is not previously known, whether a boundary box surrounding the obstacle can be estimated based on the data relating to the obstacle;
determining, upon a determination that a boundary box can be estimated based on the data relating to the obstacle, that the path avoiding the obstacle can be estimated based on the data relating to the obstacle; and
determining, upon a determination that a boundary box cannot be estimated based on the data relating to the obstacle, that the path avoiding the obstacle cannot be planned based on the data relating to the obstacle.
12. The tele-operated robot of claim 11 , wherein the instructions further cause the processor to:
estimate, upon determination that a path avoiding the obstacle can be estimated, an alternate operating path minimizing a deviation from the operating path.
13. The tele-operated robot of claim 11 , wherein the instructions further cause the processor to continue, upon determination of the alternate operating path, an autonomous operation of the tele-operated robot using the alternate operating path.
14. The tele-operated robot of claim 9 , wherein the instructions further cause the processor to: determine whether a boundary box can be estimated based on the data relating to the obstacle and the aerial image;
determine, upon a determination that a boundary box can be estimated based on the data relating to the obstacle and the aerial image, that the path avoiding the obstacle can be estimated; and
determine, at the control center, upon a determination that a boundary box cannot be estimated based on data relating to the obstacle and the aerial image, that the path avoiding the obstacle cannot be estimated and ceding control of the tele-operated robot to a human remote operator.
15. The tele-operated robot of claim 14 , wherein the instructions further cause the processor to:
estimating, at the control center, upon determination that a path avoiding the obstacle can be estimated, an alternate operating path minimizing a deviation from the operating path.
16. The tele-operated robot of claim 14 , wherein the instructions further cause the processor to continue, upon determination of the alternate operating path, an autonomous operation of the tele-operated robot using the alternate operating path.
17. The tele-operated robot of claim 9 , wherein the UAV is tethered to the tele-operated robot.
18. The tele-operated robot of claim 9 , wherein the aerial image comprises a three-dimensional (3D) geometrically corrected composite map of the property.
19. The tele-operated robot of claim 18 , wherein the instructions further cause the processor to:
transmit the 3D geometrically corrected composite map to the tele-operated robot; and
estimate a position and a distance of the tele-operated robot relative to fixed obstacles on the property in real time based on the 3D geometrically corrected composite map.
20. A method of operating a tele-operated robot for maintenance of a property, the method comprising:
obtaining, during an autonomous operation of the tele-operated robot for maintenance of the property, data relating to an obstacle in an operating path of the tele-operated robot using a sensor of the tele-operated robot;
determining, at a control center, whether one or both of a position and a classification of the obstacle is previously known, the control center comprising a processor communicatively coupled to the tele-operated robot;
determining, at the control center, upon a determination that the position and the classification of the obstacle is not previously known, whether an alternate operating path that preserves an unmaintained area of the property while avoiding the obstacle can be estimated based on the data relating to the obstacle;
estimating, at the control center, upon a determination that the alternate operating path can be estimated, the alternate operating path; and
continuing the autonomous operation of the tele-operated robot using the alternate operating path.
21. The method of claim 20 , further comprising:
determining, at the control center, upon a determination that the alternate operating path cannot be planned, whether the obstacle is removable; and
initiating, at the control center, upon a determination that the obstacle is removable, a process for manual removal of the obstacle.
22. The method of claim 20 , further comprising:
ceding, at the control center, upon a determination that the alternate operating path cannot be estimated based on data relating to the obstacle, control of the tele-operated robot to a human remote operator.
US16/989,139
2019-04-06
2020-08-10
System, devices and methods for tele-operated robotics
Active
US10906181B2
( en )
Priority Applications (4)
Application Number
Priority Date
Filing Date
Title
US16/989,139
US10906181B2
( en )
2019-04-06
2020-08-10
System, devices and methods for tele-operated robotics
US17/129,261
US11715072B2
( en )
2019-04-06
2020-12-21
System, devices and methods for tele-operated robotics
US18/227,749
US12321903B2
( en )
2019-04-06
2023-07-28
System, devices and methods for tele-operated robotics
US19/210,605
US20260099824A1
( en )
2019-04-06
2025-05-16
System, devices and methods for tele-operated robotics
Applications Claiming Priority (3)
Application Number
Priority Date
Filing Date
Title
US201962830447P
2019-04-06
2019-04-06
PCT/US2020/026846
WO2021002911A1
( en )
2019-04-06
2020-04-06
System, devices and methods for tele-operated robotics
US16/989,139
US10906181B2
( en )
2019-04-06
2020-08-10
System, devices and methods for tele-operated robotics
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
PCT/US2020/026846
Continuation
WO2021002911A1
( en )
2019-04-06
2020-04-06
System, devices and methods for tele-operated robotics
Related Child Applications (1)
Application Number
Title
Priority Date
Filing Date
US17/129,261
Continuation
US11715072B2
( en )
2019-04-06
2020-12-21
System, devices and methods for tele-operated robotics
Publications (2)
Publication Number
Publication Date
US20200368912A1
US20200368912A1 ( en )
2020-11-26
US10906181B2
true
US10906181B2 ( en )
2021-02-02
Family
ID=73457963
Family Applications (4)
Application Number
Title
Priority Date
Filing Date
US16/989,139
Active
US10906181B2
( en )
2019-04-06
2020-08-10
System, devices and methods for tele-operated robotics
US17/129,261
Active
2040-10-22
US11715072B2
( en )
2019-04-06
2020-12-21
System, devices and methods for tele-operated robotics
US18/227,749
Active
2040-05-01
US12321903B2
( en )
2019-04-06
2023-07-28
System, devices and methods for tele-operated robotics
US19/210,605
Pending
US20260099824A1
( en )
2019-04-06
2025-05-16
System, devices and methods for tele-operated robotics
Family Applications After (3)
Application Number
Title
Priority Date
Filing Date
US17/129,261
Active
2040-10-22
US11715072B2
( en )
2019-04-06
2020-12-21
System, devices and methods for tele-operated robotics
US18/227,749
Active
2040-05-01
US12321903B2
( en )
2019-04-06
2023-07-28
System, devices and methods for tele-operated robotics
US19/210,605
Pending
US20260099824A1
( en )
2019-04-06
2025-05-16
System, devices and methods for tele-operated robotics
Country Status (4)
Country
Link
US
( 4 )
US10906181B2
( en )
EP
( 1 )
EP3953115A1
( en )
JP
( 1 )
JP2022527029A
( en )
KR
( 1 )
KR20220017893A
( en )
Cited By (2)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20210129338A1
( en )
*
2019-04-06
2021-05-06
Electric Sheep Robotics, Inc.
System, devices and methods for tele-operated robotics
US11866194B2
( en )
2021-10-30
2024-01-09
Beta Air, Llc
Systems and methods for a visual system for an electric aircraft
Families Citing this family (16)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11461867B2
( en )
*
2019-09-19
2022-10-04
Sanctuary Cognitive Systems Corporation
Visual interface and communications techniques for use with robots
WO2021067757A1
( en )
*
2019-10-03
2021-04-08
The Toro Company
Site maintenance utilizing autonomous vehicles
US12252278B2
( en )
*
2020-03-12
2025-03-18
Bayer Aktiengesellschaft
Unmanned aerial vehicle
CN114863382A
( en )
*
2021-01-20
2022-08-05
æªå²å¤§é(å京)ç§ææéå ¬å¸
Recognition method and device for mowing area, electronic equipment and storage medium
CN113276130B
( en )
*
2021-05-28
2022-10-04
å±±ä¸å¤§å¦
Free-form surface spraying path planning method and system based on point cloud slice
CN113815529A
( en )
*
2021-08-23
2021-12-21
å人è¿é(ä¸æµ·)äºè®¡ç®ç§ææéå ¬å¸
Backing-up assisting method, equipment and system based on unmanned aerial vehicle and vehicle
EP4385448A4
( en )
*
2021-09-14
2024-12-04
Wuhan United Imaging Healthcare Surgical Technology Co., Ltd.
METHOD AND SYSTEM FOR CALCULATING A SPATIAL RECORDING POSE OF A SURGICAL ROBOT
US20230278223A1
( en )
*
2022-03-04
2023-09-07
Sanctuary Cognitive Systems Corporation
Robots, tele-operation systems, computer program products, and methods of operating the same
CN114639026B
( en )
*
2022-03-16
2024-12-10
é«å²èµç§æ(åé)æéå ¬å¸
Embedded Data Acquisition for UAV Analysis Systems
CN115164870A
( en )
*
2022-07-06
2022-10-11
æ²é³èªç©ºèªå¤©å¤§å¦
Indoor autonomous relative positioning navigation method in air-ground cooperation mode
CN114952870B
( en )
*
2022-07-29
2023-09-29
å±±ä¸äºå山大æºè½ç§ææéå ¬å¸
Four-axis robotic arm motion control method and system for disinfection of high-frequency contact objects
JP2024065876A
( en )
*
2022-10-31
2024-05-15
ä½åéæ©æ¢°å·¥æ¥æ ªå¼ä¼ç¤¾
Excavator and excavator control system
WO2024102415A1
( en )
*
2022-11-08
2024-05-16
Electric Sheep Robotics, Inc.
Grounds maintenance vehicle with modified cutting mechanism
CN117170411B
( en )
*
2023-11-02
2024-02-02
å±±ä¸ç¯ç»´æ¸¸ä¹è®¾å¤æéå ¬å¸
Assisted obstacle avoidance method for racing drones based on visual assistance
CN118795806B
( en )
*
2024-06-17
2025-02-07
广ä¸çº¢æµ·æ¹¾åçµæéå ¬å¸
Intelligent monitoring system of coal yard environment based on 5G technology
CN120370971B
( en )
*
2025-04-18
2025-12-26
å®æ³¢ååèªå¥¥ç§ææéå ¬å¸
Spraying route planning method for realizing automatic spraying of building outer elevation by unmanned aerial vehicle
Citations (9)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US9235763B2
( en )
*
2012-11-26
2016-01-12
Trimble Navigation Limited
Integrated aerial photogrammetry surveys
US20160093124A1
( en )
*
2014-09-30
2016-03-31
SZ DJI Technology Co., Ltd.
System and method for data recording and analysis
US20160313736A1
( en )
*
2014-01-10
2016-10-27
Pictometry International Corp.
Unmanned aircraft structure evaluation system and method
US20170185849A1
( en )
*
2015-12-23
2017-06-29
Wal-Mart Stores, Inc.
Apparatus and method for monitoring premises
US20190055018A1
( en )
*
2016-04-29
2019-02-21
SZ DJI Technology Co., Ltd.
Systems and methods for uav transport and data acquisition
US20190096058A1
( en )
*
2012-07-05
2019-03-28
Bernard Fryshman
Object image recognition and instant active response with enhanced application and utility
US10322804B2
( en )
*
2016-01-28
2019-06-18
Panasonic Intellectual Property Corporation Of America
Device that controls flight altitude of unmanned aerial vehicle
US20200159252A1
( en )
*
2018-11-21
2020-05-21
Eagle View Technologies, Inc.
Navigating unmanned aircraft using pitch
US20200255143A1
( en )
*
2017-11-07
2020-08-13
SZ DJI Technology Co., Ltd.
Three-dimensional reconstruction method, system and apparatus based on aerial photography by unmanned aerial vehicle
Family Cites Families (11)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US5903457A
( en )
*
1997-05-16
1999-05-11
Moci Co., Ltd.
Automated material storage and retrieval system for production lines
US8527094B2
( en )
*
1998-11-20
2013-09-03
Intuitive Surgical Operations, Inc.
Multi-user medical robotic system for collaboration or training in minimally invasive surgical procedures
US6321138B1
( en )
*
2000-09-15
2001-11-20
Woodson Incorporated
Storage and retrieval system with automated order make up
US6898484B2
( en )
*
2002-05-01
2005-05-24
Dorothy Lemelson
Robotic manufacturing and assembly with relative radio positioning using radio based location determination
US7304855B1
( en )
*
2003-03-03
2007-12-04
Storage Technology Corporation
Canister-based storage system
US7825650B2
( en )
*
2006-12-15
2010-11-02
Verigy (Singapore) Ptd. Ltd.
Automated loader for removing and inserting removable devices to improve load time for automated test equipment
US8463435B2
( en )
*
2008-11-25
2013-06-11
Intouch Technologies, Inc.
Server connectivity control for tele-presence robot
US10343283B2
( en )
*
2010-05-24
2019-07-09
Intouch Technologies, Inc.
Telepresence robot system that can be accessed by a cellular phone
US10464212B2
( en )
*
2017-03-28
2019-11-05
Amazon Technologies, Inc.
Method and system for tele-operated inventory management system
JP6850183B2
( en )
*
2017-04-11
2021-03-31
å·å´éå·¥æ¥æ ªå¼ä¼ç¤¾
Robot system and its operation method
EP3953115A1
( en )
*
2019-04-06
2022-02-16
Electric Sheep Robotic, Inc.
System, devices and methods for tele-operated robotics
2020
2020-04-06
EP
EP20835196.5A
patent/EP3953115A1/en
not_active
Withdrawn
2020-04-06
JP
JP2021560438A
patent/JP2022527029A/en
active
Pending
2020-04-06
KR
KR1020217036199A
patent/KR20220017893A/en
not_active
Withdrawn
2020-08-10
US
US16/989,139
patent/US10906181B2/en
active
Active
2020-12-21
US
US17/129,261
patent/US11715072B2/en
active
Active
2023
2023-07-28
US
US18/227,749
patent/US12321903B2/en
active
Active
2025
2025-05-16
US
US19/210,605
patent/US20260099824A1/en
active
Pending
Patent Citations (9)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20190096058A1
( en )
*
2012-07-05
2019-03-28
Bernard Fryshman
Object image recognition and instant active response with enhanced application and utility
US9235763B2
( en )
*
2012-11-26
2016-01-12
Trimble Navigation Limited
Integrated aerial photogrammetry surveys
US20160313736A1
( en )
*
2014-01-10
2016-10-27
Pictometry International Corp.
Unmanned aircraft structure evaluation system and method
US20160093124A1
( en )
*
2014-09-30
2016-03-31
SZ DJI Technology Co., Ltd.
System and method for data recording and analysis
US20170185849A1
( en )
*
2015-12-23
2017-06-29
Wal-Mart Stores, Inc.
Apparatus and method for monitoring premises
US10322804B2
( en )
*
2016-01-28
2019-06-18
Panasonic Intellectual Property Corporation Of America
Device that controls flight altitude of unmanned aerial vehicle
US20190055018A1
( en )
*
2016-04-29
2019-02-21
SZ DJI Technology Co., Ltd.
Systems and methods for uav transport and data acquisition
US20200255143A1
( en )
*
2017-11-07
2020-08-13
SZ DJI Technology Co., Ltd.
Three-dimensional reconstruction method, system and apparatus based on aerial photography by unmanned aerial vehicle
US20200159252A1
( en )
*
2018-11-21
2020-05-21
Eagle View Technologies, Inc.
Navigating unmanned aircraft using pitch
Non-Patent Citations (4)
* Cited by examiner, â Cited by third party
Title
Bibuli et al., The MINOAS project: Marine INspection rObotic Assistant System, 2011, IEEE, p. 1188-1193 (Year: 2011).
*
Ollero et al., Multiple eyes in the skies: architecture and perception issues in the COMETS unmanned air vehicles project, 2005, IEEE, p. 46-57 (Year: 2005).
*
Pinto et al., An autonomous surface-aerial marsupial robotic team for riverine environmental monitoring: Benefiting from coordinated aerial, underwater, and surface level perception, IEEE, p. 2015, IEEE, p. 443-450 (Year: 2015).
*
Zhang et al., A survey on multiple unmanned vehicles formation control and coordination: Normal and fault situations, 2013, IEEE, p. 1087-1096 (Year: 2013).
*
Cited By (4)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20210129338A1
( en )
*
2019-04-06
2021-05-06
Electric Sheep Robotics, Inc.
System, devices and methods for tele-operated robotics
US11715072B2
( en )
*
2019-04-06
2023-08-01
Electric Sheep Robotics, Inc.
System, devices and methods for tele-operated robotics
US12321903B2
( en )
2019-04-06
2025-06-03
Electric Sheep Robotics, Inc.
System, devices and methods for tele-operated robotics
US11866194B2
( en )
2021-10-30
2024-01-09
Beta Air, Llc
Systems and methods for a visual system for an electric aircraft
Also Published As
Publication number
Publication date
US20210129338A1
( en )
2021-05-06
US11715072B2
( en )
2023-08-01
US20200368912A1
( en )
2020-11-26
JP2022527029A
( en )
2022-05-27
US20260099824A1
( en )
2026-04-09
KR20220017893A
( en )
2022-02-14
US20240086862A1
( en )
2024-03-14
US12321903B2
( en )
2025-06-03
EP3953115A1
( en )
2022-02-16
Similar Documents
Publication
Publication Date
Title
US11715072B2
( en )
2023-08-01
System, devices and methods for tele-operated robotics
US20220357753A1
( en )
2022-11-10
Drop-off location planning for delivery vehicle
US11835953B2
( en )
2023-12-05
Adaptive autonomy system architecture
AU2020300962A1
( en )
2021-11-04
System, devices and methods for tele-operated robotics
US20240077883A1
( en )
2024-03-07
Vehicle Controllers For Agricultural And Industrial Applications
EP3428766B1
( en )
2021-04-07
Multi-sensor environmental mapping
US20170313439A1
( en )
2017-11-02
Methods and syststems for obstruction detection during autonomous unmanned aerial vehicle landings
CA3106457A1
( en )
2020-01-23
Method for exploration and mapping using an aerial vehicle
EP3210091B1
( en )
2020-02-19
Optimal safe landing area determination
CN106919178A
( en )
2017-07-04
A kind of plant protection unmanned plane autonomous flight Path Optimize Installation and its optimization method
US20250021101A1
( en )
2025-01-16
Row-based world model for perceptive navigation
EP3761136B1
( en )
2022-10-26
Control device, mobile body, and program
ES2968924T3
( en )
2024-05-14
Unmanned aerial vehicle control software method, system and product
Ngo et al.
2022
UAV platforms for autonomous navigation in GPS-denied environments for search and rescue missions
US20250021102A1
( en )
2025-01-16
Generating a mission plan with a row-based world model
Andert et al.
2011
Mapping and path planning in complex environments: An obstacle avoidance approach for an unmanned helicopter
US20250244765A1
( en )
2025-07-31
Navigating an unmanned ground vehicle
Portugall et al.
2020
Requirements Specification
Lee
2014
UPenn Multi-Robot Unmanned Vehicle System (MAGIC)
Legal Events
Date
Code
Title
Description
2020-08-10
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY
2020-08-14
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY
2020-11-04
AS
Assignment
Owner name : ELECTRIC SHEEP ROBOTICS, INC., VIRGINIA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MURTY, NAGANAND;SINGH, GUNJIT;HEROLD, JARRETT;SIGNING DATES FROM 20200404 TO 20200406;REEL/FRAME:054274/0598
2020-12-28
STPP
Information on status: patent application and granting procedure in general
Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED
2021-01-13
STCF
Information on status: patent grant
Free format text : PATENTED CASE
2024-07-31
MAFP
Maintenance fee payment
Free format text : PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY
Year of fee payment : 4