Google Patentsopen access

Systems and methods for autonomous drone flight control — Sentinel Advancements, Inc. (US20230221732A1)

ABSTRACT

Abstract

A method and system for controlling autonomous vehicles comprises a flight controller comprising at least one processor and a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight controller further comprising: an obstacle avoidance module, a state machine, a data link module, and a control computer module.

Description

TECHNICAL FIELD

Embodiments are generally related to the field of controllers. Embodiments are also related to state machines. Embodiments are further related to the field of autonomous vehicles and drones. Embodiments are also related to methods, systems, and devices for controllers for drones and/or autonomous vehicles. Embodiments are further related to systems and methods for autonomous drone flight controllers based on a state machine.

BACKGROUND

The advent of autonomous flying vehicles was a significant milestone for all engineers involved in the aerospace field. This has allowed drone deployment in many application domains. For example, drones are now used for commercial missions, surveillance, reconnaissance, mapping, cartography, homeland security, rescue, and remote sensing. Among these, last-mile delivery is becoming one of the most promising applications. The motivation for drones is to make processes faster and more flexible, in addition to improving precision and cost-efficiency.

Vertical takeoff and landing (VTOL) drones require no takeoff or landing run. Hence, they are suitable for applications where the landing area is limited. For example, in commercial missions, VTOL is particularly attractive because the aircraft can hover over the delivery place, land, and/or drop off a package. By contrast, a fixed-wing, conventional takeoff, and landing (CVTOL) drone needs ground infrastructure such as runways and land personal. Therefore, their configuration makes them suitable for long-range, high-capacity applications. However, they have poor hover-ability.

Currently, some drones are equipped with integrated mechanisms for specifying and executing low level reactive control tasks. This is the solution going from one control mode to another. For example, before switching between hovering mode and flight-to-mission mode, the attitude and the alignment of the aircraft should be verified with the path to follow. This is carried out by applying a yaw maneuver before switching the control mode. However, current drone control systems can become overloaded quickly as each task can be divided into different tasks, leading to an unmanageably large number of possible states.

As such current prior art solutions do not efficiently partition the problem to provide a granular solution using different levels of abstraction. Certain prior art methods for addressing this problem are in the works, but all known attempts are ill equipped to deal with simple configurations and complex behaviors simultaneously. Thus, for a simple configuration, stability and reachability analysis may be untenable with prior art approaches.

Accordingly, there is a need in the art for methods and systems that address the limited autonomy and processing capacity of prior art solutions by providing a state machine in the high level as disclosed herein.

SUMMARY

The following summary is provided to facilitate an understanding of some of the innovative features unique to the embodiments disclosed and is not intended to be a full description. A full appreciation of the various aspects of the embodiments can be gained by taking the entire specification, claims, drawings, and abstract as a whole.

It is, therefore, one aspect of the disclosed embodiments to provide improved methods and systems for controllers.

It is another aspect of the disclosed embodiments to provide a method, system, and apparatus for autonomous vehicles, drones, or other such unmanned vehicles.

It is another aspect of the disclosed embodiments to provide methods, systems, and apparatuses for controlling autonomous vehicles, drones, or other such unmanned vehicles.

It is another aspect of the disclosed embodiments to provide methods, systems, and apparatuses comprising state machines for controlling autonomous vehicles, drones, or other such unmanned vehicles.

In the embodiments disclosed herein, a system, method, and apparatus for a state machine that operates at a high level and interacts with both high level and low level elements of a controller are disclosed. The current embodiments include a state machine with a plurality of operating states during which, tasks of the high level components are carried out and interact with the low level to gather information already processed and available. This is integrated into a model where the low level and the high level are decoupled.

For example, in an embodiment a system comprises a flight controller comprising: at least one processor, and a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight controller further comprising: an obstacle avoidance module; a state machine; a data link module; and a control computer module.

In an embodiment, the state machine can have one of a plurality of states comprising: an initialization state, an arming state, a takeoff state, a mission state, a landing state, and teleoperation state. In an embodiment, the flight controller directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine. In an embodiment, the vehicle undergoes initialization during the initialization state. In an embodiment, the vehicle transitions to the arming state after it has been initialized. In an embodiment, the vehicle is prepared for takeoff and the control computer module and obstacle avoidance module are notified the vehicle is ready for takeoff. In an embodiment, the control computer module, obstacle avoidance module, and flight controller guide the takeoff of the vehicle and the state machine is transitioned to the mission state. In an embodiment, upon completion of a mission, the state machine transitions to the land state; and the flight controller, control computer, and obstacle avoidance guide the vehicle to landing.

In an embodiment, upon fault detection in any of the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing.

In an embodiment, the vehicle comprises an autonomous aerial vehicle controlled by the flight controller. In an embodiment, the system comprises a ground station (GS) configured to provide remote control to the autonomous aerial vehicle.

In another embodiment, a method comprises enabling a vehicle for flight with a state machine configured to control the vehicle with a plurality of states, initializing a flight in an initialization state of the state machine, arming the vehicle for obstacle avoidance in an arming state of the state machine, directing the vehicle to take off in a takeoff state of the state machine, completing a missing with the vehicle in a mission state of the state machine, overriding other states for remote control of the vehicle by a ground station in a teleoperation state of the state machine, and landing the vehicle in a land state of the state machine.

In an embodiment of the method a flight controller directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine. In an embodiment, the method comprises notifying a control computer module and an obstacle avoidance module when the vehicle is ready for takeoff. In an embodiment the method comprises checking for a fault condition and upon fault detection in any fault in the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing. In an embodiment of the method, the vehicle comprises an autonomous aerial vehicle controlled by the flight controller. In an embodiment of the method the ground station provides remote control to the autonomous aerial vehicle.

In another embodiment, a system for UAV control comprises a flight controller comprising: at least one processor and a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight controller further comprising: an obstacle avoidance module; a state machine wherein the state machine can have one of a plurality of states comprising: an initialization state; an arming state; a takeoff state; a mission state; a landing state; and teleoperation state; a data link module; a control computer module; and a ground station configured to provide remote control to the autonomous aerial vehicle.

In an embodiment, the flight computer directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine. In an embodiment, upon fault detection in any of the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing.

BRIEF DESCRIPTION OF THE FIGURES

The accompanying figures, in which reference numerals refer to identical or functionally-similar elements throughout the separate views and which are incorporated in and form a part of the specification, further illustrate the embodiments and, together with the detailed description, serve to explain the embodiments disclosed herein.

FIG. 1 depicts a block diagram of a computer system which is implemented in accordance with the disclosed embodiments;

FIG. 2 depicts a graphical representation of a network of data-processing devices in which aspects of the present embodiments may be implemented;

FIG. 3 depicts a computer software system for directing the operation of the data-processing system depicted in FIG. 1 , in accordance with an example embodiment;

FIG. 4 depicts a system diagram of an autonomous vehicle system, in accordance with the disclosed embodiments;

FIG. 5 depicts a flow chart of transitions between states in a state machine, in accordance with the disclosed embodiments;

FIG. 6 depicts a flow chart of operations associated with an initialization state, in accordance with the disclosed embodiments;

FIG. 7 depicts a flow chart of operations associated with an arming state, in accordance with the disclosed embodiments;

FIG. 8 depicts a flow chart of operations associated with a takeoff state, in accordance with the disclosed embodiments;

FIG. 9 depicts a flow chart of operations associated with a mission state, in accordance with the disclosed embodiments;

FIG. 10 depicts a flow chart of operations associated with a land state, in accordance with the disclosed embodiments;

FIG. 11 depicts a flow chart of operations associated with a teleoperation state, in accordance with the disclosed embodiments;

FIG. 12 depicts a system diagram of aspects of an autonomous vehicle system, in accordance with the disclosed embodiments;

FIG. 13 depicts a system diagram of an obstacle avoidance system, in accordance with the disclosed embodiments;

FIG. 14 depicts a system diagram of an alarm module, in accordance with the disclosed embodiments;

FIG. 15 depicts a system diagram of a ground station, in accordance with the disclosed embodiments; and

FIG. 16 depicts aspects of an unmanned autonomous vehicle (UAV), in accordance with the disclosed embodiments.

DETAILED DESCRIPTION

The particular values and configurations discussed in the following non-limiting examples can be varied, and are cited merely to illustrate one or more embodiments and are not intended to limit the scope thereof.

Example embodiments will now be described more fully hereinafter, with reference to the accompanying drawings, in which illustrative embodiments are shown. The embodiments disclosed herein can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiments to those skilled in the art. Like numbers refer to like elements throughout.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Throughout the specification and claims, terms may have nuanced meanings suggested or implied in context beyond an explicitly stated meaning. Likewise, the phrase “in one embodiment” as used herein does not necessarily refer to the same embodiment and the phrase “in another embodiment” as used herein does not necessarily refer to a different embodiment. It is intended, for example, that claimed subject matter include combinations of example embodiments in whole or in part.

TECHNICAL FIELD

Embodiments are generally related to the field of controllers. Embodiments are also related to state machines. Embodiments are further related to the field of autonomous vehicles and drones. Embodiments are also related to methods, systems, and devices for controllers for drones and/or autonomous vehicles. Embodiments are further related to systems and methods for autonomous drone flight controllers based on a state machine.

BACKGROUND

The advent of autonomous flying vehicles was a significant milestone for all engineers involved in the aerospace field. This has allowed drone deployment in many application domains. For example, drones are now used for commercial missions, surveillance, reconnaissance, mapping, cartography, homeland security, rescue, and remote sensing. Among these, last-mile delivery is becoming one of the most promising applications. The motivation for drones is to make processes faster and more flexible, in addition to improving precision and cost-efficiency.

Vertical takeoff and landing (VTOL) drones require no takeoff or landing run. Hence, they are suitable for applications where the landing area is limited. For example, in commercial missions, VTOL is particularly attractive because the aircraft can hover over the delivery place, land, and/or drop off a package. By contrast, a fixed-wing, conventional takeoff, and landing (CVTOL) drone needs ground infrastructure such as runways and land personal. Therefore, their configuration makes them suitable for long-range, high-capacity applications. However, they have poor hover-ability.

Currently, some drones are equipped with integrated mechanisms for specifying and executing low level reactive control tasks. This is the solution going from one control mode to another. For example, before switching between hovering mode and flight-to-mission mode, the attitude and the alignment of the aircraft should be verified with the path to follow. This is carried out by applying a yaw maneuver before switching the control mode. However, current drone control systems can become overloaded quickly as each task can be divided into different tasks, leading to an unmanageably large number of possible states.

As such current prior art solutions do not efficiently partition the problem to provide a granular solution using different levels of abstraction. Certain prior art methods for addressing this problem are in the works, but all known attempts are ill equipped to deal with simple configurations and complex behaviors simultaneously. Thus, for a simple configuration, stability and reachability analysis may be untenable with prior art approaches.

Accordingly, there is a need in the art for methods and systems that address the limited autonomy and processing capacity of prior art solutions by providing a state machine in the high level as disclosed herein.

SUMMARY

The following summary is provided to facilitate an understanding of some of the innovative features unique to the embodiments disclosed and is not intended to be a full description. A full appreciation of the various aspects of the embodiments can be gained by taking the entire specification, claims, drawings, and abstract as a whole.

It is, therefore, one aspect of the disclosed embodiments to provide improved methods and systems for controllers.

It is another aspect of the disclosed embodiments to provide a method, system, and apparatus for autonomous vehicles, drones, or other such unmanned vehicles.

It is another aspect of the disclosed embodiments to provide methods, systems, and apparatuses for controlling autonomous vehicles, drones, or other such unmanned vehicles.

It is another aspect of the disclosed embodiments to provide methods, systems, and apparatuses comprising state machines for controlling autonomous vehicles, drones, or other such unmanned vehicles.

In the embodiments disclosed herein, a system, method, and apparatus for a state machine that operates at a high level and interacts with both high level and low level elements of a controller are disclosed. The current embodiments include a state machine with a plurality of operating states during which, tasks of the high level components are carried out and interact with the low level to gather information already processed and available. This is integrated into a model where the low level and the high level are decoupled.

For example, in an embodiment a system comprises a flight controller comprising: at least one processor, and a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight controller further comprising: an obstacle avoidance module; a state machine; a data link module; and a control computer module.

In an embodiment, the state machine can have one of a plurality of states comprising: an initialization state, an arming state, a takeoff state, a mission state, a landing state, and teleoperation state. In an embodiment, the flight controller directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine. In an embodiment, the vehicle undergoes initialization during the initialization state. In an embodiment, the vehicle transitions to the arming state after it has been initialized. In an embodiment, the vehicle is prepared for takeoff and the control computer module and obstacle avoidance module are notified the vehicle is ready for takeoff. In an embodiment, the control computer module, obstacle avoidance module, and flight controller guide the takeoff of the vehicle and the state machine is transitioned to the mission state. In an embodiment, upon completion of a mission, the state machine transitions to the land state; and the flight controller, control computer, and obstacle avoidance guide the vehicle to landing.

In an embodiment, upon fault detection in any of the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing.

In an embodiment, the vehicle comprises an autonomous aerial vehicle controlled by the flight controller. In an embodiment, the system comprises a ground station (GS) configured to provide remote control to the autonomous aerial vehicle.

In another embodiment, a method comprises enabling a vehicle for flight with a state machine configured to control the vehicle with a plurality of states, initializing a flight in an initialization state of the state machine, arming the vehicle for obstacle avoidance in an arming state of the state machine, directing the vehicle to take off in a takeoff state of the state machine, completing a missing with the vehicle in a mission state of the state machine, overriding other states for remote control of the vehicle by a ground station in a teleoperation state of the state machine, and landing the vehicle in a land state of the state machine.

In an embodiment of the method a flight controller directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine. In an embodiment, the method comprises notifying a control computer module and an obstacle avoidance module when the vehicle is ready for takeoff. In an embodiment the method comprises checking for a fault condition and upon fault detection in any fault in the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing. In an embodiment of the method, the vehicle comprises an autonomous aerial vehicle controlled by the flight controller. In an embodiment of the method the ground station provides remote control to the autonomous aerial vehicle.

In another embodiment, a system for UAV control comprises a flight controller comprising: at least one processor and a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight controller further comprising: an obstacle avoidance module; a state machine wherein the state machine can have one of a plurality of states comprising: an initialization state; an arming state; a takeoff state; a mission state; a landing state; and teleoperation state; a data link module; a control computer module; and a ground station configured to provide remote control to the autonomous aerial vehicle.

In an embodiment, the flight computer directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine. In an embodiment, upon fault detection in any of the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing.

BRIEF DESCRIPTION OF THE FIGURES

The accompanying figures, in which reference numerals refer to identical or functionally-similar elements throughout the separate views and which are incorporated in and form a part of the specification, further illustrate the embodiments and, together with the detailed description, serve to explain the embodiments disclosed herein.

FIG. 1 depicts a block diagram of a computer system which is implemented in accordance with the disclosed embodiments;

FIG. 2 depicts a graphical representation of a network of data-processing devices in which aspects of the present embodiments may be implemented;

FIG. 3 depicts a computer software system for directing the operation of the data-processing system depicted in FIG. 1 , in accordance with an example embodiment;

FIG. 4 depicts a system diagram of an autonomous vehicle system, in accordance with the disclosed embodiments;

FIG. 5 depicts a flow chart of transitions between states in a state machine, in accordance with the disclosed embodiments;

FIG. 6 depicts a flow chart of operations associated with an initialization state, in accordance with the disclosed embodiments;

FIG. 7 depicts a flow chart of operations associated with an arming state, in accordance with the disclosed embodiments;

FIG. 8 depicts a flow chart of operations associated with a takeoff state, in accordance with the disclosed embodiments;

FIG. 9 depicts a flow chart of operations associated with a mission state, in accordance with the disclosed embodiments;

FIG. 10 depicts a flow chart of operations associated with a land state, in accordance with the disclosed embodiments;

FIG. 11 depicts a flow chart of operations associated with a teleoperation state, in accordance with the disclosed embodiments;

FIG. 12 depicts a system diagram of aspects of an autonomous vehicle system, in accordance with the disclosed embodiments;

FIG. 13 depicts a system diagram of an obstacle avoidance system, in accordance with the disclosed embodiments;

FIG. 14 depicts a system diagram of an alarm module, in accordance with the disclosed embodiments;

FIG. 15 depicts a system diagram of a ground station, in accordance with the disclosed embodiments; and

FIG. 16 depicts aspects of an unmanned autonomous vehicle (UAV), in accordance with the disclosed embodiments.

DETAILED DESCRIPTION

The particular values and configurations discussed in the following non-limiting examples can be varied, and are cited merely to illustrate one or more embodiments and are not intended to limit the scope thereof.

Example embodiments will now be described more fully hereinafter, with reference to the accompanying drawings, in which illustrative embodiments are shown. The embodiments disclosed herein can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiments to those skilled in the art. Like numbers refer to like elements throughout.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Throughout the specification and claims, terms may have nuanced meanings suggested or implied in context beyond an explicitly stated meaning. Likewise, the phrase “in one embodiment” as used herein does not necessarily refer to the same embodiment and the phrase “in another embodiment” as used herein does not necessarily refer to a different embodiment. It is intended, for example, that claimed subject matter include combinations of example embodiments in whole or in part.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

It is contemplated that any embodiment discussed in this specification can be implemented with respect to any method, kit, reagent, or composition of the invention, and vice versa. Furthermore, compositions of the invention can be used to achieve the methods of the invention.

It will be understood that particular embodiments described herein are shown by way of illustration and not as limitations of the invention. The principal features of this invention can be employed in various embodiments without departing from the scope of the invention. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, numerous equivalents to the specific procedures described herein. Such equivalents are considered to be within the scope of this invention and are covered by the claims.

The use of the word “a” or “an” when used in conjunction with the term “comprising” in the claims and/or the specification may mean “one,” but it is also consistent with the meaning of “one or more,” “at least one,” and “one or more than one.” The use of the term “or” in the claims is used to mean “and/or” unless explicitly indicated to refer to alternatives only or the alternatives are mutually exclusive, although the disclosure supports a definition that refers to only alternatives and “and/or.” Throughout this application, the term “about” is used to indicate that a value includes the inherent variation of error for the device, the method being employed to determine the value, or the variation that exists among the study subjects.

As used in this specification and claim(s), the words “comprising” (and any form of comprising, such as “comprise” and “comprises”), “having” (and any form of having, such as “have” and “has”), “including” (and any form of including, such as “includes” and “include”) or “containing” (and any form of containing, such as “contains” and “contain”) are inclusive or open-ended and do not exclude additional, unrecited elements or method steps.

The term “or combinations thereof” as used herein refers to all permutations and combinations of the listed items preceding the term. For example, “A, B, C, or combinations thereof” is intended to include at least one of: A, B, C, AB, AC, BC, or ABC, and if order is important in a particular context, also BA, CA, CB, CBA, BCA, ACB, BAC, or CAB. Continuing with this example, expressly included are combinations that contain repeats of one or more items or terms, such as BB, AAA, AB, BBC, AAABCCCC, CBBAAA, CABABB, and so forth. The skilled artisan will understand that typically there is no limit on the number of items or terms in any combination, unless otherwise apparent from the context.

All of the compositions and/or methods disclosed and claimed herein can be made and executed without undue experimentation in light of the present disclosure. While the compositions and methods of this invention have been described in terms of preferred embodiments, it will be apparent to those of skill in the art that variations may be applied to the compositions and/or methods and in the steps or in the sequence of steps of the method described herein without departing from the concept, spirit, and scope of the invention. All such similar substitutes and modifications apparent to those skilled in the art are deemed to be within the spirit, scope and concept of the invention as defined by the appended claims.

FIGS. 1 - 3 are provided as exemplary diagrams of data-processing environments in which embodiments of the present invention may be implemented. It should be appreciated that FIGS. 1 - 3 are only exemplary and are not intended to assert or imply any limitation with regard to the environments in which aspects or embodiments of the disclosed embodiments may be implemented. Many modifications to the depicted environments may be made without departing from the spirit and scope of the disclosed embodiments.

A block diagram of a computer system 100 that executes programming for implementing parts of the methods and systems disclosed herein is shown in FIG. 1 . A computing device in the form of a computer 110 configured to interface with sensors, peripheral devices, and other elements disclosed herein may include one or more processing units 102 , memory 104 , removable storage 112 , and non-removable storage 114 . Memory 104 may include volatile memory 106 and non-volatile memory 108 . Computer 110 may include or have access to a computing environment that includes a variety of transitory and non-transitory computer-readable media such as volatile memory 106 and non-volatile memory 108 , removable storage 112 and non-removable storage 114 . Computer storage includes, for example, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM) and electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD ROM), Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium capable of storing computer-readable instructions as well as data including image data.

Computer 110 may include or have access to a computing environment that includes input 116 , output 118 , and a communication connection 120 . The computer may operate in a networked environment using a communication connection 120 to connect to one or more remote computers, remote sensors, detection devices, hand-held devices, multi-function devices (MFDs), mobile devices, tablet devices, mobile phones, Smartphones, or other such devices. The remote computer may also include a personal computer (PC), server, router, network PC, RFID enabled device, a peer device or other common network node, or the like. The communication connection may include a Local Area Network (LAN), a Wide Area Network (WAN), Bluetooth connection, or other networks. This functionality is described more fully in the description associated with FIG. 2 below.

Output 118 is most commonly provided as a computer monitor, but may include any output device. Output 118 and/or input 116 may include a data collection apparatus associated with computer system 100 . In addition, input 116 , which commonly includes a computer keyboard and/or pointing device such as a computer mouse, computer trackpad, or the like, allows a user to select and instruct computer system 100 . A user interface can be provided using output 118 and input 116 . Output 118 may function as a display for displaying data and information for a user, and for interactively displaying a graphical user interface (GUI) 130 .

Note that the term “GUI” generally refers to a type of environment that represents programs, files, options, and so forth by means of graphically displayed icons, menus, and dialog boxes on a computer monitor screen. A user can interact with the GUI to select and activate such options by directly touching the screen and/or pointing and clicking with a user input device 116 such as, for example, a pointing device such as a mouse and/or with a keyboard. A particular item can function in the same manner to the user in all applications because the GUI provides standard software routines (e.g., module 125 ) to handle these elements and report the user's actions. The GUI can further be used to display the electronic service image frames as discussed below.

Computer-readable instructions, for example, program module or node 125 , which can be representative of other modules or nodes described herein, are stored on a computer-readable medium and are executable by the processing unit 102 of computer 110 . Program module or node 125 may include a computer application. A hard drive, CD-ROM, RAM, Flash Memory, and a USB drive are just some examples of articles including a computer-readable medium.

FIG. 2 depicts a graphical representation of a network of data-processing systems 200 in which aspects of the present invention may be implemented. Network data- processing system 200 is a network of computers or other such devices such as mobile phones, smartphones, sensors, detection devices, and the like in which embodiments of the present invention may be implemented. Note that the system 200 can be implemented in the context of a software module such as program module 125 . The system 200 includes a network 202 in communication with one or

more clients

210 , 212 , and 214 , and external device 205 . The external device 205 can comprise a controller, flight computer, drone system, ground station (GS) 482 , or other such component of a drone control system. Network 202 may also be in communication with one or more RFID and/or GPS enabled devices or sensors 204 , servers 206 , and storage 208 . Network 202 is a medium that can be used to provide communications links between various devices and computers connected together within a networked data-processing system such as computer system 100 . Network 202 may include connections such as wired communication links, wireless communication links of various types, fiber optic cables, quantum, or quantum encryption, or quantum teleportation networks, etc. Network 202 can communicate with one or more servers 206 , one or more external devices such as RFID and/or GPS enabled device 204 , and a memory storage unit such as, for example, memory or database 208 . It should be understood that RFID and/or GPS enabled device 204 may be embodied as a mobile device, cell phone, tablet device, monitoring device, detector device, sensor microcontroller, controller, receiver, transceiver, or other such device.

In the depicted example, RFID and/or GPS enabled device 204 , server 206 , and

clients

210 , 212 , and 214 connect to network 202 along with storage unit 208 .

Clients

210 , 212 , and 214 may be, for example, personal computers or network computers, handheld devices, mobile devices, tablet devices, smartphones, personal digital assistants, microcontrollers, recording devices, MFDs, etc. Computer system 100 depicted in FIG. 1 can be, for example, a client such as client 210 and/or 212 .

Computer system 100 can also be implemented as a server such as server 206 , depending upon design considerations. In the depicted example, server 206 provides data such as boot files, operating system images, applications, and application updates to clients

210 , 212 , and/or 214 .

Clients

210 , 212 , and 214 and RFID and/or GPS enabled device 204 are clients to server 206 in this example. Network data- processing system 200 may include additional servers, clients, and other devices not shown. Specifically, clients may connect to any member of a network of servers, which provide equivalent content.

In the depicted example, network data- processing system 200 is the Internet with network 202 representing a worldwide collection of networks and gateways that use the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers consisting of thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, network data- processing system 200 may also be implemented as a number of different types of networks such as, for example, an intranet, a local area network (LAN), or a wide area network (WAN). FIGS. 1 and 2 are intended as examples and not as architectural limitations for different embodiments of the present invention.

FIG. 3 illustrates a software system 300 , which may be employed for directing the operation of the data-processing systems such as computer system 100 depicted in FIG. 1 . Software application 305 , may be stored in memory 104 , on removable storage 112 , or on non-removable storage 114 shown in FIG. 1 , and generally includes and/or is associated with a kernel or operating system 310 and a shell or interface 315 . One or more application programs, such as module(s) or node(s) 125 , may be “loaded” (i.e., transferred from removable storage 114 into the memory 104 ) for execution by the data- processing system 100 . The data- processing system 100 can receive user commands and data through user interface 315 , which can include input 116 and output 118 , accessible by a user 320 . These inputs may then be acted upon by the computer system 100 in accordance with instructions from operating system 310 and/or software application 305 and any software module(s) 125 thereof.

Generally, program modules (e.g., module 125 ) can include, but are not limited to, routines, subroutines, software applications, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types and instructions. Moreover, those skilled in the art will appreciate that elements of the disclosed methods and systems may be practiced with other computer system configurations such as, for example, hand-held devices, mobile phones, smartphones, tablet devices, multi-processor systems, printers, copiers, fax machines, multi-function devices, data networks, microprocessor-based or programmable consumer electronics, networked personal computers, minicomputers, mainframe computers, servers, medical equipment, medical devices, and the like.

Note that the term module or node as utilized herein may refer to a collection of routines and data structures that perform a particular task or implement a particular abstract data type. Modules may be composed of two parts: an interface, which lists the constants, data types, variables, and routines that can be accessed by other modules or routines; and an implementation, which is typically private (accessible only to that module), and which includes source code that actually implements the routines in the module. The term module may also simply refer to an application such as a computer program designed to assist in the performance of a specific task such as word processing, accounting, inventory management, etc., or a hardware component designed to equivalently assist in the performance of a task.

The interface 315 (e.g., a graphical user interface 130 ) can serve to display results, whereupon a user 320 may supply additional inputs or terminate a particular session. In some embodiments, operating system 310 and GUI 130 can be implemented in the context of a “windows” system. It can be appreciated, of course, that other types of systems are possible. For example, rather than a traditional “windows” system, other operation systems such as, for example, a real time operating system (RTOS) more commonly employed in wireless systems may also be employed with respect to operating system 310 and interface 315 . The software application 305 can include, for example, module(s) 125 , which can include instructions for carrying out steps or logical operations such as those shown and described herein.

The following description is presented with respect to embodiments of the present invention, which can be embodied in the context of, or require the use of a data-processing system such as computer system 100 , in conjunction with program module 125 , and data- processing system 200 and network 202 depicted in FIGS. 1 - 3 . The present invention, however, is not limited to any particular application or any particular environment. Instead, those skilled in the art will find that the systems and methods of the present invention may be advantageously applied to a variety of system and application software including database management systems, word processors, and the like. Moreover, the present invention may be embodied on a variety of different platforms including Windows, Macintosh, UNIX, LINUX, Android, Arduino, and the like. Therefore, the descriptions of the exemplary embodiments, which follow, are for purposes of illustration and not considered a limitation.

The embodiments, disclosed herein are generally configured to control autonomous vehicles. The term “autonomous vehicle” as used herein can be interchangeably used with the terms, “drone,” “aircraft,” “vehicle,” “UAV,” etc. The disclosed embodiments comprise two-levels, the low level 1280 and the high level 1281 as depicted in FIG. 12 . The high level 1281 is provided by the autonomy payload in a flight computer (FC) 478 . The FC 478 can comprise the obstacle avoidance (OA) module 479 , the data link (DL) module 481 and the state machine 480 . The low level 1280 can include input/ output devices 1282 such as sensors and actuators and the modules that interact directly with them, i.e., the control computer (CC) 483 . The low level 1280 control architecture of a UAV 405 can include a control and management architecture, configured to analyze the parameters provided to the CC 483 . The CC 483 uses control laws associated with one or more actuators 1282 to provide mobility and stability to the UAV 405 .

The disclosed embodiments improve autonomy and processing capacity by introducing a state machine in the high level. The hardware of the state machine can help increase the level of autonomy and perform processing onboard without increasing the computational burden in the low level.

The state machine of the disclosed embodiments can operate in the high level of the FC 478 to allow each module involved to carry out its own tasks. The state machine can control and monitor the high level and can communicate with the low level.

In certain embodiments, the low level and the high level are decoupled in the system architecture. The high level includes the OA module 479 , the DL module 481 and the state machine 480 . In this scheme, it is also necessary to communicate with the CC 483 . Therefore, a state machine 480 in the high level allows decou

CLAIMS

Claims ( 20 )

What is claimed is:

1 . A system comprising:

a flight computer comprising: at least one processor; and at least a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight computer further comprising:

an obstacle avoidance module;

a state machine;

a data link module; and

a control computer module.

2 . The system of claim 1 wherein the state machine can have one of a plurality of states comprising:

an initialization state;

an arming state;

a takeoff state;

a mission state;

a landing state; and

teleoperation state.

3 . The system of claim 2 wherein the flight computer directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine.

4 . The system of claim 2 wherein the vehicle undergoes initialization during the initialization state.

5 . The system of claim 4 wherein the vehicle transitions to the arming state after it has been initialized.

6 . The system of claim 5 wherein:

the vehicle is prepared for takeoff; and

the control computer module and the obstacle avoidance module are notified that the vehicle is ready for takeoff.

7 . The system of claim 6 wherein:

the control computer module, the obstacle avoidance module, and flight computer guide the takeoff of the vehicle; and

the state machine is transitioned to the mission state.

8 . The system of claim 7 wherein:

upon completion of a mission, the state machine transitions to a land state; and

the flight computer, control computer module, and obstacle avoidance guide the vehicle to landing.

9 . The system of claim 8 wherein:

upon fault detection in any of the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing.

10 . The system of claim 1 wherein the vehicle comprises an autonomous aerial vehicle controlled by the flight computer.

11 . The system of claim 10 further comprising:

a ground station configured to provide remote control to the autonomous aerial vehicle.

12 . A method comprising:

enabling a vehicle for flight with a state machine configured to control the vehicle with a plurality of states; initializing a flight in an initialization state of the state machine; arming the vehicle for obstacle avoidance in an arming state of the state machine; directing the vehicle to take off in a takeoff state of the state machine; completing a mission with the vehicle in a mission state of the state machine; overriding other states for remote control of the vehicle by a ground station in a teleoperation state of the state machine; and landing the vehicle in a land state of the state machine.

13 . The method of claim 12 wherein a flight controller directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine.

14 . The method of claim 12 further comprising:

Notifying a control computer module and an obstacle avoidance module when the vehicle is ready for takeoff.

15 . The method of claim 12 further comprising:

checking for a fault condition; and

upon fault detection during the takeoff state, the mission state, and/or the land state, the state computer transitions to the teleoperation state to guide the vehicle to a safe landing.

16 . The method of claim 13 wherein the vehicle comprises an autonomous aerial vehicle controlled by the flight controller

17 . The method of claim 12 wherein the ground station provides remote control to the autonomous aerial vehicle.

18 . A system for UAV control comprising:

a flight computer comprising:

at least one processor; and

a computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with at least one processor, the computer program code comprising instructions executable by at least one processor and configured for controlling a vehicle, the flight computer further comprising:

an obstacle avoidance module;

a state machine wherein the state machine can have one of a plurality of states comprising: an initialization state; an arming state; a takeoff state; a mission state; a landing state; and teleoperation state;

a data link module; and

a control computer module; and

a ground station configured to provide remote control to the autonomous aerial vehicle.

19 . The system for UAV control of claim 18 wherein the flight controller directs control of the vehicle according to a current state selected from the plurality of states associated with the state machine.

20 . The system for UAV control of claim 18 wherein:

upon a stuck or lost alarm, during the takeoff state, the mission state or the landing state, the state machine informs the ground station, and the teleoperation state takes over control;

upon fault detection in any of the initialization state and the arming state, the state machine informs the ground station of the fault;

upon fault detection in any of the takeoff state, the mission state and the landing state, the state machine informs the ground station, the obstacle avoidance module, and the control computer module; and

upon a distress alarm in the initialization state, the arming state, the takeoff state, the mission state and the landing state, the state machine informs the ground station, the obstacle avoidance module, and the control computer module.

US17/572,567

2022-01-10

2022-01-10

Systems and methods for autonomous drone flight control

Abandoned

US20230221732A1

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US17/572,567

US20230221732A1

( en )

2022-01-10

2022-01-10

Systems and methods for autonomous drone flight control

Applications Claiming Priority (1)

Application Number

Priority Date

Filing Date

Title

US17/572,567

US20230221732A1

( en )

2022-01-10

2022-01-10

Systems and methods for autonomous drone flight control

Publications (1)

Publication Number

Publication Date

US20230221732A1

true

US20230221732A1 ( en )

2023-07-13

Family

ID=87069482

Family Applications (1)

Application Number

Title

Priority Date

Filing Date

US17/572,567

Abandoned

US20230221732A1

( en )

2022-01-10

2022-01-10

Systems and methods for autonomous drone flight control

Country Status (1)

Country

Link

US

( 1 )

US20230221732A1

( en )

Cited By (1)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

CN119935152A

( en )

*

2025-03-26

2025-05-06

烟台中商人工智能科技有限公司

UAV swarm path planning system for complex scene cruising

Citations (3)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20180002017A1

( en )

*

2016-03-28

2018-01-04

Zipline International Inc.

Vision Based Calibration System For Unmanned Aerial Vehicles

US20180196435A1

( en )

*

2017-01-06

2018-07-12

Aurora Flight Sciences Corporation

Collision Avoidance System and Method for Unmanned Aircraft

US20180275654A1

( en )

*

2015-09-03

2018-09-27

Commonwealth Scientific And Industrial Research Or Ganisation

Unmanned Aerial Vehicle Control Techniques

2022

2022-01-10

US

US17/572,567

patent/US20230221732A1/en

not_active

Abandoned

Patent Citations (3)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20180275654A1

( en )

*

2015-09-03

2018-09-27

Commonwealth Scientific And Industrial Research Or Ganisation

Unmanned Aerial Vehicle Control Techniques

US20180002017A1

( en )

*

2016-03-28

2018-01-04

Zipline International Inc.

Vision Based Calibration System For Unmanned Aerial Vehicles

US20180196435A1

( en )

*

2017-01-06

2018-07-12

Aurora Flight Sciences Corporation

Collision Avoidance System and Method for Unmanned Aircraft

Cited By (1)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

CN119935152A

( en )

*

2025-03-26

2025-05-06

烟台中商人工智能科技有限公司

UAV swarm path planning system for complex scene cruising

Similar Documents

Publication

Publication Date

Title

US20250336300A1

( en )

2025-10-30

Unmanned Aircraft Control Using Ground Control Station

US8626361B2

( en )

2014-01-07

System and methods for unmanned aerial vehicle navigation

EP3557358B1

( en )

2022-06-08

Adaptive autonomy system architecture

CN107831783B

( en )

2019-10-08

A ground station control system supporting autonomous flight of multiple UAVs

Nonami et al.

2010

Autonomous flying robots: unmanned aerial vehicles and micro aerial vehicles

Moffatt et al.

2020

Obstacle detection and avoidance system for small UAVs using a LiDAR

US20170269594A1

( en )

2017-09-21

Controlling an Unmanned Aerial System

US20170285631A1

( en )

2017-10-05

Unmanned aerial vehicle modular command priority determination and filtering system

Qays et al.

2020

Design and implementation of autonomous quadcopter using SITL simulator

CN111601755A

( en )

2020-08-28

Autonomous unmanned aerial vehicle and its control method

US20140166816A1

( en )

2014-06-19

Automated hazard handling routine engagement

US20230028792A1

( en )

2023-01-26

Machine learning architectures for camera-based detection and avoidance on aircrafts

CN111295627A

( en )

2020-06-16

Underwater piloting unmanned aerial vehicle system

Akib et al.

2025

Efficient route planning and navigation in drones using pixhawk autopilot

KR20210010164A

( en )

2021-01-27

System and method for managing cluster flight of unmanned aerial vehicle

Sherman et al.

2018

Cooperative search and rescue using autonomous unmanned aerial vehicles

EP3969381B1

( en )

2023-09-06

A system and method for drone release detection

Dousse et al.

2016

Extension of a ground control interface for swarms of small drones

EP4193231B1

( en )

2025-02-26

Vehicle controller

Areias et al.

2022

A control and communications platform for procedural mission planning with multiple aerial drones

EP4354414A1

( en )

2024-04-17

Collision avoidance system and method

US10558209B1

( en )

2020-02-11

System and method for cooperative operation of piloted and optionally piloted aircraft

KR20210010171A

( en )

2021-01-27

System and method for controlling flight sequence of unmanned aerial vehicle

CN117647999A

( en )

2024-03-05

Unmanned aerial vehicle control method, unmanned aerial vehicle and readable medium

Esherwi

2026

Design and Implementation of an Intelligent UAV-Based Border Surveillance System with Autonomous Navigation and Real-Time Monitoring

Legal Events

Date

Code

Title

Description

2022-01-24

AS

Assignment

Owner name : SENTINEL ADVANCEMENTS, INC., TEXAS

Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SOSSA AZUELA, JUAN HUMBERTO;NANDAYAPA ALFARO, MANUEL DE JESUS;GONZALEZ, RAUL ROJAS;AND OTHERS;SIGNING DATES FROM 20220105 TO 20220106;REEL/FRAME:058745/0992

2023-12-14

STCB

Information on status: application discontinuation

Free format text : ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

2023-12-17

STCB

Information on status: application discontinuation

Free format text : ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

Related documents

• Virtual reality system using multiple force arrays for a solver — DeepMotion, Inc. (US10657696B2)#653961
• Augmented reality system with spatialized audio tied to user manipulated … — Magic Leap, Inc. (US11231770B2)#653963
• High-efficiency fuel cell system with carbon dioxide capture assembly and … — Fuelcell Energy, Inc. (US10170782B2)#653965
• Integrated electrochemical cell power generation and electrolysis systems — Czero, Inc. (US20250087711A1)#653966
• Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)#653960