ABSTRACT
Abstract
A ground-aware drone flight planning and operation system is provided. The system may comprise a semantic map, a localization system, and a perception system. The semantic map may comprise information about ground events in a geographic area where an unmanned aerial vehicle (UAV) may operate. The localization system may localize the UAV and assist in determining nearby ground events using the semantic map. The perception system may determine real-time ground events in the vicinity of the UAV. A computerized flight planner may generate a flight path based on the localization and perception data.
Description
CROSS-REFERENCE TO RELATED APPLICATION
This application claims the benefit of U.S. Provisional Application No. 62/805,185, filed on Feb. 13, 2019, which is hereby incorporated by reference in its entirety.
BACKGROUND
Unmanned aerial vehicles (UAVs) are capable of traveling through the air without a physically-present human operator. UAVs may operate in an autonomous mode, remote-control mode, or partially autonomous mode
In a fully autonomous mode, the UAV may automatically determine its own path and operate one or more propulsion components and control components to navigate along the path. In a remote-control mode, a human operator that is remote from the UAV controls the UAV to travel along a flight path. The flight may be developed by a human or by a computer. In a partially autonomous mode, some aspects of the UAVs flight may be performed autonomously by the UAV and other aspects of the flight may be performed under remote control.
UAVs may be used for a variety of tasks. However, to date the flight paths created for UAVs have been based on coordinate systems and not based on ground-awareness. That is, the UAVs are not aware of the situation below them on the ground. This has several disadvantages. First, it can create a dangerous situation if the UAV flies over or near objects on the ground such as people or vehicles that could be injured or damaged by a falling UAV. Second, it can lead to violation of regulations implemented by the government, such as the Federal Aviation Administration and other bodies, that require UAVs to follow certain rules. Third, it can lead to non-optimal routes for the UAV if the UAV flies near ground objects that it will have to go out of its way to avoid. Other disadvantages of a non-ground-aware UAV will also become apparent from this disclosure.
SUMMARY
In some implementations, a method for generating and updating a flight path is performed based on ground-awareness.
In some embodiments, data from a semantic map, localization system, and perception system are used to generate or update a flight path based on awareness of ground objects or conditions.
In some embodiments, a UAV is provided and comprises a plurality of sensors. Localization may be performed by the localization system by applying the sensor data of the UAV. Localization may also be performed by using a semantic map to place the UAV in a coordinate system of other semantic information. Perception may be performed by the perception system by applying the sensor data of the UAV. As a result of localization and perception, ground objects and conditions are identified. A flight path is generated or updated based on the ground objects and conditions.
In some embodiments, the UAV stores an emergency landing location. The emergency landing location may also be selected based on the locations of ground objects and conditions identified through localization and perception.
BRIEF DESCRIPTION OF THE DRAWINGS
The present disclosure will become better understood from the detailed description and the drawings, a brief summary of which is provided below.
FIG. 1 illustrates an exemplary environment in which systems herein may operate.
FIG. 2 illustrates an exemplary embodiment of a UAV.
FIG. 3 illustrates an exemplary embodiment of a computer system that may be used in some embodiments.
FIG. 4 illustrates an exemplary method for updating a flight path based on ground-awareness.
FIG. 5 illustrates an exemplary method that may be performed in some embodiments to generate an initial flight path and update the flight path during flight.
FIG. 6 illustrates an exemplary method for using a cost function and flight planner to generate a flight path.
FIG. 7 illustrates an exemplary flight path comprising a plurality of waypoints.
FIG. 8 illustrates an exemplary flight path that may be used for minimizing turns and optimizing for straight line travel.
FIG. 9 illustrates an exemplary flight path that may be used for surveying and mapping roads.
FIG. 10 illustrates an exemplary flight path that may be used for traffic monitoring.
FIG. 11 illustrates an exemplary flight path that may be used for goods delivery by the UAV
DETAILED DESCRIPTION
For simplicity and illustrative purposes, the principles of the present teachings are described by referring mainly to examples of various implementations thereof. However, one of ordinary skill in the art would readily recognize that the same principles are equally applicable to, and can be implemented in, all types of information and systems, and that any such variations do not depart from the true spirit and scope of the present teachings. Moreover, in the following detailed description, references are made to the accompanying figures, which illustrate specific examples of various implementations. Logical and structural changes can be made to the examples of the various implementations without departing from the spirit and scope of the present teachings. The following detailed description is, therefore, not to be taken in a limiting sense and the scope of the present teachings is defined by the appended claims and their equivalents.
In addition, it should be understood that steps of the examples of the methods set forth in the present disclosure can be performed in different orders than the order presented in the present disclosure. Furthermore, some steps of the examples of the methods can be performed in parallel rather than being performed sequentially. Also, the steps of the examples of the methods can be performed in a network environment in which some steps are performed by different computers in the networked environment.
Some implementations are implemented by a computer system. A computer system can include a processor, a memory, and a non-transitory computer-readable medium. The memory and non-transitory medium can store instructions for performing methods and steps described herein.
Embodiments herein relate to automatically creating flight paths for UAVs, and in particular, to automatically create flight paths based on ground-awareness. Ground-awareness relates to being aware of objects and conditions on the ground underneath or in the vicinity of the UAV. For example, a ground-aware UAV may identify a location of a road on the ground and maintain a buffer distance between the UAV and the road to comply with regulations about how close a UAV may fly within a road and also to maintain safety for the vehicles on the road. For example, if the UAV encounters problems and falls to the ground, it would not interfere with the vehicles on the road. In another example, a computerized flight planner may develop a flight path that prefers to travel over less populous areas instead of more populous areas. This can reduce the risk that a falling UAV will injure someone or damage property and also reduce the incidence of noise disruptions.
A ground-aware flight plan may be generated through the combination of a semantic map, localization system, and perception system. The semantic map provides a detailed map of the environment including semantic information in the map about objects and conditions on the ground. The localization system precisely localizes the UAV in the environment and may use any one or combination of global navigation satellite system (GNSS), global positioning system (GPS), inertial measurement unit (IMU), perception-based localization systems (e.g., image-based, LIDAR based, depth-sensor based, and so on), and sensor fusion of the aforementioned approaches. After the precise localization of the UAV, the UAV is aware of the ground objects and conditions in its vicinity and relation to the UAV based on the detailed semantic map.
The perception system allows the UAV to perceive the environment in real-time using one or more sensors and a computer system for interpreting the sensor inputs. The perception system may include one or more of a camera, video camera, light detection and ranging (LIDAR), depth sensor, ultra-sonic sensor, radar, and other sensors. The sensor data collected by the perception system may be used for localization as described herein. In addition, the sensor data of the perception system may allow the UAV to detect real-time conditions in the environment that would not be known from the semantic map. For example, the perception system may be used to detect recent changes to the environment, such as new buildings, roads, or street signs, or be used to detect the real-time presence of vehicles, people, and other moving objects. The perception system may also be used to detect whether vehicles and other objects are currently moving or stationary, and thus whether it is allowed to fly over them (when stationary) or not (when moving).
A computerized flight planner may use the information from the semantic map, localization system, and perception system to generate a flight path. The UAV may then navigate along the flight path. During the flight, new detections by the perception system may cause the UAV to detect the existence of ground conditions that may cause the flight path to be re-evaluated. The ground conditions may be input to the computerized flight planner to determine an updated flight path. In some embodiments, the updated flight path may comprise delaying the UAV, for example to wait for vehicles to stop at an intersection so that the UAV may safely pass over them. In some embodiments, the updated flight path may comprise changing the coordinates of the flight path, such as if an unexpected obstacle is detected.
In some embodiments, the computerized flight planner may apply a cost function to one or more ground objects or conditions, where the ground objects or conditions may be detected from the semantic map as a result of localization or from the perception system. The cost function may accept as input an identity of the ground object or event and output weights associated with the travel of the UAV over or in the vicinity of the ground object or event. The weights may be associated with the cost of the UAV to fly along a path that would cross over or in the vicinity of the ground object or event. In other embodiments, the cost function may accept as input the raw sensor readings or semantic map data and output the weights associated with the travel of the UAV along a path over or in the vicinity of a region. Although referred to in the singular as a cost function, it should be understood that a plurality of cost functions may be used, such as for different types of ground objects and conditions. The computerized flight planner may use the weights to compute a lowest cost flight path using pathfinding algorithms such as rapidly exploring random tree algorithm (RRT), A* algorithm, Dijkstra's algorithm, D* algorithm, any-angle path finding, hierarchical path finding, and other algorithms.
In some embodiments, the ground objects or conditions detected from the semantic map or perceptions system may be impassable by the UAV, such as a tall tree or a region that cannot be passed due to government regulations. The impassable ground objects or conditions may be represented as an impassable barrier in the computerized flight planner. It may generate a path using the pathfinding algorithm by using a combination of weights generated by the cost function and the impassable obstacles. The pathfinding algorithm may generate paths based on both the path weights and the obstacles, and a lowest cost flight path may be select.
In some embodiments, the flight paths generated by the computerized flight planner are represented by a plurality of waypoints. The waypoints may represent intermediate steps on the flight path. In some embodiments, the flight path comprises flying in a straight line or substantially straight line between waypoints, so that any turns are represented by adding a new waypoint to the flight path. The UAV may deviate from the straight line path between waypoints when an obstacle or ground object or condition would require it or render a new path more efficient.
The terms âunmanned aerial vehicleâ and âUAVâ may refer to an aerial vehicle without a physically-present human operator. The terms âdrone,â âunmanned aerial vehicle systemâ (UAVS), or âunmanned aerial systemâ (UAS) may also be used to refer to a UAV.
UAVs may operate autonomously, partially autonomously, or by remote control of a human operator. An autonomous UAV may automatically develop a flight path and navigate along the flight path through a computer processor that operates one or more propulsion components and control components. In some embodiments, the autonomous UAV may require a manually developed flight path but may navigate automatically along the flight path without human control or intervention. In some embodiments, an autonomous UAV is supervised by a human operator, who can take over control if necessary, even though control is by default performed by a computer processor. A remote-control UAV may be under the control of a human operator who is remote from the UAV. The human operator may control the UAV through a control interface. Control commands may be received from the human operator at the control interface and transmitted, through wireless or wired communication, to the UAV. One or more propulsion components and control components may be controlled through operation of the human-operated control interface. Moreover, the UAV may record video, photo, and sensor data to transmit back to the human operator to allow the human operator to perceive the vicinity of the UAV. A partially autonomous UAV may include both autonomous and remote-control aspects. In one embodiment, the autonomous and remote-control commands may occur at different levels of abstraction. For example, a human operator may input commands for the UAV to travel from a start location to an end location, and an autonomous piloting system may automatically perform the low-level navigation tasks for controlling the propulsion and control systems of the UAV to fly the UAV from the start location to the end location. In such an embodiment, the human may provide high-level control and the UAV may autonomously perform low-level control. Vice versa, an autonomous UAV may perform high-level control in the form of autonomously developing a flight path and handing off the low-level control to the human-operator to perform the individual real-time control necessary to guide the UAV along the flight path. In other embodiments of a partially autonomous UAV, the split of control between the autonomous and remote-control aspect may be at the same level of abstraction. For example, the UAV may be flown in an autonomous mode until an obstacle or other difficult to navig
CROSS-REFERENCE TO RELATED APPLICATION
This application claims the benefit of U.S. Provisional Application No. 62/805,185, filed on Feb. 13, 2019, which is hereby incorporated by reference in its entirety.
BACKGROUND
Unmanned aerial vehicles (UAVs) are capable of traveling through the air without a physically-present human operator. UAVs may operate in an autonomous mode, remote-control mode, or partially autonomous mode
In a fully autonomous mode, the UAV may automatically determine its own path and operate one or more propulsion components and control components to navigate along the path. In a remote-control mode, a human operator that is remote from the UAV controls the UAV to travel along a flight path. The flight may be developed by a human or by a computer. In a partially autonomous mode, some aspects of the UAVs flight may be performed autonomously by the UAV and other aspects of the flight may be performed under remote control.
UAVs may be used for a variety of tasks. However, to date the flight paths created for UAVs have been based on coordinate systems and not based on ground-awareness. That is, the UAVs are not aware of the situation below them on the ground. This has several disadvantages. First, it can create a dangerous situation if the UAV flies over or near objects on the ground such as people or vehicles that could be injured or damaged by a falling UAV. Second, it can lead to violation of regulations implemented by the government, such as the Federal Aviation Administration and other bodies, that require UAVs to follow certain rules. Third, it can lead to non-optimal routes for the UAV if the UAV flies near ground objects that it will have to go out of its way to avoid. Other disadvantages of a non-ground-aware UAV will also become apparent from this disclosure.
SUMMARY
In some implementations, a method for generating and updating a flight path is performed based on ground-awareness.
In some embodiments, data from a semantic map, localization system, and perception system are used to generate or update a flight path based on awareness of ground objects or conditions.
In some embodiments, a UAV is provided and comprises a plurality of sensors. Localization may be performed by the localization system by applying the sensor data of the UAV. Localization may also be performed by using a semantic map to place the UAV in a coordinate system of other semantic information. Perception may be performed by the perception system by applying the sensor data of the UAV. As a result of localization and perception, ground objects and conditions are identified. A flight path is generated or updated based on the ground objects and conditions.
In some embodiments, the UAV stores an emergency landing location. The emergency landing location may also be selected based on the locations of ground objects and conditions identified through localization and perception.
BRIEF DESCRIPTION OF THE DRAWINGS
The present disclosure will become better understood from the detailed description and the drawings, a brief summary of which is provided below.
FIG. 1 illustrates an exemplary environment in which systems herein may operate.
FIG. 2 illustrates an exemplary embodiment of a UAV.
FIG. 3 illustrates an exemplary embodiment of a computer system that may be used in some embodiments.
FIG. 4 illustrates an exemplary method for updating a flight path based on ground-awareness.
FIG. 5 illustrates an exemplary method that may be performed in some embodiments to generate an initial flight path and update the flight path during flight.
FIG. 6 illustrates an exemplary method for using a cost function and flight planner to generate a flight path.
FIG. 7 illustrates an exemplary flight path comprising a plurality of waypoints.
FIG. 8 illustrates an exemplary flight path that may be used for minimizing turns and optimizing for straight line travel.
FIG. 9 illustrates an exemplary flight path that may be used for surveying and mapping roads.
FIG. 10 illustrates an exemplary flight path that may be used for traffic monitoring.
FIG. 11 illustrates an exemplary flight path that may be used for goods delivery by the UAV
DETAILED DESCRIPTION
For simplicity and illustrative purposes, the principles of the present teachings are described by referring mainly to examples of various implementations thereof. However, one of ordinary skill in the art would readily recognize that the same principles are equally applicable to, and can be implemented in, all types of information and systems, and that any such variations do not depart from the true spirit and scope of the present teachings. Moreover, in the following detailed description, references are made to the accompanying figures, which illustrate specific examples of various implementations. Logical and structural changes can be made to the examples of the various implementations without departing from the spirit and scope of the present teachings. The following detailed description is, therefore, not to be taken in a limiting sense and the scope of the present teachings is defined by the appended claims and their equivalents.
In addition, it should be understood that steps of the examples of the methods set forth in the present disclosure can be performed in different orders than the order presented in the present disclosure. Furthermore, some steps of the examples of the methods can be performed in parallel rather than being performed sequentially. Also, the steps of the examples of the methods can be performed in a network environment in which some steps are performed by different computers in the networked environment.
Some implementations are implemented by a computer system. A computer system can include a processor, a memory, and a non-transitory computer-readable medium. The memory and non-transitory medium can store instructions for performing methods and steps described herein.
Embodiments herein relate to automatically creating flight paths for UAVs, and in particular, to automatically create flight paths based on ground-awareness. Ground-awareness relates to being aware of objects and conditions on the ground underneath or in the vicinity of the UAV. For example, a ground-aware UAV may identify a location of a road on the ground and maintain a buffer distance between the UAV and the road to comply with regulations about how close a UAV may fly within a road and also to maintain safety for the vehicles on the road. For example, if the UAV encounters problems and falls to the ground, it would not interfere with the vehicles on the road. In another example, a computerized flight planner may develop a flight path that prefers to travel over less populous areas instead of more populous areas. This can reduce the risk that a falling UAV will injure someone or damage property and also reduce the incidence of noise disruptions.
A ground-aware flight plan may be generated through the combination of a semantic map, localization system, and perception system. The semantic map provides a detailed map of the environment including semantic information in the map about objects and conditions on the ground. The localization system precisely localizes the UAV in the environment and may use any one or combination of global navigation satellite system (GNSS), global positioning system (GPS), inertial measurement unit (IMU), perception-based localization systems (e.g., image-based, LIDAR based, depth-sensor based, and so on), and sensor fusion of the aforementioned approaches. After the precise localization of the UAV, the UAV is aware of the ground objects and conditions in its vicinity and relation to the UAV based on the detailed semantic map.
The perception system allows the UAV to perceive the environment in real-time using one or more sensors and a computer system for interpreting the sensor inputs. The perception system may include one or more of a camera, video camera, light detection and ranging (LIDAR), depth sensor, ultra-sonic sensor, radar, and other sensors. The sensor data collected by the perception system may be used for localization as described herein. In addition, the sensor data of the perception system may allow the UAV to detect real-time conditions in the environment that would not be known from the semantic map. For example, the perception system may be used to detect recent changes to the environment, such as new buildings, roads, or street signs, or be used to detect the real-time presence of vehicles, people, and other moving objects. The perception system may also be used to detect whether vehicles and other objects are currently moving or stationary, and thus whether it is allowed to fly over them (when stationary) or not (when moving).
A computerized flight planner may use the information from the semantic map, localization system, and perception system to generate a flight path. The UAV may then navigate along the flight path. During the flight, new detections by the perception system may cause the UAV to detect the existence of ground conditions that may cause the flight path to be re-evaluated. The ground conditions may be input to the computerized flight planner to determine an updated flight path. In some embodiments, the updated flight path may comprise delaying the UAV, for example to wait for vehicles to stop at an intersection so that the UAV may safely pass over them. In some embodiments, the updated flight path may comprise changing the coordinates of the flight path, such as if an unexpected obstacle is detected.
In some embodiments, the computerized flight planner may apply a cost function to one or more ground objects or conditions, where the ground objects or conditions may be detected from the semantic map as a result of localization or from the perception system. The cost function may accept as input an identity of the ground object or event and output weights associated with the travel of the UAV over or in the vicinity of the ground object or event. The weights may be associated with the cost of the UAV to fly along a path that would cross over or in the vicinity of the ground object or event. In other embodiments, the cost function may accept as input the raw sensor readings or semantic map data and output the weights associated with the travel of the UAV along a path over or in the vicinity of a region. Although referred to in the singular as a cost function, it should be understood that a plurality of cost functions may be used, such as for different types of ground objects and conditions. The computerized flight planner may use the weights to compute a lowest cost flight path using pathfinding algorithms such as rapidly exploring random tree algorithm (RRT), A* algorithm, Dijkstra's algorithm, D* algorithm, any-angle path finding, hierarchical path finding, and other algorithms.
In some embodiments, the ground objects or conditions detected from the semantic map or perceptions system may be impassable by the UAV, such as a tall tree or a region that cannot be passed due to government regulations. The impassable ground objects or conditions may be represented as an impassable barrier in the computerized flight planner. It may generate a path using the pathfinding algorithm by using a combination of weights generated by the cost function and the impassable obstacles. The pathfinding algorithm may generate paths based on both the path weights and the obstacles, and a lowest cost flight path may be select.
In some embodiments, the flight paths generated by the computerized flight planner are represented by a plurality of waypoints. The waypoints may represent intermediate steps on the flight path. In some embodiments, the flight path comprises flying in a straight line or substantially straight line between waypoints, so that any turns are represented by adding a new waypoint to the flight path. The UAV may deviate from the straight line path between waypoints when an obstacle or ground object or condition would require it or render a new path more efficient.
The terms âunmanned aerial vehicleâ and âUAVâ may refer to an aerial vehicle without a physically-present human operator. The terms âdrone,â âunmanned aerial vehicle systemâ (UAVS), or âunmanned aerial systemâ (UAS) may also be used to refer to a UAV.
UAVs may operate autonomously, partially autonomously, or by remote control of a human operator. An autonomous UAV may automatically develop a flight path and navigate along the flight path through a computer processor that operates one or more propulsion components and control components. In some embodiments, the autonomous UAV may require a manually developed flight path but may navigate automatically along the flight path without human control or intervention. In some embodiments, an autonomous UAV is supervised by a human operator, who can take over control if necessary, even though control is by default performed by a computer processor. A remote-control UAV may be under the control of a human operator who is remote from the UAV. The human operator may control the UAV through a control interface. Control commands may be received from the human operator at the control interface and transmitted, through wireless or wired communication, to the UAV. One or more propulsion components and control components may be controlled through operation of the human-operated control interface. Moreover, the UAV may record video, photo, and sensor data to transmit back to the human operator to allow the human operator to perceive the vicinity of the UAV. A partially autonomous UAV may include both autonomous and remote-control aspects. In one embodiment, the autonomous and remote-control commands may occur at different levels of abstraction. For example, a human operator may input commands for the UAV to travel from a start location to an end location, and an autonomous piloting system may automatically perform the low-level navigation tasks for controlling the propulsion and control systems of the UAV to fly the UAV from the start location to the end location. In such an embodiment, the human may provide high-level control and the UAV may autonomously perform low-level control. Vice versa, an autonomous UAV may perform high-level control in the form of autonomously developing a flight path and handing off the low-level control to the human-operator to perform the individual real-time control necessary to guide the UAV along the flight path. In other embodiments of a partially autonomous UAV, the split of control between the autonomous and remote-control aspect may be at the same level of abstraction. For example, the UAV may be flown in an autonomous mode until an obstacle or other difficult to navigate situation is encountered, when control is switched to remote-control by a human operator.
While embodiments herein are illustrated with reference to UAVs, the techniques herein may also be applied to ground vehicles. For example, techniques herein may be used for path planning, navigation, and operation of unmanned ground vehicles (UGVs).
FIG. 1 illustrates an exemplary environment 100 in which systems herein may operate. A UAV 101 may fly in the air above the ground 110 . The UAV may include a camera 102 and sensors 103 directed at the ground to collect photos, videos, and other sensor data indicating objects and conditions on the ground. The term âground eventsâ refers to either objects or conditions on the ground. Ground conditions may be either temporary, permanent, or semi-permanent and may refer to not just objects but also situations, such as whether vehicles are moving or stationary or whether there is traffic on a road. The term ground events refers to any sort of object, condition, or data about an occurrence on the ground. Through the use of camera 102 and sensors 103 the UAV gains awareness of ground events.
The UAV may travel over any sort of terrain, such as populated or unpopulated, urban or rural, and so on. Objects on the ground may include a road 111 and vehicle 112 . Other objects 113 may also be on the ground in the vicinity of the UAV, such as people, trees, vegetation, buildings, structures, road signs, pathways, animals, sidewalks, lawns, hills, mountains, natural formations, water, streams, canals, rivers, oceans, and so on.
A UAV may be of various forms. For example, a UAV may be a rotorcraft such as a helicopter or multicopter, a fixed-wing aircraft, a jet aircraft, a ducted fan aircraft, a lighter-than-air dirigible such as a blimp or steerable balloon, a tail-sitter aircraft, a glider aircraft, an ornithopter, and so on.
In one embodiment, a UAV is a rotorcraft. A rotorcraft includes helicopters, which typically include two rotors, and multicopters, which have more than two rotors. In a rotorcraft, the rotors provide propulsion and control for the vehicle. Each rotor includes blades attached to a motor, and the rotors may allow the rotorcraft to take off and land vertically, to maneuver in any direction, and to hover. The pitch of the blades may be adjusted as a group or differentially to allow the rotorcraft to perform aerial maneuvers. Additionally, the rotorcraft may propel and maneuver itself by adjusting the rotation rate of the motors, collectively or differentially.
In one embodiment, a UAV is a tail-sitter UAV. A tail-sitter UAV may comprise fixed wings for providing lift and allowing the UAV to glide horizontally. However, during launch the tail-sitter UAV may be positioned vertically with fins and wings resting on the ground and stabilizing the UAV in a vertical position. The tail-sitter UAV may take off by operating propellers to generate upward thrust. In the air, the tail-sitter UAV may use one or more flaps to turn itself into a horizontal position. The propellers may provide forward thrust so that the tail-sitter UAV may fly in a similar manner as a typical airplane.
In one embodiment, the UAV is a fixed-wing aircraft, which may also be referred to as an airplane, aeroplane, or a plane. A fixed-wing aircraft may comprise a fuselage and stationary wings that generate lift based on the wing shape and the vehicle's forward airspeed. In a common configuration, a fixed-wing UAV includes two horizontal wings, a vertical stabilizer (also referred to as a fin) to stabilize the plane's yaw, a horizontal stabilizer (also referred to as an elevator or tailplane) to stabilize pitch (tilt up or down), and a propulsion unit. The propulsion unit may include, for example, a motor, shaft, and propeller, or a jet engine.
The aforementioned embodiments are exemplary only and the UAV may take any number of other forms.
FIG. 2 illustrates an exemplary embodiment of a UAV 101 . UAV 101 may comprise a processor 207 and data storage 208 , including one or more program instructions 212 , in addition to sensor systems, a communication system 205 , and power system 206 .
IMU 201 comprise components for determining the orientation, position, and movement of the UAV. The IMU 201 may comprise an accelerometer and gyroscope, where the accelerometer may measure the orientation of the vehicle with respect to the earth and the gyroscope measures the rate of rotation around an axis. The IMU 201 may optionally include other sensors such as magnetometers and pressure sensors. A magnetometer may measure direction by using an electronic compass to determine heading information. A pressure sensor may be used to determine the altitude of the UAV.
Imaging system 202 may comprise components for imaging the environment in the vicinity of the UAV. In an embodiment, the imaging system 202 comprises a red, green, and blue (RGB) camera. An RGB camera may capture photographic and video imagery in the visible spectrum of RGB light. Imaging system 202 may optionally include other imaging components such as an infra-red camera for capturing light in the infra-red spectrum or a depth sensor for capturing depth information in an image. The imaging system 202 may comprise a still camera, a video camera, or both. The imaging system 202 may be used for object detection, localization, mapping, and other applications.
GNSS receiver 203 may communicate with satellites to provide coordinates of the UAV. In one example, the GNSS receiver 203 is a GPS receiver where GPS is one example of a GNSS system. A GPS receiver may provide GPS coordinates of the UAV. GPS coordinates may have a relatively high margin of error and so additional sensor systems may be used in conjunction with GPS to increase the accuracy of localization of the UAV.
LIDAR 204 may comprise an emitter that generates pulsed laser light and a detector for receiving the reflected pulses. Differences in laser return times and wave lengths may be used to generate a 3D point cloud comprising location information in 3D space and laser reflection intensities. The 3D point cloud may be processed to build a map of the 3D environment, including both topography and objects.
Communication system 205 may comprise one or more wireless interfaces or wirelines interfaces to enable the UAV to communicate via one or more networks. Wireless interfaces may enable communication over one or more wireless communication protocols, such as Bluetooth, Wi-Fi, Long-Term Evolution (LTE), WiMAX, radio-frequency ID (RFID), near-field communication (NFC), and other wireless communication protocols. Wireline interfaces may include interfaces to wired networks such as Ethernet, universal serial bus (USB), or other wired networks such as coaxial cable, optical link, fiber-optic link, and so on. Communication system 205 may enable the receiving of remote-control commands from a human operator. Communication system 205 may also enable the sending of sensor data from the UAV to remotely located computer systems for processing, storage, or display.
Power system 206 may comprise components for providing power to the UAV. In an embodiment, the power system 206 may comprise one or more batteries. In other embodiments, the power system 206 may comprise solid or liquid fuel.
Processor 207 may comprise a computer processor for executing one or more program instructions 212 on the data storage 208 . The processor may be a general-purpose processor or a special purpose processor (e.g., digital signal processors, application specific integrated circuits, and so on). The processor may be configured to execute program constructions to provide the functionality of a UAV described herein.
Data storage 208 may comprise any form of computer-readable storage that can be read or accessed by processor 207 . The data storage may be integrated with or separate from the processor 207 . Data storage may be temporary, permanent, or semi-permanent and may comprise, for example, RAM, ROM, optical media, flash memory, hard disk, solid state drives (SSD), mechanical hard drives, or other storage. While illustrated as a single data storage 208 , it should be understood that data storage 208 may comprise any number of separate or integrated data storages.
The data storage 208 may store one or more program instructions 212 for implementing the functionality described herein. Navigation system 213 may be stored as program instructions stored in the data storage 208 . The navigation system 213 may comprise instructions for moving and maneuvering the UAV by issuing instructions to the propulsion components and control components of the UAV. Service system 214 may comprise program instructions for providing a service function of the UAV. For example, services provided by the UAV may include mapping, traffic monitoring, delivery services, or emergency response. The service system 214 may comprise the instructions for providing the desired service.
In some embodiments, UAV 101 may comprise a payload 220 , such as when the service performed by the UAV 101 require delivery or use of one or more payload objects. For example, in a delivery task, the payload 220 may comprise one or more payload objects for the UAV 101 to deliver to a recipient.
UAV 101 may include additional components not illustrated in FIG. 2 . For example, UAV 101 may include a plurality of additional sensors such as radar, ultra-sonic sensors, proximity sensors, temperature sensors, light sensors, microphones, and so on. UAV 101 may also include output systems such as speakers, lights, display screens, and so on.
FIG. 3 illustrates an exemplary embodiment of a computer system 301 that may be used in some embodiments to perform functionality described herein. The computer system 301 may implement the flight planner 315 for generating a ground-aware flight path for UAV 101 as described elsewhere herein. The flight planner 315 may develop the ground-aware flight path by making use of semantic map 311 , localization system 312 , and perception system 313 in addition to a cost function 314 .
In some embodiments, the computer system 301 is onboard the UAV 101 . For example, in one embodiment, the processor 302 is the processor 207 , the communication system 303 is the communication system 205 , and the data storage 310 is the data storage 208 . In such embodiments, the UAV's onboard computer system performs the functionality of computing a ground-aware flight path using the flight planner 315 . The semantic map 311 , localization system 312 , perception system 313 , cost function 314 , flight planner 315 , and emergency landing location 330 may all be onboard the UAV 101 in its data storage 208 .
In other embodiments, the computer system 301 may be offboard the UAV 101 and may compute the ground-aware flight path remotely from the UAV 101 and transmit the resulting flight path, or control instructions to implement the flight path to the UAV 101 . For example, the computer system 301 may be a ground station computer system. For example, the computer system 301 may receive communications of sensor data from the UAV 101 via the communication system 303 and compute the flight path for transmission to the UAV 101 . In some examples, the flight path is computed remotely from the UAV 101 and displayed to a human operator who remotely controls the UAV 101 to implement the flight path. In such an example, the entire flight path does not need to be transmitted to the UAV 101 and only individual control instructions to implement the flight path are transmitted to the UAV 101 . In other embodiments, the UAV 101 may be autonomous and an offboard computer system may compute a flight path and automatically transmit control instructions to the UAV 101 to be implemented by the navigation system 213 .
The processor 302 may comprise a computer processor for executing one or more program instructions 320 on the data storage 310 . The processor may be a general-purpose processor or a special purpose processor (e.g., digital signal processors, application specific integrated circuits, and so on). The processor may be configured to execute program constructions to provide the functionality of ground-aware flight planning as described herein.
Communication system 303 may comprise one or more wireless interfaces or wirelines interfaces to enable the computer system 301 to communicate via one or more networks. Wireless interfaces may enable communication over one or more wireless communication protocols, such as Bluetooth, Wi-Fi, Long-Term Evolution (LTE), WiMAX, radio-frequency ID (RFID), near-field communication (NFC), and other wireless communication protocols. Wireline interfaces may include interfaces to wired networks such as Ethernet, universal serial bus (USB), or other wired networks such as coaxial cable, optical link, fiber-optic link, and so on. When the computer system 301 is offboard of the UAV 101 , the communication system 303 may enable the receiving of sensor data from the UAV 101 . Moreover, communication system 303 may also enable the sending of remote control instructions, or an entire or partial flight path, to the UAV 101 .
The data storage 310 may store one or more program instructions 320 and data 330 for implementing the functionality described herein.
Semantic map 311 may comprise a detailed map of an environment in which the UAV 101 operates or may operate. The semantic map may include 2D and/or 3D data, including information about topography and structures. The semantic map may comprise semantic information about ground events (comprising ground objects and conditions) that are temporary, permanent, or semi-permanent. Each ground event may comprise a name, description, type, and coordinates (X, Y, Z) of the event's location. The name of the ground event may identify it and the type may categorize the ground event into a general type. The description of the ground event may provide further information about the ground event.
Localization system 312 may comprise computer instructions for localizing UAV 101 . Localization may be performed based on the IMU, GNSS, image-based localization, LIDAR-based localization, other methods, and any combination of the foregoing.
Perception system 313 may comprise computer instructions for performing perception in the UAV 101 based on the sensor inputs from sensors including imaging system 202 , LIDAR 204 , or other sensors. Perception system may comprise an object detection system for detecting objects in the environment.
In one embodiment, perception system may use computer vision algorithms for recognizing objects and ground features. For example, in one embodiment an image or video may be input to a multi-layer neural network to generate output identifying the location and identity of one or more objects. The multi-layer neural network may include neural network layers of different types such as convolutional neural network layers and fully-connected neural network layers. The neural network may be trained on training examples comprising an image and one or more training labels that identify the objects in the image and their location. In some embodiments, semantic segmentation may be performed to identify not just the location of objects but to semantically segment the image to assign individual pixels to objects. As a result of semantic segmentation, regions of pixels may be identified corresponding to each object in the image.
In some embodiments, object detection may be performed by using feature extraction to identify specific features of an image prior to using a machine learning model. Feature extraction may comprise applying an interest point detector to identify points of interest in an image. In some embodiments, an affine invariant interest point detector, such as Scale-Invariant Feature Transform (SIFT), may be applied. An affine invariant interest point detector is relatively invariant to affine deformations to an image. After the interest point detector is applied, a descriptor generator may be run on the interest points to generate an image descriptor (or feature vector) for each interest point. Descriptor generators may be partially or fully invariant to intensity and contrast changes and geometric deformations. For example, a SIFT descriptor may be used in some embodiments. Once image descriptors have been computed at each interest point, the resulting image descriptors may be input to a machine learning model to perform detection of objects. Nonetheless, feature extraction is optional and many deep learning machine learning models do not include this step and accept the image itself, or a pre-processed version of the image, as an input for object detection. The aforementioned object detection methods may also be used to detect ground features for use in localizing the UAV by localization system 312 . Recognizable ground objects may be detected like buildings, trees, signs, and topographical features to help localize the UAV by comparing the ground objects detected by the perception system with known ground features in the semantic map 311 .
Cost function 314 may accept as input an identity of a ground event, such as a ground object or ground condition, and output a weight of a path for flying over or in the vicinity of the ground event. In some embodiments, the cost function may output different weights for flying over the ground event and flying in the vicinity of the ground event. Weights may be numerical values such as integers, real numbers, floating point numbers, Boolean values, and so on. For example, the weight for a path that flies directly over a person may be very high because it can be dangerous to do so in the event that the UAV falls to the ground, while the weight for a path that flies near a person may be somewhat lower because it is not as dangerous as flying over the person.
In some embodiments, the cost function 314 may identify certain paths as completely impassable. For example, in some embodiments, it may be entirely prohibited to fly over a person. In such an embodiment, the cost function 314 may output the location of impassable obstacles. Although illustrated as a single cost function 315 , it should be understood that a plurality of cost functions may be used.
Flight planner 315 may comprise a computer program for generating a flight path. The flight planner 315 may accept as input the weights of different paths or regions output by the cost function 314 . The flight planner 315 may apply those weights to a pathfinding algorithm to determine a lowest cost path. If the cost function 314 has identified impassable obstacles, those obstacles may also be input to and applied by the <figure-callout id="315" label="flight planner" filenames="US20200258400A1-2020081
CLAIMS
Claims ( 20 )
What is claimed is:
1 . A computer-implemented method for operating an unmanned aerial vehicle (UAV) comprising:
receiving a semantic map for a geographic area over which the UAV is deployable, the semantic map comprising data about a first set of ground events; providing instructions to a control system to navigate a UAV on a flight path; receiving, by the UAV, input sensor data from sensors on the UAV, the sensors directed toward the ground; localizing, by a localization system, the UAV based on GPS and the input sensor data; perceiving, by a perception system, real-time data about the ground in the vicinity of the UAV based on the input sensor data and detecting a second set of ground events; updating the flight path, by a flight planner, based on the first and second sets of ground events; providing instructions to the control system to navigate the UAV on the updated flight path.
2 . The computer-implemented method of claim 1 , further comprising localizing the UAV based on imaging data from an imaging system.
3 . The computer-implemented method of claim 1 , further comprising optimizing the flight path, by the flight planner, to minimize the number of turns in the updated flight path.
4 . The computer-implemented method of claim 1 , further comprising optimizing the flight path, by the flight planner, to maintain a threshold distance between the UAV and a plurality of roads.
5 . The computer-implemented method of claim 1 , further comprising optimizing the flight path, by the flight planner, to cause the UAV to wait at a road intersection until a plurality of vehicles are stopped before passing over the road intersection.
6 . The computer-implemented method of claim 1 , further comprising selecting an emergency landing location, by the flight planner, based on the first and second set of ground events.
7 . The computer-implemented method of claim 1 , further comprising:
storing a pilot position; detecting a ground obstacle and determining that the ground obstacle blocks line of sight from the pilot position to a position on the flight path; updating the flight path so that the flight path is in line of sight from the pilot position.
8 . The computer-implemented method of claim 1 , further comprising:
detecting a gap in the semantic map; collecting data from one or more sensors to fill the gap; updating the semantic map based on the collected data.
9 . The computer-implemented method of claim 1 , further comprising measuring the flow of vehicles on a road and optimizing the flight path, by the flight planner, to cause the UAV to wait until the flow of vehicles has stopped before passing over the road.
10 . A computerized flight planner and operations system for an unmanned aerial vehicle (UAV) comprising:
a processor; a non-transitory computer-readable medium comprising:
a semantic map for a geographic area over which the UAV is deployable, the semantic map comprising data about a first set of ground events;
instructions for a control system to navigate a UAV on a flight path;
instructions for receiving input sensor data from sensors on the UAV, the sensors directed toward the ground;
a localization system for localizing the UAV based on GPS and the input sensor data;
a perception system for perceiving real-time data about the ground in the vicinity of the UAV based on the input sensor data and detecting a second set of ground events;
a flight planner for updating the flight path based on the first and second sets of ground events;
instructions for the control system to navigate the UAV on the updated flight path.
11 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the localization system is configured to localize the UAV based on imaging data from an imaging system.
12 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is configured to minimize the number of turns in the updated flight path.
13 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is configured to maintain a threshold distance between the UAV and a plurality of roads.
14 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is configured to cause the UAV to wait at a road intersection until a plurality of vehicles are stopped before passing over the road intersection.
15 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is configured to select an emergency landing location based on the first and second sets of ground events.
16 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is further configured to:
store a pilot position; detect a ground obstacle and determine that the ground obstacle blocks line of sight from the pilot position to a position on the flight path; update the flight path so that the flight path is in line of sight from the pilot position.
17 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is further configured to:
detect a gap in the semantic map; collect data from one or more sensors to fill the gap; update the semantic map based on the collected data.
18 . The computerized flight planner and operations system for a UAV of claim 10 , wherein the flight planner is further configured to measure the flow of vehicles on a road and optimize the flight path to cause the UAV to wait until the flow of vehicles has stopped before passing over the road.
19 . A computer-implemented method for operating an unmanned aerial vehicle (UAV) comprising:
receiving a semantic map for a geographic area over which the UAV is deployable, the semantic map comprising data about a first set of ground events; providing instructions to a control system to navigate a UAV on a flight path; receiving, by the UAV, input sensor data from sensors on the UAV, the sensors directed toward the ground; localizing, by a localization system, the UAV based on GPS and the input sensor data; perceiving, by a perception system, real-time data about the ground in the vicinity of the UAV based on the input sensor data and detecting a second set of ground events; computing, by a cost function, a plurality of weights associated with the first and second sets of ground events based on identifiers of the ground events; updating the flight path, by a flight planner, based on the plurality of weights; providing instructions to the control system to navigate the UAV on the updated flight path.
20 . The computer-implemented method of claim 19 , further comprising:
computing, by the cost function, a plurality of weights associated with a set of candidate flight paths; applying, by the flight planner, the weights associated with the set of candidate flight paths to minimize the number of turns in the updated flight path.
US16/567,067
2019-02-13
2019-09-11
Ground-aware uav flight planning and operation system
Abandoned
US20200258400A1
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US16/567,067
US20200258400A1
( en )
2019-02-13
2019-09-11
Ground-aware uav flight planning and operation system
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US201962805185P
2019-02-13
2019-02-13
US16/567,067
US20200258400A1
( en )
2019-02-13
2019-09-11
Ground-aware uav flight planning and operation system
Publications (1)
Publication Number
Publication Date
US20200258400A1
true
US20200258400A1 ( en )
2020-08-13
Family
ID=71945255
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/567,067
Abandoned
US20200258400A1
( en )
2019-02-13
2019-09-11
Ground-aware uav flight planning and operation system
Country Status (1)
Country
Link
US
( 1 )
US20200258400A1
( en )
Cited By (45)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20200279494A1
( en )
*
2019-02-28
2020-09-03
Rockwell Collins, Inc.
Autonomous Aircraft Sensor-Based Positioning and Navigation System Using Markers
US20210110444A1
( en )
*
2019-10-09
2021-04-15
The Boeing Company
Flight route options determination systems and methods
CN112947594A
( en )
*
2021-04-07
2021-06-11
ä¸å大å¦
Unmanned aerial vehicle-oriented flight path planning method
CN113220000A
( en )
*
2021-05-11
2021-08-06
åä¸ç§æ大å¦
Unmanned ship path tracking preset performance control method and system for underwater detection operation
US20220024582A1
( en )
*
2019-08-20
2022-01-27
Rakuten Group,Inc.
Information processing system, information processing device, and information processing method
US11250711B1
( en )
*
2020-08-04
2022-02-15
Rockwell Collins, Inc.
Maneuver evaluation and route guidance through environment
US11255713B2
( en )
*
2020-06-04
2022-02-22
Zhejiang University
Device and method for measuring amount of liquid chemical in plant protection unmanned aerial vehicle (UAV)
US20220058573A1
( en )
*
2020-04-24
2022-02-24
The Trustees Of Indiana University
Aerial drone operating system and transportation network infrastructure
US20220083920A1
( en )
*
2020-09-14
2022-03-17
Ge Aviation Systems Llc
Systems and methods for providing conformance volume based airspace access
US20220100210A1
( en )
*
2020-09-30
2022-03-31
Toyota Jidosha Kabushiki Kaisha
Control method for unmanned aircraft, server, and unmanned aircraft
US11348471B2
( en )
*
2016-06-10
2022-05-31
Metal Raptor, Llc
Drone air traffic control over wireless networks for package pickup and delivery in an order defined by roads, highways, or streets
US20220198941A1
( en )
*
2020-11-11
2022-06-23
Georgia Tech Research Corporation
Lightweight Flight Control System for Miniature Indoor Aerial Robots
WO2022164832A1
( en )
*
2021-01-29
2022-08-04
Boston Dynamics, Inc.
Semantic models for robot autonomy on dynamic sites
US11417224B1
( en )
2021-08-19
2022-08-16
Beta Air, Llc
System and method for pilot assistance in an electric aircraft
US20220283584A1
( en )
*
2019-07-19
2022-09-08
Sony Group Corporation
Information processing device, information processing method, and information processing program
CN115146933A
( en )
*
2022-06-20
2022-10-04
æ·±å³å¸åå¸å ¬å ±å®å ¨ææ¯ç 究é¢æéå ¬å¸
Method, system, equipment and storage medium for dealing with explosion accident of hazardous source
US20220380043A1
( en )
*
2021-05-25
2022-12-01
Hydronalix, Inc.
Unmanned aerial vehicle with underwater sonar scanning capability
EP4102332A1
( en )
*
2021-06-11
2022-12-14
Spleenlab GmbH
Method for controlling a flight movement of an aircraft for landing or discarding a load and an aircraft
US20220398932A1
( en )
*
2021-06-09
2022-12-15
Ford Global Technologies, Llc
Systems And Methods For Operating Drone Flights Over Public Roadways
US20220415186A1
( en )
*
2021-06-29
2022-12-29
Hyundai Motor Company
Method for Controlling Operation of Aerial Vehicle and Apparatus for the Same
US11561557B1
( en )
*
2021-07-23
2023-01-24
Beta Air, Llc
System and method for initiating a command of an electric vertical take-off and landing (EVTOL) aircraft
US11579611B1
( en )
2020-03-30
2023-02-14
Amazon Technologies, Inc.
Predicting localized population densities for generating flight routes
US20230092896A1
( en )
*
2019-06-05
2023-03-23
Gal Zuckerman
Iteratively mapping-and-approaching an urban area
US20230122535A1
( en )
*
2021-10-19
2023-04-20
Wing Aviation Llc
Processes for Generating and Updating Flyable Airspace for Unmanned Aerial Vehicles
US11640764B1
( en )
2020-06-01
2023-05-02
Amazon Technologies, Inc.
Optimal occupancy distribution for route or path planning
US20230222618A1
( en )
*
2020-07-06
2023-07-13
Beijing Jingdong Qianshi Technology Co., Ltd.
Object detection method and device
CN116880329A
( en )
*
2023-08-17
2023-10-13
é西ç§æ大å¦
Intelligent autonomous landing control system and method for flapping-wing aircraft based on active disturbance rejection
US20230399124A1
( en )
*
2020-10-28
2023-12-14
The Regents Of The University Of California
Design methods and motion control algorithms for impact-resilient mobile robots
US20230409041A1
( en )
*
2020-11-18
2023-12-21
Interdigital Ce Patent Holdings, Sas
Method and apparatus for obtaining observation data of an environment
US11868145B1
( en )
*
2019-09-27
2024-01-09
Amazon Technologies, Inc.
Selecting safe flight routes based on localized population densities and ground conditions
US20240192695A1
( en )
*
2022-12-07
2024-06-13
Boston Dynamics, Inc.
Anchoring based transformation for aligning sensor data of a robot with a site model
US20240201697A1
( en )
*
2022-12-19
2024-06-20
International Business Machines Corporation
Unmanned aerial vehicle deployment parameter modification
US12148206B2
( en )
2022-03-31
2024-11-19
Wing Aviation Llc
Method for controlling an unmanned aerial vehicle to avoid obstacles
WO2024258415A1
( en )
*
2023-06-16
2024-12-19
Siemens Corporation
System and method for automated path planning and travel for autonomous vehicles
CN119272421A
( en )
*
2024-12-10
2025-01-07
ç¦å»ºç¦ç¿æºç§ææéå ¬å¸
A simulation deformation aircraft and control method
US12222723B2
( en )
2021-06-04
2025-02-11
Boston Dynamics, Inc.
Directed exploration for navigation in dynamic environments
US12304082B2
( en )
2021-06-04
2025-05-20
Boston Dynamics, Inc.
Alternate route finding for waypoint-based navigation maps
US12399508B2
( en )
2022-11-17
2025-08-26
Wing Aviation Llc
Automatic selection of delivery zones using survey flight 3D scene reconstructions
US12442640B2
( en )
2019-08-06
2025-10-14
Boston Dynamics, Inc.
Intermediate waypoint generator
US12449822B2
( en )
2022-06-23
2025-10-21
Boston Dynamics, Inc.
Ground clutter avoidance for a mobile robot
US12461531B2
( en )
2021-06-04
2025-11-04
Boston Dynamics, Inc.
Topology processing for waypoint-based navigation maps
US12564955B2
( en )
2022-06-23
2026-03-03
Boston Dynamics, Inc.
Modeling robot self-occlusion for localization
US12602924B2
( en )
2022-03-31
2026-04-14
Wing Aviation Llc
Method for semantic localization of an unmanned aerial vehicle
US12608931B2
( en )
2024-04-19
2026-04-21
Wing Aviation Llc
Generating aerial paths based on properties of aerial image data
US20260126793A1
( en )
*
2020-11-11
2026-05-07
Georgia Tech Research Corporation
Lightweight Flight Control System for Miniature Indoor Aerial Robots
2019
2019-09-11
US
US16/567,067
patent/US20200258400A1/en
not_active
Abandoned
Cited By (64)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11348471B2
( en )
*
2016-06-10
2022-05-31
Metal Raptor, Llc
Drone air traffic control over wireless networks for package pickup and delivery in an order defined by roads, highways, or streets
US11532237B2
( en )
*
2019-02-28
2022-12-20
Rockwell Collins, Inc.
Autonomous aircraft sensor-based positioning and navigation system using markers
US20200279494A1
( en )
*
2019-02-28
2020-09-03
Rockwell Collins, Inc.
Autonomous Aircraft Sensor-Based Positioning and Navigation System Using Markers
US20230092896A1
( en )
*
2019-06-05
2023-03-23
Gal Zuckerman
Iteratively mapping-and-approaching an urban area
US20220283584A1
( en )
*
2019-07-19
2022-09-08
Sony Group Corporation
Information processing device, information processing method, and information processing program
US12442640B2
( en )
2019-08-06
2025-10-14
Boston Dynamics, Inc.
Intermediate waypoint generator
US20220024582A1
( en )
*
2019-08-20
2022-01-27
Rakuten Group,Inc.
Information processing system, information processing device, and information processing method
US12116126B2
( en )
*
2019-08-20
2024-10-15
Rakuten Group, Inc.
Information processing system, information processing device, and information processing method
US11868145B1
( en )
*
2019-09-27
2024-01-09
Amazon Technologies, Inc.
Selecting safe flight routes based on localized population densities and ground conditions
US20210110444A1
( en )
*
2019-10-09
2021-04-15
The Boeing Company
Flight route options determination systems and methods
US11579611B1
( en )
2020-03-30
2023-02-14
Amazon Technologies, Inc.
Predicting localized population densities for generating flight routes
US20220058573A1
( en )
*
2020-04-24
2022-02-24
The Trustees Of Indiana University
Aerial drone operating system and transportation network infrastructure
US11853953B2
( en )
*
2020-04-24
2023-12-26
The Trustees Of Indiana University
Methods and systems providing aerial transport network infrastructures for unmanned aerial vehicles
US11640764B1
( en )
2020-06-01
2023-05-02
Amazon Technologies, Inc.
Optimal occupancy distribution for route or path planning
US11255713B2
( en )
*
2020-06-04
2022-02-22
Zhejiang University
Device and method for measuring amount of liquid chemical in plant protection unmanned aerial vehicle (UAV)
US12387285B2
( en )
*
2020-07-06
2025-08-12
Beijing Jingdong Qianshi Technology Co., Ltd.
Object detection method and device
US20230222618A1
( en )
*
2020-07-06
2023-07-13
Beijing Jingdong Qianshi Technology Co., Ltd.
Object detection method and device
US11250711B1
( en )
*
2020-08-04
2022-02-15
Rockwell Collins, Inc.
Maneuver evaluation and route guidance through environment
US20220083920A1
( en )
*
2020-09-14
2022-03-17
Ge Aviation Systems Llc
Systems and methods for providing conformance volume based airspace access
CN114333425A
( en )
*
2020-09-30
2022-04-12
丰ç°èªå¨è½¦æ ªå¼ä¼ç¤¾
Control method of unmanned aerial vehicle, server and unmanned aerial vehicle
US20220100210A1
( en )
*
2020-09-30
2022-03-31
Toyota Jidosha Kabushiki Kaisha
Control method for unmanned aircraft, server, and unmanned aircraft
US20230399124A1
( en )
*
2020-10-28
2023-12-14
The Regents Of The University Of California
Design methods and motion control algorithms for impact-resilient mobile robots
US20260126793A1
( en )
*
2020-11-11
2026-05-07
Georgia Tech Research Corporation
Lightweight Flight Control System for Miniature Indoor Aerial Robots
US20220198941A1
( en )
*
2020-11-11
2022-06-23
Georgia Tech Research Corporation
Lightweight Flight Control System for Miniature Indoor Aerial Robots
US12591242B2
( en )
*
2020-11-18
2026-03-31
Interdigital Ce Patent Holdings, Sas
Method and apparatus for obtaining observation data of an environment
US20230409041A1
( en )
*
2020-11-18
2023-12-21
Interdigital Ce Patent Holdings, Sas
Method and apparatus for obtaining observation data of an environment
US12321178B2
( en )
2021-01-29
2025-06-03
Boston Dynamics, Inc.
Semantic models for robot autonomy on dynamic sites
WO2022164832A1
( en )
*
2021-01-29
2022-08-04
Boston Dynamics, Inc.
Semantic models for robot autonomy on dynamic sites
CN112947594A
( en )
*
2021-04-07
2021-06-11
ä¸å大å¦
Unmanned aerial vehicle-oriented flight path planning method
CN113220000A
( en )
*
2021-05-11
2021-08-06
åä¸ç§æ大å¦
Unmanned ship path tracking preset performance control method and system for underwater detection operation
US20220380043A1
( en )
*
2021-05-25
2022-12-01
Hydronalix, Inc.
Unmanned aerial vehicle with underwater sonar scanning capability
US12560714B2
( en )
*
2021-05-25
2026-02-24
Hydronalix, Inc.
Unmanned aerial vehicle with underwater sonar scanning capability
US12222723B2
( en )
2021-06-04
2025-02-11
Boston Dynamics, Inc.
Directed exploration for navigation in dynamic environments
US12304082B2
( en )
2021-06-04
2025-05-20
Boston Dynamics, Inc.
Alternate route finding for waypoint-based navigation maps
US12461531B2
( en )
2021-06-04
2025-11-04
Boston Dynamics, Inc.
Topology processing for waypoint-based navigation maps
US20220398932A1
( en )
*
2021-06-09
2022-12-15
Ford Global Technologies, Llc
Systems And Methods For Operating Drone Flights Over Public Roadways
US11955020B2
( en )
*
2021-06-09
2024-04-09
Ford Global Technologies, Llc
Systems and methods for operating drone flights over public roadways
US12153443B2
( en )
*
2021-06-11
2024-11-26
Spleenlab GmbH
Method for controlling a flight movement of an aerial vehicle for landing or for dropping a cargo, and aerial vehicle
US12372972B2
( en )
*
2021-06-11
2025-07-29
Spleenlab GmbH
Method for controlling a flight movement of an aerial vehicle for landing or for dropping a cargo, and aerial vehicle
EP4102332A1
( en )
*
2021-06-11
2022-12-14
Spleenlab GmbH
Method for controlling a flight movement of an aircraft for landing or discarding a load and an aircraft
US20220397913A1
( en )
*
2021-06-11
2022-12-15
Spleenlab GmbH
Method for Controlling a Flight Movement of an Aerial Vehicle for Landing or for Dropping a Cargo, and Aerial Vehicle
US20220415186A1
( en )
*
2021-06-29
2022-12-29
Hyundai Motor Company
Method for Controlling Operation of Aerial Vehicle and Apparatus for the Same
US20230033178A1
( en )
*
2021-07-23
2023-02-02
Beta Air, Llc
System and method for initiating a command of an electric vertical take-off and landing (evtol) aircraft
US12474715B2
( en )
*
2021-07-23
2025-11-18
Beta Air Llc
System and method for initiating a command of an electric aircraft
US11561557B1
( en )
*
2021-07-23
2023-01-24
Beta Air, Llc
System and method for initiating a command of an electric vertical take-off and landing (EVTOL) aircraft
US20230168691A1
( en )
*
2021-07-23
2023-06-01
Beta Air, Llc
System and method for initating a command of an electric vertical take-off and landing (evtol) aircraft
US12315375B2
( en )
2021-08-19
2025-05-27
Beta Air Llc
System and method for pilot assistance in an electric aircraft
US11417224B1
( en )
2021-08-19
2022-08-16
Beta Air, Llc
System and method for pilot assistance in an electric aircraft
WO2024049484A3
( en )
*
2021-10-19
2024-05-10
Wing Aviation Llc
Processes for generating and updating flyable airspace for unmanned aerial vehicles
US20230122535A1
( en )
*
2021-10-19
2023-04-20
Wing Aviation Llc
Processes for Generating and Updating Flyable Airspace for Unmanned Aerial Vehicles
US12492019B2
( en )
*
2021-10-19
2025-12-09
Wing Aviation Llc
Processes for generating and updating flyable airspace for unmanned aerial vehicles
US12602924B2
( en )
2022-03-31
2026-04-14
Wing Aviation Llc
Method for semantic localization of an unmanned aerial vehicle
US12148206B2
( en )
2022-03-31
2024-11-19
Wing Aviation Llc
Method for controlling an unmanned aerial vehicle to avoid obstacles
CN115146933A
( en )
*
2022-06-20
2022-10-04
æ·±å³å¸åå¸å ¬å ±å®å ¨ææ¯ç 究é¢æéå ¬å¸
Method, system, equipment and storage medium for dealing with explosion accident of hazardous source
US12564955B2
( en )
2022-06-23
2026-03-03
Boston Dynamics, Inc.
Modeling robot self-occlusion for localization
US12449822B2
( en )
2022-06-23
2025-10-21
Boston Dynamics, Inc.
Ground clutter avoidance for a mobile robot
US12399508B2
( en )
2022-11-17
2025-08-26
Wing Aviation Llc
Automatic selection of delivery zones using survey flight 3D scene reconstructions
US20240192695A1
( en )
*
2022-12-07
2024-06-13
Boston Dynamics, Inc.
Anchoring based transformation for aligning sensor data of a robot with a site model
US20240201697A1
( en )
*
2022-12-19
2024-06-20
International Business Machines Corporation
Unmanned aerial vehicle deployment parameter modification
US12372977B2
( en )
*
2022-12-19
2025-07-29
International Business Machines Corporation
Unmanned aerial vehicle deployment parameter modification
WO2024258415A1
( en )
*
2023-06-16
2024-12-19
Siemens Corporation
System and method for automated path planning and travel for autonomous vehicles
CN116880329A
( en )
*
2023-08-17
2023-10-13
é西ç§æ大å¦
Intelligent autonomous landing control system and method for flapping-wing aircraft based on active disturbance rejection
US12608931B2
( en )
2024-04-19
2026-04-21
Wing Aviation Llc
Generating aerial paths based on properties of aerial image data
CN119272421A
( en )
*
2024-12-10
2025-01-07
ç¦å»ºç¦ç¿æºç§ææéå ¬å¸
A simulation deformation aircraft and control method
Similar Documents
Publication
Publication Date
Title
US12400552B2
( en )
2025-08-26
Unmanned aerial vehicle visual line of sight control
US12481289B2
( en )
2025-11-25
Backup navigation system for unmanned aerial vehicles
US12148316B2
( en )
2024-11-19
Unmanned aerial vehicle visual point cloud navigation
US20220357753A1
( en )
2022-11-10
Drop-off location planning for delivery vehicle
US20200301015A1
( en )
2020-09-24
Systems and methods for localization
CN110268356B
( en )
2023-08-25
Leading unmanned aerial vehicle&#39;s system
CN110226143B
( en )
2023-08-22
Leading UAV approach
AU2022252797B2
( en )
2024-09-12
Method for semantic localization of an unmanned aerial vehicle
WO2017147142A1
( en )
2017-08-31
Unmanned aerial vehicle visual line of sight control
US12608931B2
( en )
2026-04-21
Generating aerial paths based on properties of aerial image data
US20250376259A1
( en )
2025-12-11
Automatic Selection of Delivery Zones Using Survey Flight 3D Scene Reconstructions
Singh et al.
2016
Perception for safe autonomous helicopter flight and landing
US20260064121A1
( en )
2026-03-05
Unmanned aerial vehicle operation and mission planning in low light environment
US20260141813A1
( en )
2026-05-21
Unmanned Aerial Vehicle Visual Point Cloud Navigation
WO2025221369A2
( en )
2025-10-23
Image-derived text delivery location descriptions
Legal Events
Date
Code
Title
Description
2020-06-26
STPP
Information on status: patent application and granting procedure in general
Free format