Google Patentsopen access

Drone-based administration of remotely located instruments and gadgets — University Of Florida Research Foundation, Inc. (US11978023B2)

ABSTRACT

Abstract

The present disclosure describes various embodiments of systems, apparatuses, and methods for drone-based administration of remotely located devices. One such method comprises deploying an unmanned aerial vehicle from a base station, wherein the base station assigns a maintenance order to the unmanned aerial vehicle for servicing of a remote device, traveling, by the unmanned aerial vehicle, to the location of the remote device, authenticating, by the unmanned aerial vehicle, a valid identification of the remote device; upon the remote device being authenticated by the unmanned aerial vehicle, servicing the remote device by at least charging a power supply of the remote device and transferring contents of a device log to the unmanned aerial vehicle; and after completing the servicing of the remote device; returning to the base station and transferring contents of the device log to the base station.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. provisional application entitled, “Darling: Drone-Based Administration of Remotely Located Instruments and Gadgets,” having Ser. No. 63/077,303, filed Sep. 11, 2020, which is entirely incorporated herein by reference.

BACKGROUND

The advent of Internet of Things (IoT) has seen the rise in the deployment of many low-cost devices for automating various tasks in our day-to-day life. These devices are portable owing to the fact that they are battery operated and thus can be placed virtually anywhere. For example, a battery-operated security camera could be placed at a strategic angle, thereby enabling better coverage of a property and thus enhancing the security of the property. Similarly, remote surveillance cameras could be used in national parks for monitoring poaching or illegal trespassing. However, the remote nature of these devices makes the maintenance of these devices an arduous task. Routine tasks such as replacing the battery, data-backup, and debugging of these remote devices are currently performed by humans. In the instance of remote monitoring for poaching, the person would have to navigate treacherous terrain to perform these tasks. Consequently, having a person service these target remote entities (e.g., devices) may pose an occupational and environmental hazard to them.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

FIG. 1 shows a high-level overview of the an exemplary system for drone-based administration of remotely located instruments and gadgets (DARLING) for servicing multiple use cases involving a remote resource and home automation, in accordance with various embodiments of the present disclosure.

FIG. 2 shows a general architecture of an exemplary DARLING system in accordance with various embodiments of the present disclosure.

FIG. 3 shows component parts of an exemplary base station in accordance with various embodiments of the present disclosure.

FIG. 4 shows component parts of an exemplary unmanned aerial vehicle (UAV) in accordance with various embodiments of the present disclosure.

FIG. 5 shows component parts of an exemplary UAV with a detachable control unit (DCU) in accordance with various embodiments of the present disclosure.

FIG. 6 shows component parts of an exemplary serviceable remote device in accordance with various embodiments of the present disclosure.

FIG. 7 shows software units for an exemplary base station in accordance with various embodiments of the present disclosure.

FIG. 8 shows a block diagram of an exemplary UAV in accordance with various embodiments of the present disclosure.

FIG. 9 shows a flow diagram of an exemplary maintenance procedure at a remote device in accordance with various embodiments of the present disclosure.

FIG. 10 shows a diagram representation of an exemplary DCU hanging off hooks on a horizontal surface in accordance with various embodiments of the present disclosure.

FIG. 11 shows a diagram representation of an exemplary DCU laying on suction cups in accordance with various embodiments of the present disclosure.

FIG. 12 shows a diagram representation of an exemplary DCU hanging off hooks attached on a collar loop connected to a suction cup in accordance with various embodiments of the present disclosure.

FIG. 13 shows a diagram representation of an exemplary DCU hanging off hooks attached on a collar loop drilled against a vertical surface in accordance with various embodiments of the present disclosure.

FIG. 14 shows a diagram representation of an exemplary UAV swapping out a battery at a remote device in accordance with various embodiments of the present disclosure.

FIG. 15 shows an exemplary authentication key exchange diagram in accordance with various embodiments of the present disclosure.

FIG. 16 shows a block diagram of an exemplary artificial intelligence (AI) predictive maintenance unit of a base station in accordance with various embodiments of the present disclosure.

FIG. 17 shows an exemplary message passing scheme on a DARLING maintenance network in accordance with various embodiments of the present disclosure.

FIG. 18 shows a diagram representation of an exemplary rule for determining which UAV will service a distressed remote device based on an identification (ID) number in accordance with various embodiments of the present disclosure.

FIG. 19 shows a diagram representation of a process of relaying of distress signals by UAVs to the base station and the deployment of available UAVs to respond to the distress signals by the base station in accordance with various embodiments of the present disclosure.

FIGS. 20 - 21 show diagram representations of a process of relaying of distress signals by UAVs to the base station and waiting for deployment of a first available UAV to respond to the distress signals by the base station in accordance with various embodiments of the present disclosure.

FIG. 22 shows a diagram representation of a use case for remote resource monitoring utilizing an exemplary DARLING system in accordance with various embodiments of the present disclosure.

FIG. 23 shows a diagram representation of a use case for home automation servicing utilizing an exemplary DARLING system in accordance with various embodiments of the present disclosure.

DETAILED DESCRIPTION

The present disclosure describes various embodiments of systems, apparatuses, and methods for Drone-Based Administration of Remotely Located Instruments and Gadgets (DARLING). In various embodiments, DARLING provides a framework for assisting remote devices using an Unmanned Aerial Vehicle (UAV). In accordance with embodiments of the present disclosure, a DARLING architecture has three hardware components: the UAV, a base station which is responsible for housing the UAV during non-deployment, and remote devices which the UAV services. The present disclosure describes various aspects of the base station and the UAVs in order to carry out a remote maintenance procedure. Whenever it is unsafe for the UAV to land near the remote device, in various embodiments of the present disclosure, the UAV can deploy a Detachable Charging Unit (DCU) having a platform that carries a battery pack and connects to the remote device for charging while the UAV services another device. Different types of maintenance can be performed based on a schedule or unexpected issues discovered by artificial intelligence (AI) models in either of the base station or the UAVs, in various embodiments. Most maintenance tasks are referred to as regular maintenance, where standard servicing procedures occur, such as charging and data transfer. Additionally, in various embodiments, DARLING can also anticipate future issues at the base station based on the collected diagnostic information from the remote device via the UAV through predictive maintenance. For example, if the UAV predicts or discovers an issue on the remote device, the UAV can perform targeted maintenance on the remote device. Correspondingly, the present disclosure considers various use cases, such as (a) an IoT device that is in a remote location and (b) an IoT device that is in an inconvenient location—and demonstrates the role of DARLING in these scenarios.

FIG. 1 displays a high-level overview of an exemplary embodiment of the DARLING framework that contains three main components: (1) a base station 10 ; (2) unmanned aerial vehicles (UAVs) 20 ; and (3) remote devices 30 . The UAVs are the central components of DARLING and are equipped with the capabilities to navigate and perform a remote maintenance task on a remote device 30 . In addition to the UAVs 20 and the remote devices 30 , DARLING also has the base- station 10 which is used to house the idle (unspent or charging) UAVs 20 . The base station 10 additionally has the capabilities to interact with the UAVs 20 for guidance and monitoring purposes, in various embodiments. In general, a base station 10 acts as a headquarter and charging station for the UAVs 20 in between servicing periods.

As compared to the state-of-the-art, the DARLING system enables secure, reliable, and seamless monitoring and maintenance of remote devices 30 . In various embodiments, the DARLING system includes (a) a fleet of unmanned aerial vehicles (UAVs) 20 configured to perform maintenance tasks on remote devices 30 and (b) a base station 10 that is configured to serve as the UAV headquarters and command central.

In various embodiments, an exemplary design for the unmanned aerial vehicles 20 incorporates a detachable charging unit (DCU) that allows the charging of a device while the UAV 20 services other remote devices 30 . Accordingly, an exemplary UAV 20 can perform remote maintenance tasks under a variety of use cases. For example, in the case of remote resource monitoring, remote devices 30 may be deployed in remote, hard-to-access locations that require maintenance operations like battery-replacement, data-backup, debug, and in some cases emergency tamper detection. Such remote devices 30 may have no Internet connection and may require periodic monitoring. Examples include a remote surveillance camera in the forest or a buoy with weather tracking sensors in the middle of the ocean. Another case is home automation, in which remote devices 30 may be deployed in residential homes which require periodic maintenance such as surveillance cameras. For such use cases, in addition to the maintenance tasks, the UAVs 20 may also act as a portable router that helps these remote devices 30 connect to the internet for firmware upgrades and backup. The UAVs 20 can also help in optimal positioning of these devices 30 for ensuring maximum efficiency. For example, surveillance cameras are often positioned such that they are accessible to a human but these locations might not be optimal from a coverage point-of-view. Using an UAV 20 , in accordance with various embodiments of the present disclosure, would relax the accessibility constraint thereby efficiently positioning the UAVs 20 .

In various embodiments, interactions between the UAV 20 , the remote device 30 , and the base station 10 are secured using authentication mechanisms in the DARLING framework to validate the interactions between the UAVs 20 and the <figure-callout id="30" label="

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. provisional application entitled, “Darling: Drone-Based Administration of Remotely Located Instruments and Gadgets,” having Ser. No. 63/077,303, filed Sep. 11, 2020, which is entirely incorporated herein by reference.

BACKGROUND

The advent of Internet of Things (IoT) has seen the rise in the deployment of many low-cost devices for automating various tasks in our day-to-day life. These devices are portable owing to the fact that they are battery operated and thus can be placed virtually anywhere. For example, a battery-operated security camera could be placed at a strategic angle, thereby enabling better coverage of a property and thus enhancing the security of the property. Similarly, remote surveillance cameras could be used in national parks for monitoring poaching or illegal trespassing. However, the remote nature of these devices makes the maintenance of these devices an arduous task. Routine tasks such as replacing the battery, data-backup, and debugging of these remote devices are currently performed by humans. In the instance of remote monitoring for poaching, the person would have to navigate treacherous terrain to perform these tasks. Consequently, having a person service these target remote entities (e.g., devices) may pose an occupational and environmental hazard to them.

BRIEF DESCRIPTION OF THE DRAWINGS

Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.

FIG. 1 shows a high-level overview of the an exemplary system for drone-based administration of remotely located instruments and gadgets (DARLING) for servicing multiple use cases involving a remote resource and home automation, in accordance with various embodiments of the present disclosure.

FIG. 2 shows a general architecture of an exemplary DARLING system in accordance with various embodiments of the present disclosure.

FIG. 3 shows component parts of an exemplary base station in accordance with various embodiments of the present disclosure.

FIG. 4 shows component parts of an exemplary unmanned aerial vehicle (UAV) in accordance with various embodiments of the present disclosure.

FIG. 5 shows component parts of an exemplary UAV with a detachable control unit (DCU) in accordance with various embodiments of the present disclosure.

FIG. 6 shows component parts of an exemplary serviceable remote device in accordance with various embodiments of the present disclosure.

FIG. 7 shows software units for an exemplary base station in accordance with various embodiments of the present disclosure.

FIG. 8 shows a block diagram of an exemplary UAV in accordance with various embodiments of the present disclosure.

FIG. 9 shows a flow diagram of an exemplary maintenance procedure at a remote device in accordance with various embodiments of the present disclosure.

FIG. 10 shows a diagram representation of an exemplary DCU hanging off hooks on a horizontal surface in accordance with various embodiments of the present disclosure.

FIG. 11 shows a diagram representation of an exemplary DCU laying on suction cups in accordance with various embodiments of the present disclosure.

FIG. 12 shows a diagram representation of an exemplary DCU hanging off hooks attached on a collar loop connected to a suction cup in accordance with various embodiments of the present disclosure.

FIG. 13 shows a diagram representation of an exemplary DCU hanging off hooks attached on a collar loop drilled against a vertical surface in accordance with various embodiments of the present disclosure.

FIG. 14 shows a diagram representation of an exemplary UAV swapping out a battery at a remote device in accordance with various embodiments of the present disclosure.

FIG. 15 shows an exemplary authentication key exchange diagram in accordance with various embodiments of the present disclosure.

FIG. 16 shows a block diagram of an exemplary artificial intelligence (AI) predictive maintenance unit of a base station in accordance with various embodiments of the present disclosure.

FIG. 17 shows an exemplary message passing scheme on a DARLING maintenance network in accordance with various embodiments of the present disclosure.

FIG. 18 shows a diagram representation of an exemplary rule for determining which UAV will service a distressed remote device based on an identification (ID) number in accordance with various embodiments of the present disclosure.

FIG. 19 shows a diagram representation of a process of relaying of distress signals by UAVs to the base station and the deployment of available UAVs to respond to the distress signals by the base station in accordance with various embodiments of the present disclosure.

FIGS. 20 - 21 show diagram representations of a process of relaying of distress signals by UAVs to the base station and waiting for deployment of a first available UAV to respond to the distress signals by the base station in accordance with various embodiments of the present disclosure.

FIG. 22 shows a diagram representation of a use case for remote resource monitoring utilizing an exemplary DARLING system in accordance with various embodiments of the present disclosure.

FIG. 23 shows a diagram representation of a use case for home automation servicing utilizing an exemplary DARLING system in accordance with various embodiments of the present disclosure.

DETAILED DESCRIPTION

The present disclosure describes various embodiments of systems, apparatuses, and methods for Drone-Based Administration of Remotely Located Instruments and Gadgets (DARLING). In various embodiments, DARLING provides a framework for assisting remote devices using an Unmanned Aerial Vehicle (UAV). In accordance with embodiments of the present disclosure, a DARLING architecture has three hardware components: the UAV, a base station which is responsible for housing the UAV during non-deployment, and remote devices which the UAV services. The present disclosure describes various aspects of the base station and the UAVs in order to carry out a remote maintenance procedure. Whenever it is unsafe for the UAV to land near the remote device, in various embodiments of the present disclosure, the UAV can deploy a Detachable Charging Unit (DCU) having a platform that carries a battery pack and connects to the remote device for charging while the UAV services another device. Different types of maintenance can be performed based on a schedule or unexpected issues discovered by artificial intelligence (AI) models in either of the base station or the UAVs, in various embodiments. Most maintenance tasks are referred to as regular maintenance, where standard servicing procedures occur, such as charging and data transfer. Additionally, in various embodiments, DARLING can also anticipate future issues at the base station based on the collected diagnostic information from the remote device via the UAV through predictive maintenance. For example, if the UAV predicts or discovers an issue on the remote device, the UAV can perform targeted maintenance on the remote device. Correspondingly, the present disclosure considers various use cases, such as (a) an IoT device that is in a remote location and (b) an IoT device that is in an inconvenient location—and demonstrates the role of DARLING in these scenarios.

FIG. 1 displays a high-level overview of an exemplary embodiment of the DARLING framework that contains three main components: (1) a base station 10 ; (2) unmanned aerial vehicles (UAVs) 20 ; and (3) remote devices 30 . The UAVs are the central components of DARLING and are equipped with the capabilities to navigate and perform a remote maintenance task on a remote device 30 . In addition to the UAVs 20 and the remote devices 30 , DARLING also has the base- station 10 which is used to house the idle (unspent or charging) UAVs 20 . The base station 10 additionally has the capabilities to interact with the UAVs 20 for guidance and monitoring purposes, in various embodiments. In general, a base station 10 acts as a headquarter and charging station for the UAVs 20 in between servicing periods.

As compared to the state-of-the-art, the DARLING system enables secure, reliable, and seamless monitoring and maintenance of remote devices 30 . In various embodiments, the DARLING system includes (a) a fleet of unmanned aerial vehicles (UAVs) 20 configured to perform maintenance tasks on remote devices 30 and (b) a base station 10 that is configured to serve as the UAV headquarters and command central.

In various embodiments, an exemplary design for the unmanned aerial vehicles 20 incorporates a detachable charging unit (DCU) that allows the charging of a device while the UAV 20 services other remote devices 30 . Accordingly, an exemplary UAV 20 can perform remote maintenance tasks under a variety of use cases. For example, in the case of remote resource monitoring, remote devices 30 may be deployed in remote, hard-to-access locations that require maintenance operations like battery-replacement, data-backup, debug, and in some cases emergency tamper detection. Such remote devices 30 may have no Internet connection and may require periodic monitoring. Examples include a remote surveillance camera in the forest or a buoy with weather tracking sensors in the middle of the ocean. Another case is home automation, in which remote devices 30 may be deployed in residential homes which require periodic maintenance such as surveillance cameras. For such use cases, in addition to the maintenance tasks, the UAVs 20 may also act as a portable router that helps these remote devices 30 connect to the internet for firmware upgrades and backup. The UAVs 20 can also help in optimal positioning of these devices 30 for ensuring maximum efficiency. For example, surveillance cameras are often positioned such that they are accessible to a human but these locations might not be optimal from a coverage point-of-view. Using an UAV 20 , in accordance with various embodiments of the present disclosure, would relax the accessibility constraint thereby efficiently positioning the UAVs 20 .

In various embodiments, interactions between the UAV 20 , the remote device 30 , and the base station 10 are secured using authentication mechanisms in the DARLING framework to validate the interactions between the UAVs 20 and the remote devices 30 which helps in monitoring for tampering of the devices 30 . The inclusion of a predictive maintenance mechanism in an exemplary embodiment of the DARLING system, including an AI Predictive Maintenance Unit in the base station&#39;s computing hardware, can anticipate future issues for remote devices 30 and creates fixes that the base station 10 can roll out at an appropriate time. Accordingly, the use of machine learning models, such as Big-Little Deep Neural Networks (BL-DNN), of different power and computational capabilities on both the base station 10 and the UAV 20 can aid in anticipating future issues and confirming detected problems encountered during regular maintenance.

Regarding communication capabilities, in various embodiments, there is UAV-to-base station communication, in which an exemplary base station 10 is capable of connecting to the Internet to retrieve software upgrades for the remote device 30 . Additionally, in various use cases, the UAV 20 and/or the remote device 30 may also have the capability to connect to the Internet. In such cases, the UAV 10 and the remote device 30 may enlist cloud-based services, such as cloud-based authentication.

FIG. 2 summarizes the general architecture for an exemplary DARLING system, which has three principal components: 1) a base station 10 , 2) unmanned aerial vehicles (UAVs) 20 , and 3) remote devices 30 . In various embodiments, the UAVs 20 can service the remote devices 30 by charging their batteries, moving and clearing data from their storage, and other maintenance tasks. The first two maintenance tasks may be considered as part of the regular maintenance of these devices 30 . While the primary focus of this technology is the first two, the UAVs 20 can perform other servicing procedures, such as correcting the remote device&#39;s physical placement, recalibrating the remote device&#39;s internal sensors or itself, physically cleaning the exterior, debugging hardware (HW) and/or software (SW) features on the field, and bringing replacement parts to the remote device 30 or its components if replacements are necessary. These primary and additional tasks can occur out of a regular schedule as the base station 10 deems it necessary using a predictive maintenance AI model. In essence, the base station 10 serves as the UAV headquarters, which can assign maintenance orders to the UAVs 20 for the remote devices 30 . Thus, the UAVs 20 can perform the bulk of the maintenance work on the remote device 30 in the present disclosure as the UAV 20 physically travels to the remote device 30 for servicing.

In various embodiments, the base station 10 makes the high level decision in determining appropriate maintenance tasks for the UAVs 20 to perform on certain devices 30 in an appropriate time frame. As the base station 10 serves as the home for the UAVs 20 in between servicing periods, the base station 10 will have charging stations for the UAV 20 to recharge batteries and a secure storage module 50 ( FIG. 3 ) to transfer data from devices. In order to perform their tasks effectively, in various embodiments, the UAVs 20 are equipped with a camera, retractable robot arms with an optional drill, cable connectors, edge artificial intelligence (AI) unit, GPS, and batteries. Depending on environmental factors, the UAVs 20 can be additionally equipped with a detachable charging unit (DCU) platform, hooks/carabiners, ropes, suction cups, vacuum pump, and screwable collar loop. These additional parts makes it possible for the UAV 20 to charge a device where a safe landing zone is not feasible. These parts also ensure modifications to the remote devices 30 are not needed, especially when these changes involve having a person travel to the device&#39;s remote and/or inconvenient location.

An exemplary non-limiting general maintenance procedure begins with the base station 10 issuing maintenance instructions to the UAVs 20 along with relevant information about the remote devices 30 they will service. The base station 10 will provide more information about the remote device 30 and its general vicinity if it has been previously serviced. Otherwise, the general location and some basic information about the remote device 30 are provided to the UAV 20 . The UAV 20 will use this location information and self-navigate to the general area of the remote device 30 . As the UAV 20 approaches this location, the UAV 20 will either use firsthand knowledge about the remote device&#39;s environment or gather information using its camera and communicate with the remote device 30 itself to determine the proper maintenance configuration. Afterwards, the UAV 20 will perform pre-maintenance routines to check the health of the remote device 330 as well as authenticate the communication between the UAV 20 and the remote device 30 . If successful, maintenance begins and, depending on the maintenance configuration, the UAV 20 may leave the current location to service another device while the current one charges. After completing maintenance of this device, the UAV 20 will continue servicing other remote devices 30 and return to the base station 10 after all of the remote devices 30 finish to charge its own batteries and transfer data to the secure storage. This procedure can repeat periodically for scheduled maintenance or as needed for emergency repairs.

Referring back to FIG. 2 , the base station 10 acts as a centralized hub for the UAVs 20 as well as a main data center of all the remote devices 30 . Since charging is a major maintenance task for the UAVs 20 , the base station 10 can come equipped with charging stations for not only their batteries but also battery packs 40 for charging the devices 30 they service, as shown in FIG. 3 . Another important task of data transfers requires the base station 10 to have a secure storage module 50 (as shown in FIG. 3 ) to collect the data from the remote device 30 for analyzing the sensor information as well as improve future maintenance by processing diagnostics data from the remote devices 30 . Using the collected data, the base station 10 can employ an AI computing unit, such as using the “Big” portion of a two-part Big Little Deep Neural Network (BL-DNN). Such an AI unit can feature a powerful processor (e.g., CPUs and GPUs) in the base station&#39;s computer(s). These parts make the operation of the base station 10 as a delegator and data collection center very efficient.

Next, FIG. 4 shows component parts for the UAV 20 . At a minimum, the UAV 20 (with a payload carrier 25 ) can be equipped with a camera 21 ; a retractable robotic arm 22 ; cable connectors; a ranger/tracker unit 23 (e.g., GPS (global positioning system) and LIDAR (light detection and ranging)); batteries 24 , storage media 26 , and a processor 27 with an edge AI unit and a wireless transmitter/receiver module. Such parts allow the UAV 20 to service a remote device 30 as long as there is a safe landing zone for the UAV 20 to stay in place during the maintenance procedure. The camera 21 gives visual feedback for navigating through obstacles, finding the remote device 30 in the prescribed location, and coordinating the retractable arm 22 , akin to the human eye guiding the arm as it grabs objects. The retractable arm 22 enables the UAV 20 to move objects between its payload chassis 25 and the remote device 30 . The cable connectors provide the physical communication media between the UAV&#39;s processor 27 and the remote device&#39;s processor as well as from the DCU&#39;s battery pack to the remote device 30 . To aid the UAV&#39;s self-navigation algorithm, the UAV receives feedback from a ranger or tracker unit 23 , such as a global positioning system (GPS) and Light Detection and Ranging (LIDAR) to locate itself during flight. An important payload in the UAV 20 for this technology are the battery packs 24 and the storage media 60 which will extend the longevity of the remote device 30 as the UAV 20 replenish its power and data resources. At the heart of controlling all these parts is a processor 27 with edge AI computing capabilities. This processor 27 can make some predictions about possible faults in the device based on collected diagnostic information. In order to communicate with nearby servicing UAVs 20 and remote devices 30 at an area, the wireless transmitter and receiver modules enable the UAVs 20 to determine if other devices nearby require targeted maintenance. All in all, these parts constitute the basic requirements for a UAV 20 in an exemplary embodiment of the DARLING framework.

An extension to the basic UAV hardware is the detachable charging unit (DCU), which is utilized when there is no safe landing zone around the remote device 30 for the UAV. Given this scenario, the purpose of the DCU is to charge the remote device 30 without needing the UAV 20 to land. Accordingly, the DCU platform unit lets the UAV 20 service other remote devices 30 while the DCU charges the remote device 30 . In accordance with various embodiments, FIG. 5 shows an exemplary UAV 20 with a DCU platform 65 having additional parts that make up the DCU 65 , which includes suction cups 66 ; vacuum pump 67 ; hooks/ carabiners 68 ; support ropes 69 ; electric drill; collar loop; and magnetic platforms. The suction cups 66 of the DCU platform help to hold the DCU 65 in place by sticking to smooth surfaces. A vacuum pump 67 can also pull air in and out of the cup to remove and stick the suction cup 66 onto a surface, respectively. When there is a surface/area to which the DCU 65 can latch onto, the DCU 65 can use hooks/ carabiners 68 , and support ropes 69 to attach or hang the DCU 65 during the charging process. The DCU 65 can also use an electric drill to puncture holes into surfaces that the DCU 65 cannot latch onto and create an attachment point for a collar loop to which the DCU 65 along with hooks, carabiners, and support ropes. With all these means of attaching the DCU 65 , the DCU 65 holds the battery pack 24 using a magnetic platform which keeps the battery pack 24 in place as well as itself onto attachment configurations, such as with suction cups 66 . Except for the DCU platforms, any combination of the parts to attach the DCU is contingent upon the local environmental information around the remote device.

FIG. 6 shows component parts for a serviceable remote device 30 . Since the primary maintenance tasks of the UAVs 20 are to replenish power and data resources, the remote devices 30 to be serviced should be equipped with a battery 32 (removable or internal); a storage medium 34 (e.g., SD card, hard disk drive, solid state drive, etc.); a connection port 35 (if applicable); and a wireless transmitter/receiver 36 (if applicable).

Based on the type of remote device 30 , the UAV 20 will handle removable and internal batteries and storage media in distinct ways. Removable components are, of course, conducive to quicker maintenance as swapping out these components takes a shorter time compared to charging the battery and transferring files onto the storage media on the UAV 20 . These basic components are necessary on the remote devices 30 for a UAV 20 in the DARLING framework to service them properly.

When there is a connection port 35 on the remote device 30 , the port allows the UAV 20 to collect device information and diagnostic data for predictive maintenance. Depending on the AI model&#39;s prediction on the UAV 20 , the retrieved data help inform the UAV 20 and the base station 10 of pending problems. The latter of these will prompt the base station 10 to perform targeted maintenance on this particular device 30 . If this targeted maintenance also applies to devices 30 of the same type, the base station 10 will deploy UAVs 20 to service affected devices 30 once more to address the discovered issue.

The addition of a wireless transmitter and receiver 36 enables the remote device 30 to relay their own problems or problems of other nearby devices 30 to servicing UAVs 20 , thereby improving coverage for targeted maintenance and addressing affected devices 30 quickly rather than waiting for the next regular maintenance schedule. This part can use any wireless communication standard, such as Wi-Fi, Bluetooth, and Zigbee. Fly by UAVs 20 can also detect the wireless signals in order to take notes of which devices 30 require attention along their maintenance route. Such parts improve the capabilities of the DARLING system in addressing device issues as quickly as they occur in the network.

Along with the hardware parts, the base station 10 and the UAVs 20 have their corresponding software units and constructs that allow each component to make their own decisions regarding the maintenance scenarios, in various embodiments. An exemplary DARLING framework handles different types of maintenance based on a schedule or unexpected issues determined by an AI model both in the long-term and in the immediate future, such as Regular Maintenance—based on a predetermined schedule stored in a database; Predictive Maintenance—based on long-term issues detected by a base station&#39;s AI model; and Targeted Maintenance—based on short-term issues detected by UAV&#39;s and a base station&#39;s AI model.

As shown in FIG. 7 , an exemplary base station 10 can have the following software units and constructs on its computing system: a device database 72 , a maintenance scheduler 74 , a service solution unit 76 , and an AI predictive maintenance unit 78 . For example, the device database 72 can be configured to keep track of device information for the UAV 20 as well as help other software units to make high-level decisions for the system which includes, but is not limited to, the following information: device identification (ID) and type, location information (e.g. coordinates, nearby environment), available software/firmware (SW/FW) updates (e.g., utilizes Internet connection), next estimated maintenance schedule, expected end of life; and previous device issues.

The maintenance scheduler 74 can be configured to dispatch and give instructions to the UAVs 20 to service a set of remote devices 30 based on a timetable provided by the device database 72 or the AI predictive maintenance unit 78 , whereas the service solution unit 76 can be configured to recommend fixes, patches, or SW/FW upgrades to resolve problems found by the predictive maintenance unit or during recent maintenance using diagnostic information and predicted/discovered issue. The AI predictive maintenance unit 78 can be configured to perform predictive analysis on collected diagnostic information from the remote devices 30 via the <figure-callout id="20" label="UAVs

CLAIMS

Claims ( 20 )

The invention claimed is:

1. A method comprising:

deploying, from a base station, an unmanned aerial vehicle from a group of unmanned aerial vehicles, wherein the base station assigns a maintenance order to the unmanned aerial vehicle for servicing of a remote device, wherein the maintenance order includes a location for the remote device;

traveling, by the unmanned aerial vehicle, to the location of the remote device, wherein the unmanned aerial vehicle self-navigates to the location of the remote device;

authenticating, by the unmanned aerial vehicle, a valid identification of the remote device;

upon the remote device being authenticated by the unmanned aerial vehicle, servicing the remote device by at least charging a power supply of the remote device and transferring contents of a device log to the unmanned aerial vehicle;

predicting, by the unmanned aerial vehicle using an artificial intelligence model, that the transferred device log shows that remote device has a maintenance issue that is not currently scheduled to be serviced by the base station;

servicing, by the unmanned aerial vehicle, the maintenance issue before returning to the base station; and

after completing the servicing of the remote device, returning to the base station and transferring contents of the device log to the base station.

2. The method of claim 1 , further comprising after returning to the base station, charging a power supply of the unmanned aerial vehicle.

3. The method of claim 1 , wherein the unmanned aerial vehicle is equipped with a camera, the method further comprising using the camera to gather information on the remote device and identify a maintenance configuration for accessing a charge port and a device log port of the remote device to enable charging of the power supply from the unmanned aerial vehicle and transferring of the device log of the remote device.

4. The method of claim 1 , wherein the unmanned aerial vehicle is equipped with a camera and a robotic arm, the method further comprising using the camera to gather information on the remote device and identify a maintenance configuration for accessing a swappable battery and a swappable memory card containing the device log, wherein the unmanned aerial vehicle utilizes the robotic arm to physically swap out the swappable battery with a replacement battery and the swappable memory card with a replacement memory card on the remote device.

5. The method of claim 4 , further comprising:

assessing, by the unmanned aerial vehicle, whether it is safe to land near the remote device; and

if the unmanned aerial vehicle determines that it is not safe to land near the remote device, detaching a maintenance platform that is secured to a terrain next to the remote device using the robotic arm, wherein the maintenance platform carries a power source for charging or replacing the power source of the remote device and a data transfer storage medium for storing the contents of the device log, wherein the maintenance platform is configured to carry out servicing of the remote device while the unmanned aerial vehicle returns to the base station or services another remote device.

6. The method of claim 1 , further comprising before servicing the remote device, performing, by the unmanned aerial vehicle, a pre-maintenance routine to check a health of the remote device.

7. The method of claim 1 , wherein the remote device authenticates an identification of the unmanned aerial vehicle to be valid before allowing a transfer of the device log to the unmanned aerial vehicle.

8. The method of claim 1 , further comprising predicting, by the base station using an artificial intelligence model, a maintenance issue involving the remote device based on a plurality of device logs returned from a plurality of unmanned aerial vehicles that service a plurality of remote devices.

9. The method of claim 8 , wherein the unmanned aerial vehicle and the base station collectively utilize an artificial intelligence for modeling.

10. The method of claim 8 , wherein the base station authenticates an identification of the unmanned aerial vehicle to be valid before allowing a transfer of the device log from the unmanned aerial vehicle to the base station.

11. The method of claim 1 , wherein the remote device comprises a home automation device, a security device, a sensor device, or a medical device.

12. The method of claim 1 , further comprising:

detecting, by the unmanned aerial vehicle, a distress signal issued by a different remote device that is not being serviced by the unmanned aerial vehicle; and

relaying, by the unmanned aerial vehicle, the distress signal to another unmanned aerial vehicle, another remote device, or the base station, wherein the distress signal is relayed until the distress signal is transmitted to the base station.

13. A system comprising:

a base station; and

an unmanned aerial vehicle configured to:

receive a maintenance order for servicing of a remote device from the base station, wherein the maintenance order includes a location for the remote device;

travel to the location of the remote device, wherein the unmanned aerial vehicle self-navigates to the location of the remote device;

authenticate a valid identification of the remote device;

upon the remote device being authenticated by the unmanned aerial vehicle, service the remote device by at least charging a power supply of the remote device and transferring contents from a device, including its log, to the unmanned aerial vehicle; and

after completing the servicing of the remote device, return to the base station and transfer contents from a device, including its log, to the base station,

wherein the unmanned aerial vehicle is configured to predict, using an artificial intelligence model, that the transferred device log show that remote device has a maintenance issue that is not currently scheduled to be serviced by the base station.

14. The system of claim 13 , wherein the unmanned aerial vehicle is equipped with a camera, wherein the unmanned aerial vehicle is configured to gather information on the remote device using the camera and identify a maintenance configuration for accessing a charge port and a device log port of the remote device to enable charging of the power supply from the unmanned aerial vehicle and transferring of the device log of the remote device.

15. The system of claim 13 , wherein the unmanned aerial vehicle is equipped with a camera and a robotic arm, wherein the unmanned aerial vehicle is configured to gather information on the remote device using the camera and identify a maintenance configuration for accessing a swappable battery and a swappable memory card containing the device log, wherein the unmanned aerial vehicle is configured to connect a maintenance platform carrying a replacement battery and a replacement memory card to the remote device and utilize the robotic arm to physically swap out the swappable battery with the replacement battery and the swappable memory card with the replacement memory card on the remote device.

16. The system of claim 13 , wherein the base station is configured to predict, using an artificial intelligence model, a maintenance issue involving the remote device based on a plurality of device logs returned from a plurality of unmanned aerial vehicles that service a plurality of remote devices, wherein the plurality of unmanned aerial vehicles comprise the unmanned aerial vehicle.

17. The system of claim 16 , wherein the base station is configured to authenticate an identification of the unmanned aerial vehicle to be valid before allowing a transfer of the device log from the unmanned aerial vehicle to the base station.

18. The system of claim 13 , wherein the unmanned aerial vehicle is configured to detect a distress signal issued by a different remote device that is not being serviced by the unmanned aerial vehicle and is configured to relay the distress signal to another unmanned aerial vehicle, another remote device, or the base station, wherein the distress signal is relayed until the distress signal is transmitted to the base station.

19. A method comprising:

deploying, from a base station, an unmanned aerial vehicle from a group of unmanned aerial vehicles, wherein the base station assigns a maintenance order to the unmanned aerial vehicle for servicing of a remote device, wherein the maintenance order includes a location for the remote device, wherein the unmanned aerial vehicle is equipped with a camera and a robotic arm;

traveling, by the unmanned aerial vehicle, to the location of the remote device, wherein the unmanned aerial vehicle self-navigates to the location of the remote device;

assessing, by the unmanned aerial vehicle, whether it is safe to land near the remote device;

if the unmanned aerial vehicle determines that it is not safe to land near the remote device, detaching a maintenance platform that is secured to a terrain next to the remote device using the robotic arm, wherein the maintenance platform carries a power source for charging or replacing the power source of the remote device and a data transfer storage medium for storing contents of a device log, wherein the maintenance platform is configured to carry out servicing of the remote device while the unmanned aerial vehicle returns to the base station or services another remote device;

authenticating, by the unmanned aerial vehicle, a valid identification of the remote device;

upon the remote device being authenticated by the unmanned aerial vehicle, servicing the remote device by at least charging a power supply of the remote device and transferring contents of the device log to the unmanned aerial vehicle;

using the camera to gather information on the remote device and identify a maintenance configuration for accessing a swappable battery and a swappable memory card containing the device log, wherein the unmanned aerial vehicle utilizes the robotic arm to physically swap out the swappable battery with a replacement battery and the swappable memory card with a replacement memory card on the remote device; and

after completing the servicing of the remote device, returning to the base station and transferring contents of the device log to the base station.

20. The method of claim 19 , further comprising:

detecting, by the unmanned aerial vehicle, a distress signal issued by a different remote device that is not being serviced by the unmanned aerial vehicle; and

relaying, by the unmanned aerial vehicle, the distress signal to another unmanned aerial vehicle, another remote device, or the base station, wherein the distress signal is relayed until the distress signal is transmitted to the base station.

US17/467,823

2020-09-11

2021-09-07

Drone-based administration of remotely located instruments and gadgets

Active

2042-04-20

US11978023B2

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US17/467,823

US11978023B2

( en )

2020-09-11

2021-09-07

Drone-based administration of remotely located instruments and gadgets

Applications Claiming Priority (2)

Application Number

Priority Date

Filing Date

Title

US202063077303P

2020-09-11

2020-09-11

US17/467,823

US11978023B2

( en )

2020-09-11

2021-09-07

Drone-based administration of remotely located instruments and gadgets

Publications (2)

Publication Number

Publication Date

US20220083987A1

US20220083987A1 ( en )

2022-03-17

US11978023B2

true

US11978023B2 ( en )

2024-05-07

Family

ID=80626810

Family Applications (1)

Application Number

Title

Priority Date

Filing Date

US17/467,823

Active

2042-04-20

US11978023B2

( en )

2020-09-11

2021-09-07

Drone-based administration of remotely located instruments and gadgets

Country Status (1)

Country

Link

US

( 1 )

US11978023B2

( en )

Families Citing this family (18)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US12379217B2

( en )

2014-11-30

2025-08-05

Raymond Anthony Joao

Personal monitoring apparatus and method

US11765547B2

( en )

2019-07-30

2023-09-19

Raymond Anthony Joao

Personal monitoring apparatus and methods

US11682307B2

( en )

*

2020-07-07

2023-06-20

Honeywell International Inc.

Situation-aware, intelligent data-synchronization methods for UAV-inspection applications

CN120498642A

( en )

*

2020-10-09

2025-08-15

崔云虎

Authentication chain using public key infrastructure

US11405801B1

( en )

*

2021-03-18

2022-08-02

Amazon Technologies, Inc.

Managing radio-based network infrastructure using unmanned vehicles

CN112990146B

( en )

*

2021-05-06

2021-07-27

成都理工大学

An intelligent identification method for abnormal hidden dangers of petroleum pipeline inspection based on UAV

US12154439B2

( en )

*

2021-05-11

2024-11-26

Honeywell International Inc.

Systems and methods for ground-based automated flight management of urban air mobility vehicles

US11755001B2

( en )

*

2021-05-18

2023-09-12

Ford Global Technologies, Llc

Modular systems for industrial machinery

US12556664B2

( en )

*

2021-11-12

2026-02-17

Raymond Anthony Joao

Personal monitoring apparatus and methods

US12307908B2

( en )

*

2022-02-08

2025-05-20

International Business Corporation

Drone deployment for distributed asset maintenance and repair

US12579498B2

( en )

*

2022-02-24

2026-03-17

International Business Machines Corporation

Maintenance aware robot-based delivery system

US12136095B2

( en )

*

2022-04-20

2024-11-05

Dell Products L.P.

Optimized hardware product returns for subscription services

TWI824753B

( en )

*

2022-10-04

2023-12-01

國立陽明交通大學

Charging service system and method thereof

US12346105B2

( en )

2022-10-05

2025-07-01

Skydio, Inc.

Task management for unmanned aerial vehicles

US20240227304A9

( en )

*

2022-10-20

2024-07-11

International Business Machines Corporation

Dynamic sensor printing and deployment

US12192266B2

( en )

2022-10-20

2025-01-07

Skydio, Inc.

Data transfer management for unmanned aerial vehicles

CN116823192A

( en )

*

2023-08-31

2023-09-29

浙江钢信检测技术有限公司

Iron tower intelligent detection system

CN117093020B

( en )

*

2023-10-08

2024-01-23

天津云圣智能科技有限责任公司

Unmanned aerial vehicle centering device, centering control method, computer device and storage medium

Citations (6)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US9421869B1

( en )

*

2015-09-25

2016-08-23

Amazon Technologies, Inc.

Deployment and adjustment of airborne unmanned aerial vehicles

US9659502B1

( en )

*

2015-12-18

2017-05-23

International Business Machines Corporation

Drone range extension via host vehicles

US20180141453A1

( en )

*

2016-11-22

2018-05-24

Wal-Mart Stores, Inc.

System and method for autonomous battery replacement

US20180217382A1

( en )

*

2017-01-27

2018-08-02

Otoy, Inc.

Headphone based modular vr/ar platform

US20190112048A1

( en )

*

2016-03-30

2019-04-18

Matthew CULVER

Systems and methods for unmanned aerial vehicles

US20220390964A1

( en )

*

2021-06-04

2022-12-08

Thomas Andrew Youmans

Cloud &amp; hybrid-cloud flight vehicle &amp; robotic control system ai &amp; ml enabled cloud-based software &amp; data system method for the optimization and distribution of flight control &amp; robotic system solutions and capabilities

2021

2021-09-07

US

US17/467,823

patent/US11978023B2/en

active

Active

Patent Citations (6)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US9421869B1

( en )

*

2015-09-25

2016-08-23

Amazon Technologies, Inc.

Deployment and adjustment of airborne unmanned aerial vehicles

US9659502B1

( en )

*

2015-12-18

2017-05-23

International Business Machines Corporation

Drone range extension via host vehicles

US20190112048A1

( en )

*

2016-03-30

2019-04-18

Matthew CULVER

Systems and methods for unmanned aerial vehicles

US20180141453A1

( en )

*

2016-11-22

2018-05-24

Wal-Mart Stores, Inc.

System and method for autonomous battery replacement

US20180217382A1

( en )

*

2017-01-27

2018-08-02

Otoy, Inc.

Headphone based modular vr/ar platform

US20220390964A1

( en )

*

2021-06-04

2022-12-08

Thomas Andrew Youmans

Cloud &amp; hybrid-cloud flight vehicle &amp; robotic control system ai &amp; ml enabled cloud-based software &amp; data system method for the optimization and distribution of flight control &amp; robotic system solutions and capabilities

Also Published As

Publication number

Publication date

US20220083987A1

( en )

2022-03-17

Similar Documents

Publication

Publication Date

Title

US20220083987A1

( en )

2022-03-17

Drone-based administration of remotely located instruments and gadgets

US9908239B1

( en )

2018-03-06

Mobile robot system

US11462934B2

( en )

2022-10-04

Premises security system with power over Ethernet having an uninterruptible power supply and uninterruptible Internet connectivity

JP7136750B2

( en )

2022-09-13

Unmanned driving vehicle dispatch method, device, equipment and storage medium

US9824592B2

( en )

2017-11-21

Method and apparatus for ensuring the operation and integrity of a three-dimensional integrated logistical system

US20240397350A1

( en )

2024-11-28

Drone wireless communication diagnosis and correction

US11924720B2

( en )

2024-03-05

Autonomous drone with image sensor

JP4521448B2

( en )

2010-08-11

Remote monitoring / diagnosis system

US20230147814A1

( en )

2023-05-11

Method and system for routing/orchestration/management of aerial vehicles or data on a network

US11577857B2

( en )

2023-02-14

Systems and methods for performing remote maintenance

US11860637B2

( en )

2024-01-02

Mobile docking station

US12162135B2

( en )

2024-12-10

Managing shared robots in a data center

US11693410B2

( en )

2023-07-04

Optimizing a navigation path of a robotic device

US11677912B2

( en )

2023-06-13

Robot sensor installation

CN114186919A

( en )

2022-03-15

Method and apparatus for enhanced power notification in a wireless node network

WO2012069814A1

( en )

2012-05-31

System for monitoring and deploying engineers

CN113070906B

( en )

2022-04-26

Service robot system and network fault diagnosis method and device thereof

JPWO2016199251A1

( en )

2017-07-27

Equipment maintenance management system, equipment maintenance apparatus and program

KR20220074563A

( en )

2022-06-03

Multi-robot based security method

CN118485241A

( en )

2024-08-13

Unmanned aerial vehicle-based illumination lamp inspection maintenance system and method

KR20160129662A

( en )

2016-11-09

Self-diagnosis device and device having the same

JP6569247B2

( en )

2019-09-04

Fault verification apparatus, fault verification method, verification target apparatus, wireless communication system, computer program

JP2006338093A

( en )

2006-12-14

Patrol security support system and patrol route management device used therefor

US20230005274A1

( en )

2023-01-05

Security system and monitoring method

US12600568B1

( en )

2026-04-14

Integrated autonomous warehouse robot

Legal Events

Date

Code

Title

Description

2021-09-07

FEPP

Fee payment procedure

Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

2021-09-14

AS

Assignment

Owner name : UNIVERSITY OF FLORIDA RESEARCH FOUNDATION, INC., FLORIDA

Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BHUNIA, SWARUP;CHAKRABORTY, PRABUDDHA;DIZON, REINER;AND OTHERS;SIGNING DATES FROM 20210909 TO 20210913;REEL/FRAME:057473/0126

2021-09-16

FEPP

Fee payment procedure

Free format text : ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

2021-10-16

STPP

Information on status: patent application and granting procedure in general

Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION

2023-06-15

STPP

Information on status: patent application and granting procedure in general

Free format text : NON FINAL ACTION MAILED

2023-09-15

STPP

Information on status: patent application and granting procedure in general

Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

2023-10-03

STPP

Information on status: patent application and granting procedure in general

Free format text : FINAL REJECTION MAILED

2023-12-07

STPP

Information on status: patent application and granting procedure in general

Free format text : RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

2024-01-22

STPP

Information on status: patent application and granting procedure in general

Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

2024-01-24

ZAAB

Notice of allowance mailed

Free format text : ORIGINAL CODE: MN/=.

2024-04-04

STPP

Information on status: patent application and granting procedure in general

Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

2024-04-05

STPP

Information on status: patent application and granting procedure in general

Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

2024-04-17

STCF

Information on status: patent grant

Free format text : PATENTED CASE

Related documents

• Virtual reality system using multiple force arrays for a solver — DeepMotion, Inc. (US10657696B2)#653961
• Augmented reality system with spatialized audio tied to user manipulated … — Magic Leap, Inc. (US11231770B2)#653963
• High-efficiency fuel cell system with carbon dioxide capture assembly and … — Fuelcell Energy, Inc. (US10170782B2)#653965
• Integrated electrochemical cell power generation and electrolysis systems — Czero, Inc. (US20250087711A1)#653966
• Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)#653960