Google Patentsopen access

Autonomous vehicle system — Intel Corporation (US20220126863A1)

ABSTRACT

Abstract

An apparatus comprising at least one interface to receive a signal identifying a second vehicle in proximity of a first vehicle; and processing circuitry to obtain a behavioral model associated with the second vehicle, wherein the behavioral model defines driving behavior of the second vehicle; use the behavioral model to predict actions of the second vehicle; and determine a path plan for the first vehicle based on the predicted actions of the second vehicle.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of and priority from U.S. Provisional Patent Application No. 62/826,955 entitled “Autonomous Vehicle System” and filed Mar. 29, 2019, the entire disclosure of which is incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates in general to the field of computer systems and, more particularly, to computing systems enabling autonomous vehicles.

BACKGROUND

Some vehicles are configured to operate in an autonomous mode in which the vehicle navigates through an environment with little or no input from a driver. Such a vehicle typically includes one or more sensors that are configured to sense information about the environment. The vehicle may use the sensed information to navigate through the environment. For example, if the sensors sense that the vehicle is approaching an obstacle, the vehicle may navigate around the obstacle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified illustration showing an example autonomous driving environment in accordance with at least one embodiment.

FIG. 2 is a simplified block diagram illustrating an example implementation of a vehicle (and corresponding in-vehicle computing system) equipped with autonomous driving functionality in accordance with at least one embodiment.

FIG. 3 illustrates an example portion of a neural network in accordance with certain embodiments in accordance with at least one embodiment.

FIG. 4 is a simplified block diagram illustrating example levels of autonomous driving, which may be supported in various vehicles (e.g., by their corresponding in-vehicle computing systems in accordance with at least one embodiment.

FIG. 5 is a simplified block diagram illustrating an example autonomous driving flow which may be implemented in some autonomous driving systems in accordance with at least one embodiment.

FIG. 6 depicts an example “sense, plan, act” model for controlling autonomous vehicles in accordance with at least one embodiment.

FIG. 7 illustrates a simplified social norm understanding model 700 in accordance with at least one embodiment.

FIG. 8 depicts diagrams illustrating aspects of coordination between vehicles in an environment where at least a portion of the vehicles are semi- or full-autonomous in accordance with at least one embodiment.

FIG. 9 is a block diagram illustrating example information exchange between two vehicles in accordance with at least one embodiment.

FIG. 10 is a simplified block diagram illustrating an example road intersection in accordance with at least one embodiment.

FIG. 11 depicts diagrams illustrating determination of localized behavioral model consensus in accordance with at least one embodiment.

FIG. 12 illustrates an example “Pittsburgh Left” scenario in accordance with at least one embodiment.

FIG. 13 illustrates an example “road rage” scenario by a human-driven vehicle in accordance with at least one embodiment.

FIG. 14 is a simplified block diagram showing an irregular/anomalous behavior tracking model for an autonomous vehicle in accordance with at least one embodiment.

FIG. 15 illustrates a contextual graph that tracks how often a driving pattern occurs in a given context in accordance with at least one embodiment.

FIG. 16 is a flow diagram of an example process of tracking irregular behaviors observed by vehicles in accordance with at least one embodiment.

FIG. 17 is a flow diagram of an example process of identifying contextual behavior patterns in accordance with at least one embodiment.

FIG. 18 illustrates a fault and intrusion detection system for highly automated and autonomous vehicles in accordance with at least one embodiment.

FIG. 19 illustrates an example of a manipulated graphic in accordance with at least one embodiment.

FIG. 20 is a block diagram of a simplified centralized vehicle control architecture for a vehicle according to at least one embodiment.

FIG. 21 is a simplified block diagram of an autonomous sensing and control pipeline in accordance with at least one embodiment.

FIG. 22 is a simplified block diagram illustrating an example x-by-wire architecture of a highly automated or autonomous vehicle in accordance with at least one embodiment.

FIG. 23 is a simplified block diagram illustrating an example safety reset architecture of a highly automated or autonomous vehicle according to at least one embodiment.

FIG. 24 is a simplified block diagram illustrating an example of a general safety architecture of a highly automated or autonomous vehicle according to at least one embodiment.

FIG. 25 is a simplified block diagram illustrating an example operational flow of a fault and intrusion detection system for highly automated and autonomous vehicles according to at least one embodiment.

FIG. 26 is a simplified flowchart that illustrates a high level possible flow of operations associated with a fault and intrusion detection system in accordance with at least one embodiment.

FIG. 27 is a simplified flowchart that illustrates a high level possible flow of operations associated with a fault and intrusion detection system in accordance with at least one embodiment.

FIG. 28A is a simplified flowchart that illustrates a high level possible flow 2800 of operations associated with a fault and intrusion detection system in accordance with at least one embodiment.

FIG. 28B is a simplified flowchart that illustrates a high level possible flow 2850 of additional operations associated with a comparator operation in accordance with at least one embodiment.

FIG. 29 illustrates an example of sensor arrays commonly found on autonomous vehicles in accordance with at least one embodiment.

FIG. 30 illustrates an example of a Dynamic Autonomy Level Detection (“DALD”) System that adapts autonomous vehicle functionalities based on the sensing and processing capabilities available to the vehicle in accordance with at least one embodiment.

FIG. 31 illustrates example positions of two vehicles in accordance with at least one embodiment.

FIG. 32 illustrates an Ackerman model for a vehicle in accordance with at least one embodiment.

FIG. 33 illustrates an example of a vehicle with an attachment in accordance with at least one embodiment.

FIG. 34 illustrates an example of the use of a simple method of tracing the new dimensions of the vehicle incorporating dimensions added by an extension coupled to the vehicle in accordance with at least one embodiment.

FIG. 35 illustrates an example of a vehicle model occlusion compensation flow in accordance with at least one embodiment.

FIGS. 36-37 are block diagrams of exemplary computer architectures that may be used in accordance with at least one embodiment.

DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 is a simplified illustration 100 showing an example autonomous driving environment. Vehicles (e.g., 105 , 110 , 115 , etc.) may be provided with varying levels of autonomous driving capabilities facilitated through in-vehicle computing systems with logic implemented in hardware, firmware, and/or software to enable respective autonomous driving stacks. Such autonomous driving stacks may allow vehicles to self-control or provide driver assistance to detect roadways, navigate from one point to another, detect other vehicles and road actors (e.g., pedestrians (e.g., 135 ), bicyclists, etc.), detect obstacles and hazards (e.g., 120 ), and road conditions (e.g., traffic, road conditions, weather conditions, etc.), and adjust control and guidance of the vehicle accordingly. Within the present disclosure, a “vehicle” may be a manned vehicle designed to carry one or more human passengers (e.g., cars, trucks, vans, buses, motorcycles, trains, aerial transport vehicles, ambulance, etc.), an unmanned vehicle to drive with or without human passengers (e.g., freight vehicles (e.g., trucks, rail-based vehicles, etc.)), vehicles for transporting non-human passengers (e.g., livestock transports, etc.), and/or drones (e.g., land-based or aerial drones or robots, which are to move within a driving environment (e.g., to collect information concerning the driving environment, provide assistance with the automation of other vehicles, perform road maintenance tasks, provide industrial tasks, provide public safety and emergency response tasks, etc.)). In some implementations, a vehicle may be a system configured to operate alternatively in multiple different modes (e.g., passenger vehicle, unmanned vehicle, or drone vehicle), among other examples. A vehicle may “drive” within an environment to move the vehicle along the ground (e.g., paved or unpaved road, path, or landscape), through water, or through the air. In this sense, a “road” or “roadway”, depending on the implementation, may embody an outdoor or indoor ground-based path, a water channel, or a defined aerial boundary. Accordingly, it should be appreciated that the following disclosure and related embodiments may apply equally to various contexts and vehicle implementation examples.

In some implementations, vehicles (e.g., 105 , 110 , 115 ) within the environment may be “connected” in that the in-vehicle computing systems include communication modules to support wireless communication using one or more technologies (e.g., IEEE 802.11 communications (e.g., WiFi), cellular data networks (e.g., 3rd Generation Partnership Project (3GPP) networks, Global System for Mobile Communication (GSM), general packet radio service, code division multiple access (CDMA), etc.), 4G, 5G, 6G, Bluetooth, millimeter wave (mmWave), ZigBee, Z-Wave, etc.), allowing the in-vehicle computing systems to connect to and communicate with other computing systems, such as the in-vehicle computing systems of other vehicles, roadside units, cloud-based computing systems, or other supporting infrastructure. For instance, in some implementations, vehicles (e.g., 105 , 110 , 115 ) may communicate with computing systems providing sensors, data, and services in support of the vehicles' own autonomous driving capabilities. For instance, as shown in the illustrative example of FIG. 1 , supporting drones 180 (e.g., ground-based and/or aerial), roadside computing devices (e.g., 140 ), various external (to the vehicle, or “extraneous”) sensor devices (e.g., 160 , 165 , 170 , 175 , etc.), and other devices may be provided as autonomous driving infrastructure separate from the computing systems, sensors, and logic implemented on the vehicles (e.g., 105 , 110 , 115 ) to support and improve autonomous driving results provided through the vehicles, among other examples. Vehicles may also communicate with other connected vehicles over wireless communication channels to share data and coordinate movement within an autonomous driving environment, among other example communications.

As illustrated in the example of FIG. 1 , autonomous driving infrastructure may incorporate a variety of different systems. Such systems may vary depending on the location, with more developed roadways (e.g., roadways controlled by specific municipalities or toll authorities, roadways in urban areas, sections of roadways known to be problematic for autonomous vehicles, etc.) having a greater number or more advanced supporting infrastructure devices than other sections of roadway, etc. For instance, supplemental sensor devices (e.g., 160 , 165 , 170 , 175 ) may be provided, which include sensors for observing portions of roadways and vehicles moving within the environment and generating corresponding data describing or embodying the observations of the sensors. As examples, sensor devices may be embedded within the roadway itself (e.g., sensor 160 ), on roadside or overhead signage (e.g., sensor</figure-c

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of and priority from U.S. Provisional Patent Application No. 62/826,955 entitled “Autonomous Vehicle System” and filed Mar. 29, 2019, the entire disclosure of which is incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates in general to the field of computer systems and, more particularly, to computing systems enabling autonomous vehicles.

BACKGROUND

Some vehicles are configured to operate in an autonomous mode in which the vehicle navigates through an environment with little or no input from a driver. Such a vehicle typically includes one or more sensors that are configured to sense information about the environment. The vehicle may use the sensed information to navigate through the environment. For example, if the sensors sense that the vehicle is approaching an obstacle, the vehicle may navigate around the obstacle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified illustration showing an example autonomous driving environment in accordance with at least one embodiment.

FIG. 2 is a simplified block diagram illustrating an example implementation of a vehicle (and corresponding in-vehicle computing system) equipped with autonomous driving functionality in accordance with at least one embodiment.

FIG. 3 illustrates an example portion of a neural network in accordance with certain embodiments in accordance with at least one embodiment.

FIG. 4 is a simplified block diagram illustrating example levels of autonomous driving, which may be supported in various vehicles (e.g., by their corresponding in-vehicle computing systems in accordance with at least one embodiment.

FIG. 5 is a simplified block diagram illustrating an example autonomous driving flow which may be implemented in some autonomous driving systems in accordance with at least one embodiment.

FIG. 6 depicts an example “sense, plan, act” model for controlling autonomous vehicles in accordance with at least one embodiment.

FIG. 7 illustrates a simplified social norm understanding model 700 in accordance with at least one embodiment.

FIG. 8 depicts diagrams illustrating aspects of coordination between vehicles in an environment where at least a portion of the vehicles are semi- or full-autonomous in accordance with at least one embodiment.

FIG. 9 is a block diagram illustrating example information exchange between two vehicles in accordance with at least one embodiment.

FIG. 10 is a simplified block diagram illustrating an example road intersection in accordance with at least one embodiment.

FIG. 11 depicts diagrams illustrating determination of localized behavioral model consensus in accordance with at least one embodiment.

FIG. 12 illustrates an example “Pittsburgh Left” scenario in accordance with at least one embodiment.

FIG. 13 illustrates an example “road rage” scenario by a human-driven vehicle in accordance with at least one embodiment.

FIG. 14 is a simplified block diagram showing an irregular/anomalous behavior tracking model for an autonomous vehicle in accordance with at least one embodiment.

FIG. 15 illustrates a contextual graph that tracks how often a driving pattern occurs in a given context in accordance with at least one embodiment.

FIG. 16 is a flow diagram of an example process of tracking irregular behaviors observed by vehicles in accordance with at least one embodiment.

FIG. 17 is a flow diagram of an example process of identifying contextual behavior patterns in accordance with at least one embodiment.

FIG. 18 illustrates a fault and intrusion detection system for highly automated and autonomous vehicles in accordance with at least one embodiment.

FIG. 19 illustrates an example of a manipulated graphic in accordance with at least one embodiment.

FIG. 20 is a block diagram of a simplified centralized vehicle control architecture for a vehicle according to at least one embodiment.

FIG. 21 is a simplified block diagram of an autonomous sensing and control pipeline in accordance with at least one embodiment.

FIG. 22 is a simplified block diagram illustrating an example x-by-wire architecture of a highly automated or autonomous vehicle in accordance with at least one embodiment.

FIG. 23 is a simplified block diagram illustrating an example safety reset architecture of a highly automated or autonomous vehicle according to at least one embodiment.

FIG. 24 is a simplified block diagram illustrating an example of a general safety architecture of a highly automated or autonomous vehicle according to at least one embodiment.

FIG. 25 is a simplified block diagram illustrating an example operational flow of a fault and intrusion detection system for highly automated and autonomous vehicles according to at least one embodiment.

FIG. 26 is a simplified flowchart that illustrates a high level possible flow of operations associated with a fault and intrusion detection system in accordance with at least one embodiment.

FIG. 27 is a simplified flowchart that illustrates a high level possible flow of operations associated with a fault and intrusion detection system in accordance with at least one embodiment.

FIG. 28A is a simplified flowchart that illustrates a high level possible flow 2800 of operations associated with a fault and intrusion detection system in accordance with at least one embodiment.

FIG. 28B is a simplified flowchart that illustrates a high level possible flow 2850 of additional operations associated with a comparator operation in accordance with at least one embodiment.

FIG. 29 illustrates an example of sensor arrays commonly found on autonomous vehicles in accordance with at least one embodiment.

FIG. 30 illustrates an example of a Dynamic Autonomy Level Detection (“DALD”) System that adapts autonomous vehicle functionalities based on the sensing and processing capabilities available to the vehicle in accordance with at least one embodiment.

FIG. 31 illustrates example positions of two vehicles in accordance with at least one embodiment.

FIG. 32 illustrates an Ackerman model for a vehicle in accordance with at least one embodiment.

FIG. 33 illustrates an example of a vehicle with an attachment in accordance with at least one embodiment.

FIG. 34 illustrates an example of the use of a simple method of tracing the new dimensions of the vehicle incorporating dimensions added by an extension coupled to the vehicle in accordance with at least one embodiment.

FIG. 35 illustrates an example of a vehicle model occlusion compensation flow in accordance with at least one embodiment.

FIGS. 36-37 are block diagrams of exemplary computer architectures that may be used in accordance with at least one embodiment.

DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 is a simplified illustration 100 showing an example autonomous driving environment. Vehicles (e.g., 105 , 110 , 115 , etc.) may be provided with varying levels of autonomous driving capabilities facilitated through in-vehicle computing systems with logic implemented in hardware, firmware, and/or software to enable respective autonomous driving stacks. Such autonomous driving stacks may allow vehicles to self-control or provide driver assistance to detect roadways, navigate from one point to another, detect other vehicles and road actors (e.g., pedestrians (e.g., 135 ), bicyclists, etc.), detect obstacles and hazards (e.g., 120 ), and road conditions (e.g., traffic, road conditions, weather conditions, etc.), and adjust control and guidance of the vehicle accordingly. Within the present disclosure, a “vehicle” may be a manned vehicle designed to carry one or more human passengers (e.g., cars, trucks, vans, buses, motorcycles, trains, aerial transport vehicles, ambulance, etc.), an unmanned vehicle to drive with or without human passengers (e.g., freight vehicles (e.g., trucks, rail-based vehicles, etc.)), vehicles for transporting non-human passengers (e.g., livestock transports, etc.), and/or drones (e.g., land-based or aerial drones or robots, which are to move within a driving environment (e.g., to collect information concerning the driving environment, provide assistance with the automation of other vehicles, perform road maintenance tasks, provide industrial tasks, provide public safety and emergency response tasks, etc.)). In some implementations, a vehicle may be a system configured to operate alternatively in multiple different modes (e.g., passenger vehicle, unmanned vehicle, or drone vehicle), among other examples. A vehicle may “drive” within an environment to move the vehicle along the ground (e.g., paved or unpaved road, path, or landscape), through water, or through the air. In this sense, a “road” or “roadway”, depending on the implementation, may embody an outdoor or indoor ground-based path, a water channel, or a defined aerial boundary. Accordingly, it should be appreciated that the following disclosure and related embodiments may apply equally to various contexts and vehicle implementation examples.

In some implementations, vehicles (e.g., 105 , 110 , 115 ) within the environment may be “connected” in that the in-vehicle computing systems include communication modules to support wireless communication using one or more technologies (e.g., IEEE 802.11 communications (e.g., WiFi), cellular data networks (e.g., 3rd Generation Partnership Project (3GPP) networks, Global System for Mobile Communication (GSM), general packet radio service, code division multiple access (CDMA), etc.), 4G, 5G, 6G, Bluetooth, millimeter wave (mmWave), ZigBee, Z-Wave, etc.), allowing the in-vehicle computing systems to connect to and communicate with other computing systems, such as the in-vehicle computing systems of other vehicles, roadside units, cloud-based computing systems, or other supporting infrastructure. For instance, in some implementations, vehicles (e.g., 105 , 110 , 115 ) may communicate with computing systems providing sensors, data, and services in support of the vehicles&#39; own autonomous driving capabilities. For instance, as shown in the illustrative example of FIG. 1 , supporting drones 180 (e.g., ground-based and/or aerial), roadside computing devices (e.g., 140 ), various external (to the vehicle, or “extraneous”) sensor devices (e.g., 160 , 165 , 170 , 175 , etc.), and other devices may be provided as autonomous driving infrastructure separate from the computing systems, sensors, and logic implemented on the vehicles (e.g., 105 , 110 , 115 ) to support and improve autonomous driving results provided through the vehicles, among other examples. Vehicles may also communicate with other connected vehicles over wireless communication channels to share data and coordinate movement within an autonomous driving environment, among other example communications.

As illustrated in the example of FIG. 1 , autonomous driving infrastructure may incorporate a variety of different systems. Such systems may vary depending on the location, with more developed roadways (e.g., roadways controlled by specific municipalities or toll authorities, roadways in urban areas, sections of roadways known to be problematic for autonomous vehicles, etc.) having a greater number or more advanced supporting infrastructure devices than other sections of roadway, etc. For instance, supplemental sensor devices (e.g., 160 , 165 , 170 , 175 ) may be provided, which include sensors for observing portions of roadways and vehicles moving within the environment and generating corresponding data describing or embodying the observations of the sensors. As examples, sensor devices may be embedded within the roadway itself (e.g., sensor 160 ), on roadside or overhead signage (e.g., sensor 165 on sign 125 ), sensors (e.g., 170 , 175 ) attached to electronic roadside equipment or fixtures (e.g., traffic lights (e.g., 130 ), electronic road signs, electronic billboards, etc.), dedicated road side units (e.g., 140 ), among other examples. Sensor devices may also include communication capabilities to communicate their collected sensor data directly to nearby connected vehicles or to fog- or cloud-based computing systems (e.g., 140 , 150 ). Vehicles may obtain sensor data collected by external sensor devices (e.g., 160 , 165 , 170 , 175 , 180 ), or data embodying observations or recommendations generated by other systems (e.g., 140 , 150 ) based on sensor data from these sensor devices (e.g., 160 , 165 , 170 , 175 , 180 ), and use this data in sensor fusion, inference, path planning, and other tasks performed by the in-vehicle autonomous driving system. In some cases, such extraneous sensors and sensor data may, in actuality, be within the vehicle, such as in the form of an after-market sensor attached to the vehicle, a personal computing device (e.g., smartphone, wearable, etc.) carried or worn by passengers of the vehicle, etc. Other road actors, including pedestrians, bicycles, drones, unmanned aerial vehicles, robots, electronic scooters, etc., may also be provided with or carry sensors to generate sensor data describing an autonomous driving environment, which may be used and consumed by autonomous vehicles, cloud- or fog-based support systems (e.g., 140 , 150 ), other sensor devices (e.g., 160 , 165 , 170 , 175 , 180 ), among other examples.

As autonomous vehicle systems may possess varying levels of functionality and sophistication, support infrastructure may be called upon to supplement not only the sensing capabilities of some vehicles, but also the computer and machine learning functionality enabling autonomous driving functionality of some vehicles. For instance, compute resources and autonomous driving logic used to facilitate machine learning model training and use of such machine learning models may be provided on the in-vehicle computing systems entirely or partially on both the in-vehicle systems and some external systems (e.g., 140 , 150 ). For instance, a connected vehicle may communicate with road-side units, edge systems, or cloud-based devices (e.g., 140 ) local to a particular segment of roadway, with such devices (e.g., 140 ) capable of providing data (e.g., sensor data aggregated from local sensors (e.g., 160 , 165 , 170 , 175 , 180 ) or data reported from sensors of other vehicles), performing computations (as a service) on data provided by a vehicle to supplement the capabilities native to the vehicle, and/or push information to passing or approaching vehicles (e.g., based on sensor data collected at the device 140 or from nearby sensor devices, etc.). A connected vehicle (e.g., 105 , 110 , 115 ) may also or instead communicate with cloud-based computing systems (e.g., 150 ), which may provide similar memory, sensing, and computational resources to enhance those available at the vehicle. For instance, a cloud-based system (e.g., 150 ) may collect sensor data from a variety of devices in one or more locations and utilize this data to build and/or train machine-learning models which may be used at the cloud-based system (to provide results to various vehicles (e.g., 105 , 110 , 115 ) in communication with the cloud-based system 150 , or to push to vehicles for use by their in-vehicle systems, among other example implementations. Access points (e.g., 145 ), such as cell-phone towers, road-side units, network access points mounted to various roadway infrastructure, access points provided by neighboring vehicles or buildings, and other access points, may be provided within an environment and used to facilitate communication over one or more local or wide area networks (e.g., 155 ) between cloud-based systems (e.g., 150 ) and various vehicles (e.g., 105 , 110 , 115 ). Through such infrastructure and computing systems, it should be appreciated that the examples, features, and solutions discussed herein may be performed entirely by one or more of such in-vehicle computing systems, fog-based or edge computing devices, or cloud-based computing systems, or by combinations of the foregoing through communication and cooperation between the systems.

In general, “servers,” “clients,” “computing devices,” “network elements,” “hosts,” “platforms”, “sensor devices,” “edge device,” “autonomous driving systems”, “autonomous vehicles”, “fog-based system”, “cloud-based system”, and “systems” generally, etc. discussed herein can include electronic computing devices operable to receive, transmit, process, store, or manage data and information associated with an autonomous driving environment. As used in this document, the term “computer,” “processor,” “processor device,” or “processing device” is intended to encompass any suitable processing apparatus, including central processing units (CPUs), graphical processing units (GPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), tensor processors and other matrix arithmetic processors, among other examples. For example, elements shown as single devices within the environment may be implemented using a plurality of computing devices and processors, such as server pools including multiple server computers. Further, any, all, or some of the computing devices may be adapted to execute any operating system, including Linux, UNIX, Microsoft Windows, Apple OS, Apple iOS, Google Android, Windows Server, etc., as well as virtual machines adapted to virtualize execution of a particular operating system, including customized and proprietary operating systems.

Any of the flows, methods, processes (or portions thereof) or functionality of any of the various components described below or illustrated in the figures may be performed by any suitable computing logic, such as one or more modules, engines, blocks, units, models, systems, or other suitable computing logic. Reference herein to a “module”, “engine”, “block”, “unit”, “model”, “system” or “logic” may refer to hardware, firmware, software and/or combinations of each to perform one or more functions. As an example, a module, engine, block, unit, model, system, or logic may include one or more hardware components, such as a micro-controller or processor, associated with a non-transitory medium to store code adapted to be executed by the micro-controller or processor. Therefore, reference to a module, engine, block, unit, model, system, or logic, in one embodiment, may refers to hardware, which is specifically configured to recognize and/or execute the code to be held on a non-transitory medium. Furthermore, in another embodiment, use of module, engine, block, unit, model, system, or logic refers to the non-transitory medium including the code, which is specifically adapted to be executed by the microcontroller or processor to perform predetermined operations. And as can be inferred, in yet another embodiment, a module, engine, block, unit, model, system, or logic may refer to the combination of the hardware and the non-transitory medium. In various embodiments, a module, engine, block, unit, model, system, or logic may include a microprocessor or other processing element operable to execute software instructions, discrete logic such as an application specific integrated circuit (ASIC), a programmed logic device such as a field programmable gate array (FPGA), a memory device containing instructions, combinations of logic devices (e.g., as would be found on a printed circuit board), or other suitable hardware and/or software. A module, engine, block, unit, model, system, or logic may include one or more gates or other circuit components, which may be implemented by, e.g., transistors. In some embodiments, a module, engine, block, unit, model, system, or logic may be fully embodied as software. Software may be embodied as a software package, code, instructions, instruction sets and/or data recorded on non-transitory computer readable storage medium. Firmware may be embodied as code, instructions or instruction sets and/or data that are hard-coded (e.g., nonvolatile) in memory devices. Furthermore, logic boundaries that are illustrated as separate commonly vary and potentially overlap. For example, a first and second module (or multiple engines, blocks, units, models, systems, or logics) may share hardware, software, firmware, or a combination thereof, while potentially retaining some independent hardware, software, or firmware.

The flows, methods, and processes described below and in the accompanying figures are merely representative of functions that may be performed in particular embodiments. In other embodiments, additional functions may be performed in the flows, methods, and processes. Various embodiments of the present disclosure contemplate any suitable signaling mechanisms for accomplishing the functions described herein. Some of the functions illustrated herein may be repeated, combined, modified, or deleted within the flows, methods, and processes where appropriate. Additionally, functions may be performed in any suitable order within the flows, methods, and processes without departing from the scope of particular embodiments.

With reference now to FIG. 2 , a simplified block diagram 200 is shown illustrating an example implementation of a vehicle (and corresponding in-vehicle computing system) 105 equipped with autonomous driving functionality. In one example, a vehicle 105 may be equipped with one or more processors 202 , such as central processing units (CPUs), graphical processing units (GPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), tensor processors and other matrix arithmetic processors, among other examples. Such processors 202 may be coupled to or have integrated hardware accelerator devices (e.g., 204 ), which may be provided with hardware to accelerate certain processing and memory access functions, such as functions relating to machine learning inference or training (including any of the machine learning inference or training described below), processing of particular sensor data (e.g., camera image data, LIDAR point clouds, etc.), performing certain arithmetic functions pertaining to autonomous driving (e.g., matrix arithmetic, convolutional arithmetic, etc.), among other examples. One or more memory elements (e.g., 206 ) may be provided to store machine-executable instructions implementing all or a portion of any one of the modules or sub-modules of an autonomous driving stack implemented on the vehicle, as well as storing machine learning models (e.g., 256 ), sensor data (e.g., 258 ), and other data received, generated, or used in connection with autonomous driving functionality to be performed by the vehicle (or used in connection with the examples and solutions discussed herein). Various communication modules (e.g., 212 ) may also be provided, implemented in hardware circuitry and/or software to implement communication capabilities used by the vehicle&#39;s system to communicate with other extraneous computing systems over one or more network channels employing one or more network communication technologies. These various processors 202 , accelerators 204 , memory devices 206 , and network communication modules 212 , may be interconnected on the vehicle system through one or more interconnect fabrics or links (e.g., 208 ), such as fabrics utilizing technologies such as a Peripheral Component Interconnect Express (PCIe), Ethernet, OpenCAPI™, Gen-Z™, UPI, Universal Serial Bus, (USB), Cache Coherent Interconnect for Accelerators (CCIX™), Advanced Micro Device™&#39;s (AMD™) Infinity™, Common Communication Interface (CCI), or Qualcomm™&#39;s Centriq™ interconnect, among others.

Continuing with the example of FIG. 2 , an example vehicle (and corresponding in-vehicle computing system) 105 may include an in- vehicle processing system 210 , driving controls (e.g., 220 ), sensors (e.g., 225 ), and user/passenger interface(s) (e.g., 230 ), among other example modules implemented functionality of the autonomous vehicle in hardware and/or software. For instance, an in- vehicle processing system 210 , in some implementations, may implement all or a portion of an autonomous driving stack and process flow (e.g., as shown and discussed in the example of FIG. 5 ). The autonomous driving stack may be implemented in hardware, firmware, or software. A machine learning engine 232 may be provided to utilize various machine learning models (e.g., 256 ) provided at the vehicle 105 in connection with one or more autonomous functions and features provided and implemented at or for the vehicle, such as discussed in the examples herein. Such machine learning models 256 may include artificial neural network models, convolutional neural networks, decision tree-based models, support vector machines (SVMs), Bayesian models, deep learning models, and other example models. In some implementations, an example machine learning engine 232 may include one or more model trainer engines 252 to participate in training (e.g., initial training, continuous training, etc.) of one or more of the machine learning models 256 . One or more inference engines 254 may also be provided to utilize the trained machine learning models 256 to derive various inferences, predictions, classifications, and other results. In some embodiments, the machine learning model training or inference described herein may be performed off-vehicle, such as by computing system

140 or 150 .

The machine learning engine(s) 232 provided at the vehicle may be utilized to support and provide results for use by other logical components and modules of the in- vehicle processing system 210 implementing an autonomous driving stack and other autonomous-driving-related features. For instance, a data collection module 234 may be provided with logic to determine sources from which data is to be collected (e.g., for inputs in the training or use of various machine learning models 256 used by the vehicle). For instance, the particular source (e.g., internal sensors (e.g., 225 ) or extraneous sources (e.g., 115 , 140 , 150 , 180 , 215 , etc.)) may be selected, as well as the frequency and fidelity at which the data may be sampled is selected. In some cases, such selections and configurations may be made at least partially autonomously by the data collection module 234 using one or more corresponding machine learning models (e.g., to collect data as appropriate given a particular detected scenario).

A sensor fusion module 236 may also be used to govern the use and processing of the various sensor inputs utilized by the machine learning engine 232 and other modules (e.g., 238 , 240 , 242 , 244 , 246 , etc.) of the in-vehicle processing system. One or more sensor fusion modules (e.g., 236 ) may be provided, which may derive an output from multiple sensor data sources (e.g., on the vehicle or extraneous to the vehicle). The sources may be homogenous or heterogeneous types of sources (e.g., multiple inputs from multiple instances of a common type of sensor, or from instances of multiple different types of sensors). An example sensor fusion module 236 may apply direct fusion, indirect fusion, among other example sensor fusion techniques. The output of the sensor fusion may, in some cases by fed as an input (along with potentially additional inputs) to another module of the in-vehicle processing system and/or one or more machine learning models in connection with providing autonomous driving functionality or other functionality, such as described in the example solutions discussed herein.

A perception engine 238 may be provided in some examples, which may take as inputs various sensor data (e.g., 258 ) including data, in some instances, from extraneous sources and/or sensor fusion module 236 to perform object recognition and/or tracking of detected objects, among other example functions corresponding to autonomous perception of the environment encountered (or to be encountered) by the vehicle 105 . Perception engine 238 may perform object recognition from sensor data inputs using deep learning, such as through one or more convolutional neural networks and other machine learning models 256 . Object tracking may also be performed to autonomously estimate, from sensor data inputs, whether an object is moving and, if so, along what trajectory. For instance, after a given object is recognized, a perception engine 238 may detect how the given object moves in relation to the vehicle. Such functionality may be used, for instance, to detect objects such as other vehicles, pedestrians, wildlife, cyclists, etc. moving within an environment, which may affect the path of the vehicle on a roadway, among other example uses.

A localization engine 240 may also be included within an in- vehicle processing system 210 in some implementation. In some cases, localization engine 240 may be implemented as a sub-component of a perception engine 238 . The localization engine 240 may also make use of one or more machine learning models 256 and sensor fusion (e.g., of LIDAR and GPS data, etc.) to determine a high confidence location of the vehicle and the space it occupies within a given physical space (or “environment”).

A vehicle 105 may further include a path planner 242 , which may make use of the results of various other modules, such as data collection 234 , sensor fusion 236 , perception engine 238 , and localization engine (e.g., 240 ) among others (e.g., recommendation engine 244 ) to determine a path plan and/or action plan for the vehicle, which may be used by drive controls (e.g., 220 ) to control the driving of the vehicle 105 within an environment. For instance, a path planner 242 may utilize these inputs and one or more machine learning models to determine probabilities of various events within a driving environment to determine effective real-time plans to act within the environment.

In some implementations, the vehicle 105 may include one or more recommendation engines 244 to generate various recommendations from sensor data generated by the vehicle&#39;s 105 own sensors (e.g., 225 ) as well as sensor data from extraneous sensors (e.g., on

sensor devices

115 , 180 , 215 , etc.). Some recommendations may be determined by the recommendation engine 244 , which may be provided as inputs to other components of the vehicle&#39;s autonomous driving stack to influence determinations that are made by these components. For instance, a recommendation may be determined, which, when considered by a path planner 242 , causes the path planner 242 to deviate from decisions or plans it would ordinarily otherwise determine, but for the recommendation. Recommendations may also be generated by recommendation engines (e.g., 244 ) based on considerations of passenger comfort and experience. In some cases, interior features within the vehicle may be manipulated predictively and autonomously based on these recommendations (which are determined from sensor data (e.g., 258 ) captured by the vehicle&#39;s sensors and/or extraneous sensors, etc.

As introduced above, some vehicle implementations may include user/passenger experience engines (e.g., 246 ), which may utilize sensor data and outputs of other modules within the vehicle&#39;s autonomous driving stack to control a control unit of the vehicle in order to change driving maneuvers and effect changes to the vehicle&#39;s cabin environment to enhance the experience of passengers within the vehicle based on the observations captured by the sensor data (e.g., 258 ). In some instances, aspects of user interfaces (e.g., 230 ) provided on the vehicle to enable users to interact with the vehicle and its autonomous driving system may be enhanced. In some cases, informational presentations may be generated and provided through user displays (e.g., audio, visual, and/or tactile presentations) to help affect and improve passenger experiences within a vehicle (e.g., 105 ) among other example uses.

In some cases, a system manager 250 may also be provided, which monitors information collected by various sensors on the vehicle to detect issues relating to the performance of a vehicle&#39;s autonomous driving system. For instance, computational errors, sensor outages and issues, availability and quality of communication channels (e.g., provided through communication modules 212 ), vehicle system checks (e.g., issues relating to the motor, transmission, battery, cooling system, electrical system, tires, etc.), or other operational events may be detected by the system manager 250 . Such issues may be identified in system report data generated by the system manager 250 , which may be utilized, in some cases as inputs to machine learning models 256 and related autonomous driving modules (e.g., 232 , 234 , 236 , 238 , 240 , 242 , 244 , 246 , etc.) to enable vehicle system health and issues to also be considered along with other information collected in sensor data 258 in the autonomous driving functionality of the vehicle 105 .

In some implementations, an autonomous driving stack of a vehicle 105 may be coupled with drive controls 220 to affect how the vehicle is driven, including steering controls (e.g., 260 ), accelerator/throttle controls (e.g., 262 ), braking controls (e.g., 264 ), signaling controls (e.g., 266 ), among other examples. In some cases, a vehicle may also be controlled wholly or partially based on user inputs. For instance, user interfaces (e.g., 230 ), may include driving controls (e.g., a physical or virtual steering wheel, accelerator, brakes, clutch, etc.) to allow a human driver to take control from the autonomous driving system (e.g., in a handover or following a driver assist action). Other sensors may be utilized to accept user/passenger inputs, such as speech detection 292 , gesture detection cameras 294 , and other examples. User interfaces (e.g., 230 ) may capture the desires and intentions of the passenger-users and the autonomous driving stack of the vehicle 105 may consider these as additional inputs in controlling the driving of the vehicle (e.g., drive controls 220 ). In some implementations, drive controls may be governed by external computing systems, such as in cases where a passenger utilizes an external device (e.g., a smartphone or tablet) to provide driving direction or control, or in cases of a remote valet service, where an external driver or system takes over control of the vehicle (e.g., based on an emergency event), among other example implementations.

As discussed above, the autonomous driving stack of a vehicle may utilize a variety of sensor data (e.g., 258 ) generated by various sensors provided on and external to the vehicle. As an example, a vehicle 105 may possess an array of sensors 225 to collect various information relating to the exterior of the vehicle and the surrounding environment, vehicle system status, conditions within the vehicle, and other information usable by the modules of the vehicle&#39;s processing system 210 . For instance, such sensors 225 may include global positioning (GPS) sensors 268 , light detection and ranging (LIDAR) sensors 270 , two-dimensional (2D) cameras 272 , three-dimensional (3D) or stereo cameras 274 , acoustic sensors 276 , inertial measurement unit (IMU) sensors 278 , thermal sensors 280 , ultrasound sensors 282 , bio sensors 284 (e.g., facial recognition, voice recognition, heart rate sensors, body temperature sensors, emotion detection sensors, etc.), radar sensors 286 , weather sensors (not shown), among other example sensors. Such sensors may be utilized in combination to determine various attributes and conditions of the environment in which the vehicle operates (e.g., weather, obstacles, traffic, road conditions, etc.), the passengers within the vehicle (e.g., passenger or driver awareness or alertness, passenger comfort or mood, passenger health or physiological conditions, etc.), other contents of the vehicle (e.g., packages, livestock, freight, luggage, etc.), subsystems of the vehicle, among other examples. Sensor data 258 may also (or instead) be generated by sensors that are not integrally coupled to the vehicle, including sensors on other vehicles (e.g., 115 ) (which may be communicated to the vehicle 105 through vehicle-to-vehicle communications or other techniques), sensors on ground-based or aerial drones 180 , sensors of user devices 215 (e.g., a smartphone or wearable) carried by human users inside or outside the vehicle 105 , and sensors mounted or provided with other roadside elements, such as a roadside unit (e.g., 140 ), road sign, traffic light, streetlight, etc. Sensor data from such extraneous sensor devices may be provided directly from the sensor devices to the vehicle or may be provided through data aggregation devices or as results generated based on these sensors by other computing systems (e.g., 140 , 150 ), among other example implementations.

In some implementations, an autonomous vehicle system 105 may interface with and leverage information and services provided by other computing systems to enhance, enable, or otherwise support the autonomous driving functionality of the device 105 . In some instances, some autonomous driving features (including some of the example solutions discussed herein) may be enabled through services, computing logic, machine learning models, data, or other resour

CLAIMS

Claims ( 24 )

1 .- 27 . (canceled)

28 . An apparatus comprising:

at least one interface to receive a signal identifying a second vehicle in proximity of a first vehicle; and processing circuitry to:

obtain a behavioral model associated with the second vehicle, wherein the behavioral model defines driving behavior of the second vehicle;

use the behavioral model to predict actions of the second vehicle; and

determine a path plan for the first vehicle based on the predicted actions of the second vehicle.

29 . The apparatus of claim 28 , the processing circuitry to determine trustworthiness of the behavioral model associated with the second vehicle prior to using the behavioral model to predict actions of the second vehicle.

30 . The apparatus of claim 29 , wherein determining trustworthiness of the behavioral model comprises verifying a format of the behavioral model.

31 . The apparatus of claim 28 , wherein determining trustworthiness of the behavioral model comprises verifying accuracy of the behavioral model.

32 . The apparatus of claim 31 , wherein verifying accuracy of the behavioral model comprises:

storing inputs provided to at least one machine learning model and corresponding outputs of the at least one machine learning model; and providing the inputs to the behavioral model and comparing outputs of the behavioral model to the outputs of the at least one machine learning model.

33 . The apparatus of claim 31 , wherein verifying accuracy of the behavioral model comprises:

determining expected behavior of the second vehicle according to the behavioral model based on inputs corresponding to observed conditions; observing behavior of the second vehicle corresponding to the observed conditions; and comparing the observed behavior with the expected behavior.

34 . The apparatus of claim 28 , wherein the behavior model associated with the second vehicle corresponds to at least one machine learning model used by the second vehicle to determine autonomous driving behavior of the second vehicle.

35 . The apparatus of claim 28 , wherein the processing circuitry is to communicate with the second vehicle to obtain the behavioral model, wherein communicating with the second vehicle comprises establishing a secure communication session between the first vehicle and the second vehicle, and receiving the behavioral model via communications within the secure communication session.

36 . The apparatus of claim 35 , wherein establishing the secure communication session comprises exchanging tokens between the first and second vehicles, and each token comprises a respective identifier of a corresponding vehicle, a respective public key, and a shared secret value.

37 . The apparatus of claim 28 , wherein the signal comprises a beacon to indicate an identity and position of the second vehicle.

38 . The apparatus of claim 28 , further comprising a transmitter to broadcast a signal to other vehicles in the proximity of the first vehicle to identify the first vehicle to the other vehicles.

39 . The apparatus of claim 28 , wherein the processing circuitry is to initiate communication of a second behavioral model to the second vehicle in an exchange of behavior models including the behavioral model, the second behavioral model defining driving behavior of the first vehicle.

40 . The apparatus of claim 28 , wherein the processing circuitry is to determine whether the behavioral model associated with the second vehicle is in a behavioral model database of the first vehicle, wherein the behavioral model associated with the second vehicle is obtained based on a determination that the behavioral model associated with the second vehicle is not yet in the behavioral model database.

41 . The apparatus of claim 28 , wherein the second vehicle is capable of operating in a human driving mode and the behavior model associated with the second vehicle models characteristics of at least one human driver of the second vehicle during operation of the second vehicle in the human driving mode.

42 . The apparatus of claim 28 , wherein the behavioral model associated with the second vehicle comprises one of a set of behavioral models for the second vehicle, and the set of behavioral models comprises a plurality of scenario-specific behavioral models.

43 . The apparatus of claim 42 , the processing circuitry to:

determine a particular scenario based at least in part on sensor data generated by the first vehicle; determine that a particular behavioral model in the set of behavioral models corresponds to the particular scenario; and use the particular behavioral model to predict actions of the second vehicle based on determining that the particular behavioral model corresponds to the particular scenario.

44 . A vehicle comprising:

a plurality of sensors to generate sensor data; a control system to physically control movement of the vehicle; at least one interface to receive a signal identifying a second vehicle in proximity of the vehicle; and processing circuitry to:

obtain a behavioral model associated with the second vehicle, wherein the behavioral model defines driving behavior of the second vehicle;

use the behavioral model to predict actions of the second vehicle;

determine a path plan for the vehicle based on the predicted actions of the second vehicle and the sensor data; and

communicate with the control system to move the vehicle in accordance with the path plan.

45 . The vehicle of claim 44 , the processing circuitry to determine trustworthiness of the behavioral model associated with the second vehicle prior to using the behavioral model to predict actions of the second vehicle.

46 . The vehicle of claim 45 , wherein determining trustworthiness of the behavioral model comprises verifying accuracy of the behavioral model.

47 . The vehicle of claim 44 , wherein the behavior model corresponds to at least one machine learning model used by the second vehicle to determine autonomous driving behavior of the second vehicle.

48 . The vehicle of claim 44 , wherein the behavioral model associated with the second vehicle comprises one of a set of behavioral models for the second vehicle, and the set of behavioral models comprises a plurality of scenario-specific behavioral models.

49 . A computer-readable medium to store instructions, wherein the instructions, when executed by a machine, cause the machine to:

receive a signal identifying a second vehicle in proximity of a first vehicle; obtain a behavioral model associated with the second vehicle, wherein the behavioral model defines driving behavior of the second vehicle; use the behavioral model to predict actions of the second vehicle; and determine a path plan for the first vehicle based on the predicted actions of the second vehicle.

50 . A method comprising:

receiving a signal identifying a second vehicle in proximity of a first vehicle; obtaining a behavioral model associated with the second vehicle, wherein the behavioral model defines driving behavior of the second vehicle; using the behavioral model to predict actions of the second vehicle; and determining a path plan for the first vehicle based on the predicted actions of the second vehicle.

US17/434,716

2019-03-29

2020-03-27

Autonomous vehicle system

Abandoned

US20220126863A1

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US17/434,716

US20220126863A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

Applications Claiming Priority (3)

Application Number

Priority Date

Filing Date

Title

US201962826955P

2019-03-29

2019-03-29

US17/434,716

US20220126863A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

PCT/US2020/025501

WO2020205648A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

Publications (1)

Publication Number

Publication Date

US20220126863A1

true

US20220126863A1 ( en )

2022-04-28

Family

ID=72666352

Family Applications (4)

Application Number

Title

Priority Date

Filing Date

US17/434,710

Abandoned

US20220126864A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

US17/434,716

Abandoned

US20220126863A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

US17/434,721

Pending

US20220161815A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

US17/434,713

Pending

US20220126878A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

Family Applications Before (1)

Application Number

Title

Priority Date

Filing Date

US17/434,710

Abandoned

US20220126864A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

Family Applications After (2)

Application Number

Title

Priority Date

Filing Date

US17/434,721

Pending

US20220161815A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

US17/434,713

Pending

US20220126878A1

( en )

2019-03-29

2020-03-27

Autonomous vehicle system

Country Status (7)

Country

Link

US

( 4 )

US20220126864A1

( en )

EP

( 5 )

EP3947095B1

( en )

JP

( 5 )

JP2022524932A

( en )

KR

( 4 )

KR20210134634A

( en )

CN

( 5 )

CN113811473A

( en )

DE

( 4 )

DE112020001642T5

( en )

WO

( 4 )

WO2020205629A1

( en )

Cited By (48)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20210043061A1

( en )

*

2014-07-21

2021-02-11

State Farm Mutual Automobile Insurance Company

Methods of facilitating emergency assistance

US20210094576A1

( en )

*

2019-09-30

2021-04-01

Ghost Locomotion Inc.

Determining control operations for an autonomous vehicle

US20210139052A1

( en )

*

2019-11-11

2021-05-13

Beijing Baidu Netcom Science Technology Co., Ltd.

Task scheduling method, apparatus, device, and computer readable storage medium

US20210261125A1

( en )

*

2018-08-02

2021-08-26

Robert Bosch Gmbh

Method for Guiding a Motor Vehicle in an at Least Partly Automated Manner

US20220092981A1

( en )

*

2019-10-17

2022-03-24

Verizon Patent And Licensing Inc.

Systems and methods for controlling vehicle traffic

US20220105956A1

( en )

*

2020-10-06

2022-04-07

Volkswagen Aktiengesellschaft

Vehicle, device, computer program and method for implementation in a vehicle

US20220111870A1

( en )

*

2021-12-22

2022-04-14

Amar Srivastava

Systems and methods for brokering peer-assisted safety models for autonomous and assisted-driving vehicles

US20220198921A1

( en )

*

2020-12-23

2022-06-23

Sensible 4 Oy

Data collection and modeling systems and methods for autonomous vehicles

US20220332337A1

( en )

*

2021-04-15

2022-10-20

Cavh Llc

Vehicle intelligent unit

CN115378653A

( en )

*

2022-07-25

2022-11-22

中国电子科技集团公司第三十研究所

A network security situation awareness and prediction method and system based on LSTM and random forest

US11531865B2

( en )

*

2020-02-28

2022-12-20

Toyota Research Institute, Inc.

Systems and methods for parallel autonomy of a vehicle

US20220410900A1

( en )

*

2019-12-02

2022-12-29

Gm Cruise Holdings Llc

Assertive vehicle detection model generation

US20230027584A1

( en )

*

2021-07-20

2023-01-26

Autobrains Technologies Ltd

Environmental model based on audio

CN116067434A

( en )

*

2023-03-07

2023-05-05

中铁大桥局集团有限公司

A visual installation system and installation method of a large-segment bridge

CN116546458A

( en )

*

2023-05-09

2023-08-04

西安电子科技大学

Internet of vehicles bidirectional multi-hop communication method under mixed traffic scene

CN116665064A

( en )

*

2023-07-27

2023-08-29

城云科技（中国）有限公司

Generation Method of Urban Change Map Based on Generative Distillation and Feature Perturbation and Its Application

CN116811918A

( en )

*

2023-08-17

2023-09-29

宁波博登智能科技有限公司

Automatic driving decision system and method based on context awareness

US20230331254A1

( en )

*

2020-09-02

2023-10-19

Volkswagen Aktiengesellschaft

A Method for Controlling a Driver Assistance System During Operation of a Vehicle

US20230339459A1

( en )

*

2022-04-20

2023-10-26

Gm Cruise Holdings Llc

Safety measurement of autonomous vehicle driving in simulation

KR20230167292A

( en )

*

2022-05-31

2023-12-08

모셔널 에이디 엘엘씨

Motion prediction in an autonomous vehicle using fused synthetic and camera images

US20230399016A1

( en )

*

2022-06-14

2023-12-14

Gm Cruise Holdings Llc

Multi-mode vehicle controller

CN117278328A

( en )

*

2023-11-21

2023-12-22

广东车卫士信息科技有限公司

Data processing method and system based on Internet of vehicles

CN117395726A

( en )

*

2023-12-12

2024-01-12

江西师范大学

A path planning-based mobile edge computing service migration method

DE102022117676A1

( en )

2022-07-14

2024-01-25

Bayerische Motoren Werke Aktiengesellschaft

Method, system and computer program for training a neural network designed to operate a vehicle and for operating a vehicle with a neural network

US20240031468A1

( en )

*

2020-09-11

2024-01-25

Nippon Telegraph And Telephone Corporation

Recommended action selection apparatus, recommended action selection method, and recommended action selection program

DE102022117683A1

( en )

2022-07-14

2024-01-25

Bayerische Motoren Werke Aktiengesellschaft

Method, system and computer program for training a neural network designed to operate a vehicle and for operating a vehicle with a neural network

WO2024051435A1

( en )

*

2022-09-09

2024-03-14

北京极智嘉科技股份有限公司

Robot scheduling method and device

US20240190474A1

( en )

*

2022-12-09

2024-06-13

Plusai, Inc.

Hierarchical Vehicle Action Prediction

US20240210939A1

( en )

*

2022-12-21

2024-06-27

Apollo Autonomous Driving USA LLC

Camera image compression for autonomous driving vehicles

US12062281B1

( en )

*

2022-05-02

2024-08-13

John Gregory Baker

Roadway sight restriction early warning alarm system

US12071161B1

( en )

*

2022-07-06

2024-08-27

Waymo Llc

Intervention behavior prediction

WO2024187273A1

( en )

*

2023-03-10

2024-09-19

LoopX Innovation Inc.

Systems and methods for estimating a state for positioning autonomous vehicles transitioning between different environments

CN118736834A

( en )

*

2024-07-24

2024-10-01

招商局重庆交通科研设计院有限公司

Multi-vehicle interactive joint decision-making method based on roadside computing unit

US20240391464A1

( en )

*

2020-05-20

2024-11-28

State Farm Mutual Automobile Insurance Company

Accounting for region-specific driver profiles when controlling autonomous vehicles

US12168486B2

( en )

2019-12-06

2024-12-17

Hitachi Astemo, Ltd.

Steering holding determination device, steering control device, and steering device

US12168488B2

( en )

2019-12-06

2024-12-17

Hitachi Astemo, Ltd.

Steering holding determination device, steering control device, and steering device

WO2025045555A1

( en )

*

2023-08-30

2025-03-06

Bayerische Motoren Werke Aktiengesellschaft

Method in connection with the movement of vehicles, and a user device, system and computer program product

US12269371B2

( en )

2022-05-30

2025-04-08

Toyota Connected North America, Inc.

In-cabin detection framework

US20250171047A1

( en )

*

2023-11-29

2025-05-29

Fca Us Llc

Real-time road condition monitoring and communication amongst autonomous vehicles for improved safety

US20250171042A1

( en )

*

2023-11-27

2025-05-29

GM Global Technology Operations LLC

Individualized vehicle power consumption profile generation

US20250206351A1

( en )

*

2023-12-22

2025-06-26

Zoox, Inc.

Probabilistic collision detection system

US12397821B2

( en )

*

2023-05-25

2025-08-26

Gm Cruise Holdings Llc

Metrics for modeling construction zones

US12418947B2

( en )

2022-05-30

2025-09-16

Toyota Connected North America, Inc.

Occupant condition detection and response

US12472983B2

( en )

*

2022-03-07

2025-11-18

Hyundai Motor Company

Autonomous vehicle control apparatus and method thereof

CN121050415A

( en )

*

2025-07-24

2025-12-02

国家能源集团东台海上风电有限责任公司

A Method and System for Autonomous Obstacle Avoidance and Dynamic Path Planning of Unmanned Aerial Vehicles Based on Multimodal Perception and Hybrid Intelligent Decision Making

US12535827B2

( en )

2023-12-21

2026-01-27

Zoox, Inc.

Systems and methods for determining driving scenario probability metrics and vehicle performance metrics

US12541212B1

( en )

*

2023-12-21

2026-02-03

Zoox, Inc.

Systems and methods for identifying road segments and determining vehicle performance metrics

US12583461B2

( en )

2023-09-18

2026-03-24

Ford Global Technologies, Llc

Systems and methods for detecting driver behavior

Families Citing this family (738)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20220018906A1

( en )

*

2014-02-27

2022-01-20

Invently Automotive Inc.

Predicting An Outcome Associated With A Driver Of A vehicle

CN107111314B

( en )

*

2014-11-07

2021-10-08

索尼公司

Control system, control method, and storage medium

US20180012492A1

( en )

*

2015-02-06

2018-01-11

Delphi Technologies, Inc.

Method of automatically controlling an autonomous vehicle based on electronic messages from roadside infrastructure or other vehicles

US11231287B2

( en )

*

2016-12-22

2022-01-25

Nissan North America, Inc.

Autonomous vehicle service system

EP3828657A1

( en )

*

2016-12-23

2021-06-02

Mobileye Vision Technologies Ltd.

Navigational system

WO2018125928A1

( en )

2016-12-29

2018-07-05

DeepScale, Inc.

Multi-channel sensor simulation for autonomous control systems

WO2018176000A1

( en )

2017-03-23

2018-09-27

DeepScale, Inc.

Data synthesis for autonomous control systems

US11893393B2

( en )

2017-07-24

2024-02-06

Tesla, Inc.

Computational array microprocessor system with hardware arbiter managing memory requests

US11157441B2

( en )

2017-07-24

2021-10-26

Tesla, Inc.

Computational array microprocessor system using non-consecutive data formatting

US10671349B2

( en )

2017-07-24

2020-06-02

Tesla, Inc.

Accelerated mathematical engine

US11409692B2

( en )

2017-07-24

2022-08-09

Tesla, Inc.

Vector computational unit

US10569784B2

( en )

*

2017-09-28

2020-02-25

Waymo Llc

Detecting and responding to propulsion and steering system errors for autonomous vehicles

US12307350B2

( en )

2018-01-04

2025-05-20

Tesla, Inc.

Systems and methods for hardware-based pooling

US11215984B2

( en )

*

2018-01-09

2022-01-04

Uatc, Llc

Systems and methods for controlling an autonomous vehicle

US11561791B2

( en )

2018-02-01

2023-01-24

Tesla, Inc.

Vector computational unit receiving data elements in parallel from a last row of a computational array

EP3794421A4

( en )

*

2018-05-09

2022-12-21

Cavh Llc

INTELLIGENCE ALLOCATION TRAINING SYSTEMS AND METHODS BETWEEN VEHICLES AND HIGHWAYS

US10467903B1

( en )

*

2018-05-11

2019-11-05

Arnold Chase

Passive infra-red pedestrian detection and avoidance system

US20210247199A1

( en )

*

2018-05-14

2021-08-12

3M Innovative Properties Company

Autonomous navigation systems for temporary zones

US11215999B2

( en )

2018-06-20

2022-01-04

Tesla, Inc.

Data pipeline and deep learning system for autonomous driving

GB2588340B

( en )

2018-06-21

2023-08-09

Sierra Nevada Corp

Net edge composite core splices for aircraft wing

US11167836B2

( en )

2018-06-21

2021-11-09

Sierra Nevada Corporation

Devices and methods to attach composite core to a surrounding structure

JP7199545B2

( en )

2018-07-20

2023-01-05

メイ モビリティー，インコーポレイテッド

A Multi-view System and Method for Action Policy Selection by Autonomous Agents

US10909866B2

( en )

*

2018-07-20

2021-02-02

Cybernet Systems Corp.

Autonomous transportation system and methods

US11361457B2

( en )

2018-07-20

2022-06-14

Tesla, Inc.

Annotation cross-labeling for autonomous control systems

US10768629B2

( en )

*

2018-07-24

2020-09-08

Pony Ai Inc.

Generative adversarial network enriched driving simulation

US10614709B2

( en )

2018-07-24

2020-04-07

May Mobility, Inc.

Systems and methods for implementing multimodal safety operations with an autonomous agent

US11636333B2

( en )

2018-07-26

2023-04-25

Tesla, Inc.

Optimizing neural network structures for embedded systems

US11080337B2

( en )

2018-07-31

2021-08-03

Marvell Asia Pte, Ltd.

Storage edge controller with a metadata computational engine

US11562231B2

( en )

2018-09-03

2023-01-24

Tesla, Inc.

Neural networks for embedded devices

EP3850546A4

( en )

2018-09-10

2022-06-15

Drisk, Inc.

GRAPH-BASED AI TRAINING SYSTEMS AND METHODS

JP7186867B2

( en )

*

2018-09-24

2022-12-09

ロベルト・ボッシュ・ゲゼルシャフト・ミト・ベシュレンクテル・ハフツング

Method and apparatus for monitoring motorcycles

KR102637599B1

( en )

*

2018-10-08

2024-02-19

주식회사 에이치엘클레무브

Apparatus and Method for Controlling Lane Changing using Vehicle-to-Vehicle Communication and Tendency Information Calculation Apparatus therefor

ES3063986T3

( en )

2018-10-11

2026-04-21

Tesla Inc

Systems and methods for training machine models with augmented data

US12330646B2

( en )

2018-10-18

2025-06-17

Autobrains Technologies Ltd

Off road assistance

US11196678B2

( en )

2018-10-25

2021-12-07

Tesla, Inc.

QOS manager for system on a chip communications

US10748038B1

( en )

2019-03-31

2020-08-18

Cortica Ltd.

Efficient calculation of a robust signature of a media unit

US11816585B2

( en )

2018-12-03

2023-11-14

Tesla, Inc.

Machine learning models operating at different frequencies for autonomous vehicles

US11537811B2

( en )

2018-12-04

2022-12-27

Tesla, Inc.

Enhanced object detection for autonomous vehicles based on field view

US11610117B2

( en )

2018-12-27

2023-03-21

Tesla, Inc.

System and method for adapting a neural network model on a hardware platform

US10997461B2

( en )

2019-02-01

2021-05-04

Tesla, Inc.

Generating ground truth for machine learning from time series elements

US11150664B2

( en )

2019-02-01

2021-10-19

Tesla, Inc.

Predicting three-dimensional features for autonomous driving

US11567514B2

( en )

2019-02-11

2023-01-31

Tesla, Inc.

Autonomous and user controlled vehicle summon to a target

US10969470B2

( en )

2019-02-15

2021-04-06

May Mobility, Inc.

Systems and methods for intelligently calibrating infrastructure devices using onboard sensors of an autonomous agent

US10956755B2

( en )

2019-02-19

2021-03-23

Tesla, Inc.

Estimating object properties using visual image data

US11953333B2

( en )

*

2019-03-06

2024-04-09

Lyft, Inc.

Systems and methods for autonomous vehicle performance evaluation

US11694088B2

( en )

*

2019-03-13

2023-07-04

Cortica Ltd.

Method for object detection using knowledge distillation

CN110069064B

( en )

*

2019-03-19

2021-01-29

驭势科技（北京）有限公司

A method for upgrading an automatic driving system, an automatic driving system and an in-vehicle device

US11440471B2

( en )

*

2019-03-21

2022-09-13

Baidu Usa Llc

Automated warning system to detect a front vehicle slips backwards

DE102019204318A1

( en )

*

2019-03-28

2020-10-01

Conti Temic Microelectronic Gmbh

Automatic detection and classification of adversarial attacks

DE102019205520A1

( en )

*

2019-04-16

2020-10-22

Robert Bosch Gmbh

Method for determining driving courses

US11930024B2

( en )

*

2019-04-18

2024-03-12

Oracle International Corporation

Detecting behavior anomalies of cloud users

MX2021012911A

( en )

*

2019-04-25

2022-01-18

Huawei Tech Co Ltd

Recommender system selecting a driver out of multiple candidates.

US11417212B1

( en )

2019-04-29

2022-08-16

Allstate Insurance Company

Risk management system with internet of things

EP3739521A1

( en )

*

2019-05-14

2020-11-18

Robert Bosch GmbH

Training system for training a generator neural network

US11775770B2

( en )

2019-05-23

2023-10-03

Capital One Services, Llc

Adversarial bootstrapping for multi-turn dialogue model training

CN112055364B

( en )

*

2019-06-06

2023-04-07

华为技术有限公司

Network system grading method and device

US11526174B2

( en )

*

2019-06-07

2022-12-13

Tata Consultancy Services Limited

Method and a system for hierarchical network based diverse trajectory proposal

DE102019208735B4

( en )

*

2019-06-14

2021-12-23

Volkswagen Aktiengesellschaft

Method for operating a driver assistance system for a vehicle and a driver assistance system for a vehicle

US11287829B2

( en )

*

2019-06-20

2022-03-29

Cisco Technology, Inc.

Environment mapping for autonomous vehicles using video stream sharing

WO2020262716A1

( en )

*

2019-06-24

2020-12-30

엘지전자 주식회사

Travel method of autonomous vehicle

US20200410368A1

( en )

*

2019-06-25

2020-12-31

International Business Machines Corporation

Extended rule generation

US11645685B2

( en )

*

2019-06-26

2023-05-09

Lyft, Inc.

Dynamically adjusting transportation provider pool size

US11727265B2

( en )

*

2019-06-27

2023-08-15

Intel Corporation

Methods and apparatus to provide machine programmed creative support to a user

US12248852B2

( en )

*

2019-06-28

2025-03-11

Intel Corporation

Decision architecture for autonomous systems </td

Related documents

• Generating augmented reality experiences utilizing physical objects to … — Meta Platforms Technologies, Llc (US20260037214A1)#653960
• Virtual reality system using multiple force arrays for a solver — DeepMotion, Inc. (US10657696B2)#653961
• Tracking position of device inside-out for virtual reality interactivity — Sony Interactive Entertainment America Llc (US10535153B2)#653962
• Augmented reality system with spatialized audio tied to user manipulated … — Magic Leap, Inc. (US11231770B2)#653963
• Big data-based autonomous flight drone system and autonomous flight method … — Rgblab Co., Ltd. (US20210390867A1)#653964