Conceptio › Archive › Google Patents

Google Patentsopen access

Systems and methods for safe and reliable autonomous vehicles — Nvidia Corporation (US20240045426A1)

Nvidia Corporation · Google Patents

Google Patents · Patents · License: Open Access

patent, google patents, intellectual property, US20240045426A1, Nvidia Corporation, Michael Alan DITTY, en, 2024

ABSTRACT

Abstract

Autonomous driving is one of the world's most challenging computational problems. Very large amounts of data from cameras, RADARs, LIDARs, and HD-Maps must be processed to generate commands to control the car safely and comfortably in real-time. This challenging task requires a dedicated supercomputer that is energy-efficient and low-power, complex high-performance software, and breakthroughs in deep learning AI algorithms. To meet this task, the present technology provides advanced systems and methods that facilitate autonomous driving functionality, including a platform for autonomous driving Levels 3, 4, and/or 5. In preferred embodiments, the technology provides an end-to-end platform with a flexible architecture, including an architecture for autonomous vehicles that leverages computer vision and known ADAS techniques, providing diversity and redundancy, and meeting functional safety standards. The technology provides for a faster, more reliable, safer, energy-efficient and space-efficient System-on-a-Chip, which may be integrated into a flexible, expandable platform that enables a wide-range of autonomous vehicles, including cars, taxis, trucks, and buses, as well as watercraft and aircraft.

Description

I. Claim of Priority

This application is a continuation of U.S. patent application Ser. No. 16/186,473, filed Nov. 9, 2018, now U.S. Pat. No. 11,644,834, which claims priority to, and the benefit of U.S. Provisional Patent Application 62/584,549, entitled âSystems and Methods for Safe and Reliable Autonomous Vehiclesâ, filed Nov. 10, 2017, all of which are incorporated herein by reference in their entirety and for all purposes.

II. Abstract

III. Federally Sponsored Research or Development

None.

IV. Applications Incorporated by Reference

The following U.S. patent applications are incorporated by reference herein for all purposes as if expressly set forth:

âProgrammable Vision Acceleratorâ, U.S. patent application Ser. No. 15/141,703 (Attorney Docket Number 15-SC-0128-US02) filed Apr. 28, 2016, still pending. âReliability Enhancement Systems and Methodsâ U.S. patent application Ser. No. 15/338,247 (Attorney Docket Number 15-SC-0356US01) filed Oct. 28, 2016, now U.S. Pat. No. 10,289,469. âMethodology of Using a Single Controller (ECU) For a Fault-Tolerant/Fail-Operational Self-Driving Systemâ, U.S. Provisional Patent Application Ser. No. 62/524,283 (Attorney Docket Number 16-SC-0130-US01) filed on Jun. 23, 2017. âMethod Of Using A Single Controller (ECU) For A Fault-Tolerant/Fail-Operational Self-Driving Systemâ U.S. patent application Ser. No. 15/881,426 (Attorney Docket No. 16-SC-0130US02) filed on Jan. 26, 2018, now U.S. Pat. No. 11,214,273.

V. Background

Many vehicles today include Advanced Driver Assistance Systems (âADASâ), such as automatic lane keeping systems and smart cruise control systems. These systems rely on a human driver to take control of the vehicle in the event of a significant mechanical failures, such as tire blow-outs, brake malfunctions, or unexpected behavior by other drivers.

Driver assistance features, including ADAS and autonomous vehicles, are generally described in terms of automation levels, defined by Society of Automotive Engineers (SAE) âTaxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehiclesâ Standard No. J3016-201806 published on Jun. 15, 2018; Standard No. J3016-201609 published on Sep. 30, 2016, and prior and future versions of this standard; and National Highway Traffic Safety Administration (NHTSA), US Department of Transportation. FIG. 1 illustrates the autonomous driving levels, ranging from driver-only (Level 0), Assisted (Level 1), Partial Automation (Level 2), Conditional Automation (Level 3), High Automation (Level 4) to Full Automation (Level 5). Today's commercially available ADAS systems generally provide only Level 1 or 2 functionality.

A human driver is required to be in the control loop for automation levels 0-2 but is not required for automation levels 3-5. The ADAS system must provide for a human driver to take control within about one second for levels 1 and 2, within several seconds for level 3, and within a couple of minutes for levels 4 and 5. A human driver must stay attentive and not perform other activities while driving during level 0-2, while the driver may perform other, limited activities for automation level 3, and even sleep for automation levels 4 and 5. Level 4 functionality allows the driver to go to sleep, and if any condition such that the car can no longer drive automatically, and the driver does not take over, the car will pull over safely. Level 5 functionality includes robot-taxis, where driverless taxis operate within a city or campus that has been previously mapped.

The success of Level 1 and Level 2 ADAS products, coupled with the promise of dramatic increases in traffic safety and convenience, have driven investments in self-driving vehicle technology. Yet despite that immense investment, no vehicle is available today that provides Level 4 or Level 5 functionality and meets industry safety standards, and autonomous driving remains one of the world's most challenging computational problems. Very large amounts of data from cameras, RADAR, LIDAR, and HD-Maps must be processed to generate commands to control the car safely and comfortably in real-time. Ensuring that cars can react correctly in a fraction of a second to constant- and rapidly-changing circumstances requires interpreting the torrent of data rushing at it from a vast range of sensors, such as cameras, RADAR, LIDAR and ultrasonic sensors. First and foremost, this requires a massive amount of computational horsepower. This challenging task requires a dedicated supercomputer that is energy-efficient and low-power, complex high-performance software, and breakthroughs in deep learning AI algorithms.

In addition, systems for Level 4-5 autonomous vehicles requires a completely different approach to meet industry safety standards, such as the Industry Organization for Standardization (âISOâ) 26262 standard entitled âRoad vehiclesâFunctional safetyâ (2011 en) and future versions and enhancements of this standard, which defines a process for establishing the safety rating of automotive components and equipment. ISO 26262 addresses possible hazards caused by the malfunctioning of electronic and electrical systems in passenger vehicles, determined by the Automotive Safety Integrity Level (âASILâ). ASIL addresses four different risk levels, âAâ, âBâ, âCâ and âDâ, determined by three factors: (1) Exposure (hazard probability), (2) Controllability (by the driver), and (3) Severity (in terms of injuries). The ASIL risk level is roughly defined as the combination of Severity, Exposure, and Controllability. As FIG. 2 illustrates, ISO 26262 âRoad vehiclesâFunctional safetyâPart 9: Automotive Safety Integrity Level (ASIL)-oriented and safety-oriented analysesâ (ISO 26262-9:2011(en)) defines the ASIL âDâ risk as a combination of the highest probability of exposure (E4), the highest possible controllability (C3), and the highest severity (S3). An automotive equipment rated as ASIL âDâ means that the equipment can safely address hazards that pose the most severe risks. A reduction in any one of the Severity, Exposure, and Controllability classifications from its maximum corresponds to a single level reduction in ASIL âAâ, âBâ, âCâ and âDâ ratings.

Basic ADAS systems ( Level 1 or 2) can be easily designed to meet automotive industry functional safety standards, including the ISO 26262 standard, because they rely on the human driver to take over and assert control over the vehicle. For example, if an ADAS system fails, resulting in a dangerous condition, the driver may take command of the vehicle and override that software function and recover to a safe state. Similarly, when the vehicle encounters an environment/situation that the ADAS system cannot adequately control (e.g., tire blow-out, black ice, sudden obstacle) the human driver is expected to take over and perform corrective or mitigating action.

In contrast, Level 3-5 autonomous vehicles require the system, on its own, to be safe even without immediate corrective action from the driver. A fully autonomous vehicle cannot count on a human driver to handle exceptional situationsâthe vehicle's control system, on its own, must identify, manage, and mitigate all faults, malfunctions, and extraordinary operating conditions. Level 4-5 autonomous vehicles have the most rigorous safety requirementsâthey must be designed to handle everything that may go wrong, without relying on any human driver to grab the wheel and hit the brakes. Thus, providing ASIL D level functional safety for Level 4 and Level 5 full autonomous driving is a challenging task. The cost for making a single software sub-system having ASIL D functional safety is cost prohibitive, as ASIL D demands unprecedented precision in design of hardware and software. Another approach is required.

Achieving ASIL D functional safety for Level 4-5 autonomous vehicles requires a dedicated supercomputer that performs all aspects of the dynamic driving task, providing appropriate responses to relevant objects and events, even if a driver does not respond appropriately to a request to resume performance of a dynamic driving task. This ambitious goal requires new System-on-a-Chip technologies, new architectures, and new design approaches.

VI. Some Relevant Art

A. ADAS Systems

Today's ADAS systems include Autonomous/adaptive/automatic cruise control (âACCâ), Forward Crash Warning (âFCWâ), Auto Emergency Braking (âAEBâ), Lane Departure Warning (âLDWâ), Blind Spot Warning (âBSWâ), and Rear Cross-Traffic Warning (âRCTWâ), among others.

ACC can be broadly classified into longitudinal ACC and lateral ACC. Longitudinal ACC monitors and controls the distance to the vehicle immediately ahead of the host or âego vehicleâ. Typical longitudinal ACC systems automatically adjust the vehicle speed to maintain a safe distance from vehicles ahead. Lateral ACC performs distance keeping, and advises the host vehicle to change lanes when necessary. Lateral ACC is related to other ADAS applications such as Lane Change Assist (âLCAâ) and Collision Warning Systems (âCWSâ).

The most common ACC systems use a single RADAR, though other combinations (multiple RADARs, such as one long range RADAR coupled with two short range RADARs, or combinations of LIDAR and cameras) are possible. Longitudinal ACC systems use algorithms that can be divided into two main groups: rule-based and model-based approaches. Rule-based longitudinal ACC approaches use ifâthen rules, which may be executed on any processor, including an FPGA, CPU, or ASIC. The input signals typically include distance to the vehicle ahead, and current speed of vehicle, etc. and the outputs are typically throttle and brake. For example, a longitudinal ACC system may use a rule that is familiar to most drivers: if the distance between the ego car and the car ahead is traversable in less than two seconds, reduce vehicle speed. If the vehicle speed is 88 feet per second (equivalent to 60 miles per hour) and the following distance is 22 feet, the time to traverse that distance is only 0.25 seconds. Under these circumstances, a longitudinal ACC system may reduce speed, by controlling the throttle, and if necessary, the brake. Preferably the throttle is used (reducing throttle will slow the vehicle) but if the distance is small and decreasing, the ACC system may use the brake, or disengage and signal a warning to the driver.

Model-based systems are typically based on proportionalâintegralâderivative controller (âPID controllerâ) or model predictive control (âMPCâ) techniques. Based on the vehicle's position, distance and the speed of the vehicle ahead, the controller optimally calculates the wheel torque taking into consideration driving safety and energy cost.

I. Claim of Priority

II. Abstract

III. Federally Sponsored Research or Development

None.

IV. Applications Incorporated by Reference

The following U.S. patent applications are incorporated by reference herein for all purposes as if expressly set forth:

V. Background

VI. Some Relevant Art

A. ADAS Systems

Cooperative Adaptive Cruise Control (âCACCâ) uses information from other vehicles. This information may be received through an antenna and a modem directly from other vehicles (in proximity), via wireless link, or indirectly, from a network connection. Direct links may be provided by vehicle-to-vehicle (âV2Vâ) communication link, while indirect links are often referred to as infrastructure-to-vehicle (â12Vâ) links. In general, the V2V communication concept provides information about the immediately preceding vehicles (i.e., vehicles immediately ahead of and in the same lane as the ego vehicle), while the 12V communication concept provides information about traffic further ahead. CACC systems can include either or both 12V and V2V information sources. Given the information of the vehicles ahead of the host vehicle, CACC can be more reliable and it has potential to improve traffic flow smoothness and reduce congestion on the road.

ACC systems are in wide use in commercial vehicles today, but often overcompensate or overreact to road conditions. For example, commercial ACC systems may overreact, slowing excessively when a car merges in front, and then regain speed too slowly when the vehicle has moved out of the way. ACC systems have played an important role in providing vehicle safety and driver convenience, but they fall far short of meeting requirements for Level 3-5 autonomous vehicle functionality.

Forward Crash Warning (âFCWâ) ADAS systems are designed to alert the driver to a hazard, so that the driver can take corrective action. Typical FCW ADAS systems use front-facing camera or RADAR sensors, coupled to a dedicated processor, DSP, FPGA, or ASIC that is electrically coupled to driver feedback, such as a display, speaker, or vibrating component. FCW systems typically provide a warning onlyâthey do not take over the vehicle or actuate the brakes or take other corrective action. Rather, when the FCW system detects a hazard, it activates a warning, in the form of a sound, visual warning, vibration and/or a quick brake pulse. FCW systems are in wide use today, but often provide false alerts. According to a 2017 Consumer Reports survey, about 45 percent of the vehicles with FCW experienced at least one false alert, with several modes reporting over 60 percent false alerts. RADAR-based FCW systems are subject to false positives, because RADAR may report the presence of manhole covers, large cans, drainage grates, and other metallic objects, which can be misinterpreted as indicating a vehicle. Like ACC systems, FCW systems have played an important role in providing vehicle safety and driver convenience, but fall far short of meeting requirements for Level 3-5 autonomous vehicle functionality.

Automatic emergency braking (âAEBâ) ADAS systems detect an impending forward collision with another vehicle or other object, and may automatically apply the brakes if the driver does not take corrective action within a specified time or distance parameter. Typical AEB ADAS systems use front-facing camera or RADAR sensors, coupled to a dedicated processor, DSP, FPGA, or ASIC. When the AEB system detects a hazard, it typically first alerts the driver to take corrective action to avoid the collision, similar to a FCW system. If the driver does not take corrective action, the AEB system may automatically apply the brakes in an effort to prevent, or at least mitigate, the impact of the predicted collision. AEB systems, may include techniques such as dynamic brake support (âDBSâ) and/or crash imminent braking (âCIBâ). A DBS system provides a driver-warning, similar to a FCW or typical AEB system. If the driver brakes in response to the warning but the dedicated processor, FPGA, or ASIC determines that the driver's action is insufficient to avoid the crash, the DBS system automatically supplements the driver's braking, attempting to avoid a crash. AEB systems are in wide use today, but have been criticized for oversensitivity, and even undesirable âcorrections.â

Lane-departure warning (âLDWâ) ADAS systems provide visual, audible, and/or tactile warningsâsuch as steering wheel or seat vibrationsâto alert the driver when the car crosses lane markings. A LDW system does not activate when the driver indicates an intentional lane departure, by activating a turn signal. Typical LDW ADAS systems use front-side facing cameras, coupled to a dedicated processor, DSP, FPGA, or ASIC that is electrically coupled to driver feedback, such as a display, speaker, or vibrating component. LDW ADAS systems are in wide use today, but have been criticized for inconsistent performance, at times allowing a vehicle to drift out of a lane and/or toward a shoulder. LDW ADAS systems are also criticized for providing erroneous and intrusive feedback, especially on curvy roads.

Lane-keeping assist (âLKAâ) ADAS systems are a variation of LDW systems. LKA systems provide steering input or braking to correct the vehicle if it starts to exit the lane. LKA systems have been criticized for providing counterproductive controls signals, particularly when the vehicle encounters a bicyclist or pedestrians, especially on narrower roads. In particular, when a driver attempts to give an appropriately wide berth to a cyclist or pedestrian, LKW systems have been known to cause the system to steer the car back toward the center of the lane and thus toward the cyclist or pedestrian.

Blind Spot Warning (âBSWâ) ADAS systems detects and warn the driver of vehicles in an automobile's blind spot. Typical BSW systems provide a visual, audible, and/or tactile alert to indicate that merging or changing lanes is unsafe. The system may provide an additional warning when the driver uses a turn signal. Typical BSW ADAS systems use rear-side facing camera or RADAR sensors, coupled to a dedicated processor, DSP, FPGA, or ASIC that is electrically coupled to driver feedback, such as a display, speaker, or vibrating component. BSW systems are in wide use today, but have been criticized for false positives.

Rear cross-traffic warning (âRCTWâ) ADAS systems provide visual, audible, and/or tactile notification when an object is detected outside the rear camera range when a vehicle is backing up. Some RCTW systems include AEB to ensure that the vehicle brakes are applied to avoid a crash. Typical RCTW ADAS systems use one or more rear-facing RADAR sensor, coupled to a dedicated processor, DSP, FPGA, or ASIC that is electrically coupled to driver feedback, such as a display, speaker, or vibrating component. RCTW systems, like other ADAS systems, have been criticized for false positives.

Prior art ADAS systems have been commercially successful, but none of them provide the functionality needed for Level 3-5 autonomous vehicle performance.

B. Design Approaches.

1. Classical Computer Vision and the Rules-Based Approach

Two distinctly different approaches have been proposed for autonomous vehicles. The first approach, computer vision, is the process of automatically perceiving, analyzing, understanding, and/or interpreting visual data. Such visual data may include any combination of videos, images, real-time or near real-time data captured by any type of camera or video recording device. Computer vision applications implement computer vision algorithms to solve high-level problems. For example, an ADAS system can implement real-time object detection algorithms to detect pedestrians/bikes, recognize traffic signs, and/or issue lane departure warnings based on visual data captured by an in-vehicle camera or video recording device.

Traditional computer vision approaches attempt to extract specified features (such as edges, corners, color) that are relevant for the given task. A traditional computer vision approach includes an object detector, which performs feature detection based on heuristics hand-tuned by human engineers. Pattern-recognition tasks typically use an initial-stage feature extraction stage, followed by a classifier.

Classic computer vision is used in many ADAS applications, but is not well-suited to Level 3-5 system performance. Because classic computer vision follows a rules-based approach, an autonomous vehicle using classic computer vision must have a set of express, programmed decision guidelines, intended to cover all possible scenarios. Given the enormous number of driving situations, environments, and objects, classic computer vision cannot solve the problems that must be solved to arrive at Level 3-5 autonomous vehicles. No system has been able to provide rules for every possible scenario and all driving challenges, including snow, ice, heavy rain, big open parking lots, pedestrians, reflections, merging into oncoming traffic, and the like.

2. Neural Networks and Autonomous Vehicles

Neural networks are widely viewed as an alternative approach to classical computer vision. Neural networks have been proposed for autonomous vehicles for many years, beginning with Pomerleau's Autonomous Land Vehicle in a Neural Network (âALVINNâ) system research in 1989.

ALVINN inspired the Defense Advanced Research Projects Agency (âDARPAâ) seedling project in 2004 known as DARPA Autonomous Vehicle (âDAVEâ), in which a sub-scale radio-controlled car drove through a junk-filled alley way. DAVE was trained on hours of human driving in similar, but not identical, environments. The training data included video from two cameras and the steering commands sent by a human operator. DAVE demonstrated the potential of neural networks, but DAVE's performance was not sufficient to meet the requirements of Level 3-5 autonomous vehicles. To the contrary, DAVE's mean distance between crashes was about 20 meters in complex environments.

After DAVE, two developments spurred further research in neural networks. First, large, labeled data sets such as the ImageNet Large Scale Visual Recognition Challenge (âILSVRCâ) became widely available for training and validation. The ILSRVC data-set contains over ten million images in over 1000 categories.

Second, neural networks are now implemented on massively parallel graphics processing units (âGPUsâ), tremendously accelerating learning and inference ability. The term âGPUâ is a legacy term, but does not imply that the GPUs of the present technology are, in fact, used for graphics processing. To the contrary, the GPUs described herein are domain specific, parallel processing accelerators. While a CPU typically consists of a few cores optimized for sequential serial processing, a GPU typically has a massively parallel architecture consisting of thousands of smaller, more efficient computing cores designed for handling multiple tasks simultaneously. GPUs are used for many purposes beyond graphics, including to accelerate high performance computing, deep learning and artificial intelligence, analytics, and other engineering applications.

GPUs are ideal for deep learning and neural networks. GPUs perform an extremely large number of simultaneous calculations, cutting the time that it takes to train neural networks to just hours, from days with conventional CPU technology.

Deep neural networks are largely âblack boxes,â comprised of millions of nodes and tuned over time. A DNN's decisions can be difficult if not impossible to interpret, making troubleshooting and refinement challenging. With deep learning, a neural network learns many levels of abstraction. They range from simple concepts to complex ones. Each layer categorizes information. It then refines it and passes it along to the next. Deep learning stacks the layers, allowing the machine to learn a âhierarchical representation.â For example, a first layer might look for edges. The next layer may look for collections of edges that form angles. The next might look for patterns of edges. After many layers, the neural network learns the concept of, say, a pedestrian crossing the street.

FIG. 3 illustrates the training of a neural network to recognize traffic signs. The neural network is comprised of an input layer ( 6010 ), a plurality of hidden layers ( 6020 ), and an output layer ( 6030 ). Training image information ( 6000 ) is input into nodes ( 300 ), and propagates forward through the network. The correct result ( 6040 ) is used to adjust the weights of the nodes ( 6011 , 6021 , 6031 ), and the process is used for thousands of images, each resulting in revised weights. After sufficient training, the neural network can accurately identify images, with even greater precision than humans.

C. NVIDIA's Parker SoC and Drive PX Platforms.

GPUs have demonstrated that a CNN could be used to steer a car when properly trained. A Level 3-5 autonomous vehicle must make numerous instantaneous decisions to navigate the environment. These choices are far more complicated than the lane-following and steering applications of the early ALVINN, and DAVE systems.

Following its early work, NVIDIA adapted a System-on-a-Chip called Parkerâinitially designed for mobile applicationsâfor a controller for a self-driving system called DRIVETMPX 2. The DRIVETMPX 2 platform with Parker supported Autochauffeur and AutoCruise functionality.

To date, no company has successfully built an autonomous driving system for Level 4-5 functionality capable of meeting industry safety standards. It is a daunting task. It has not been done successfully before, and requires numerous technologies spanning different architectural, hardware, and software-based systems. Given infinite training and computing power, all decision-making canâat least theoreticallyâbe handled best with deep learning methodologies. The autonomous vehicle would not need to be programmed with explicit rules, but rather, would be operated by a neural network trained with massive amounts of data depicting every possible driving scenario and the proper outcome. The autonomous vehicle would have the benefit of an infinite collective experience, and would be far more skilled at driving than the average human driver. The collective experiences would, in theory, also include localized information regarding local driving customsâsome driving conventions are informal, parochial, and known to locals rather than being codified in traffic laws.

But a single, unified neural network likely cannot make every decision necessary for driving. Many different AI neural networks, combined with traditional technologies, are necessary to operate the vehicle. Using a variety of AI networks, each responsible for an area of expertise, will increase safety and reliability in autonomous vehicles. In addition to a network that controls steering, autonomous vehicles must have networks trained and focused on specific tasks like pedestrian detection, lane detection, sign reading, collision avoidance and many more. Even if a single combination of neural networks could achieve Level 3-5 functionality, the âblack boxâ nature of neural networks makes achieving ASIL D functionality impractical.

VII. Summary

What is needed to solve the problems in existing autonomous driving approaches is an end-to-end platform with one flexible architecture that spans Level 3-5âa comprehensive functional safety architecture that leverages and makes efficient use of computer vision and/or ADAS techniques for diversity and redundancy, and provides a platform for a flexible, reliable driving software stack, along with deep learning tools. What is needed is a faster, more reliable, and even more energy-efficient and space-efficient SoC, integrated into a flexible, expandable platform that enables a wide range of autonomous vehicles, including cars, taxis, trucks, buses, and other vehicles. What is needed is a system that can provide safe, reliable, and comfortable autonomous driving, without the false positives and oversensitivity that have plagued commercial ADAS systems.

Embodiments include systems and methods that facilitate autonomous driving functionality for Levels 3, 4, and/or 5. In some example embodiments herein, conditional, high and

full automation levels

3, 4, and 5 are maintained even when a processor component fails. The technology further provides an end-to-end platform with a flexible architecture that provides a comprehensive functional safety architecture that leverages and makes efficient use of computer vision and/or ADAS techniques for diversity and redundancy, provides a platform for a flexible, reliable driving software stack, along with deep learning tools. The example non-limiting technology herein provides a faster, reliable, energy-efficient and space-efficient SoC, integrated into a flexible, expandable platform that enables a wide range of autonomous vehicles, including cars, taxis, trucks, buses, and other vehicles.

VIII. Brief Description of the Drawings

FIG. 1 is a diagram illustrating Levels of Driver Assistance, ADAS, and Autonomous Driving, in accordance with embodiments of the present technology.

FIG. 2 presents a table of example factors for determining ASIL risk, in accordance with embodiments of the present technology.

FIG. 3 is a diagram of an example data flow for training neural networks to recognize objects, in accordance with embodiments of the present technology.

FIG. 4 is a diagram of an example autonomous vehicle, in accordance with embodiments of the present technology.

FIG. 5 is diagram of example camera types and locations on a vehicle, in accordance with embodiments of the present technology.

FIG. 6 is an illustration of an example data flow process for communication between a cloud-based datacenter and an autonomous vehicle, in accordance with embodiments of the present technology.

FIG. 7 is a block diagram illustrating an example autonomous driving hardware platform, in accordance with embodiments of the present technology.

FIG. 8 is a block diagram illustrating an example processing architecture for an advanced System-on-a-Chip (SoC) in an autonomous vehicle, in accordance with embodiments of the present technology.

FIG. 9 Is a component diagram of an example advanced SoC in an autonomous vehicle, in accordance with embodiments of the present technology.

FIG. 10 is a block diagram of an example Programmable Vision Accelerator (PVA), in accordance with embodiments of the present technology.

FIG. 11 is a diagram of an example Hardware Acceleration Cluster Memory architecture, in accordance with embodiments of the present technology.

FIG. 12 is a diagram depicting an example configuration of multiple Neural Networks running on a Deep Learning Accelerator (DLA) to interpret traffic signals, in accordance with embodiments of the present technology.

FIG. 13 is a system diagram of an example advanced SoC architecture for controlling an autonomous vehicle, in accordance with embodiments of the present technology.

FIG. 14 presents a table of example non-limiting ASIL Requirements, in accordance with embodiments of the present technology.

FIG. 15 depicts a block diagram of functional safety features in an advanced SoC, in accordance with embodiments of the present technology.

FIG. 16 depicts an example hardware platform with three SoCs, in accordance with embodiments of the present technology.

FIG. 17 depicts an example hardware platform architecture, in accordance with embodiments of the present technology.

FIG. 18 depicts an example hardware platform architecture including a CPU, in accordance with embodiments of the present technology.

FIG. 19 depicts an alternate example hardware platform architecture that includes a CPU, in accordance with embodiments of the present technology.

FIG. 20 depicts an example hardware platform architecture with communication interfaces, in accordance with embodiments of the present technology.

FIG. 21 is a system diagram for an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 22 is an example architecture of an autonomous driving system that includes eight advanced SoCs and discrete GPUs (dGPUs), in accordance with embodiments of the present technology.

FIG. 23 is an example architecture of an autonomous driving system that includes an advanced SoC and four dGPUs, in accordance with embodiments of the present technology.

FIG. 24 is a block diagram of a high-level system architecture with allocated ASILs, in accordance with embodiments of the present technology.

FIG. 25 is a block diagram of example data flow during an arbitration procedure, in accordance with embodiments of the present technology.

FIG. 26 depicts an example system architecture with allocated ASILs, in accordance with embodiments of the present technology.

FIG. 27 depicts an example configuration of an advanced ADAS system, in accordance with embodiments of the present technology.

FIG. 28 depicts an example virtual machine configuration for autonomous driving applications, in accordance with embodiments of the present technology.

FIG. 29 depicts an example allocation of applications on virtual machines in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 30 illustrates an example workflow for performing compute instructions with preemption, in accordance with embodiments of the present technology.

FIG. 31 depicts an example configuration of partitioning services for functional safety in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 32 depicts an example communication and security system architecture in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 33 depicts an example software stack corresponding to a hardware infrastructure in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 34 depicts an example configuration with functional safety features in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 35 depicts an example interaction topology for virtual machine applications in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 36 is a flowchart for monitoring errors in a Guest Operating system executing in a virtual machine in an autonomous driving system, in accordance with embodiments of the present technology.

FIG. 37 depicts an example error reporting procedure in a safety framework for errors detected in system components, in accordance with embodiments of the present technology.

FIG. 38 is an example flow diagram for error handling during a hardware detection case, in accordance with embodiments of the present technology.

FIG. 39 is an example flow diagram for error handling during a software detection case, in accordance with embodiments of the present technology.

FIG. 40 depicts an example configuration of partitions corresponding to peripheral components, in accordance with embodiments of the present technology.

FIG. 41 is an example software system diagram for autonomous driving, in accordance with embodiments of the present technology.

FIG. 42 is another example software system diagram for autonomous driving, in accordance with embodiments of the present technology.

FIG. 43 depicts an example tracked lane graph, in accordance with embodiments of the present technology.

FIG. 44 depicts an example annotation of valid path as input for training a neural network to perform lane detection, in accordance with embodiments of the present technology.

FIG. 45 depicts an example output from detecting virtual landmarks for performing sensor calibration, in accordance with embodiments of the present technology.

FIG. 46 depicts in an example output from a point detector for tracking features over multiple images produced from sensors, in accordance with embodiments of the present technology.

FIG. 47 depicts an example output from performing iterative closest point alignment between frames with spatial separation generated by a LIDAR sensor, in accordance with embodiments of the present technology.

FIG. 48 is a block diagram of an example automated self-calibrator, in accordance with embodiments of the present technology.

FIG. 49 is a block diagram of an example trajectory estimator, in accordance with embodiments of the present technology.

FIG. 50 depicts example pixel-wise class output images and bounding boxes, in accordance with embodiments of the present technology.

FIG. 51 depicts example output from object tracking, in accordance with embodiments of the present technology.

FIG. 52 depicts an example output from performing a process for determining a temporal baseline based on a range of relative motion, in accordance with embodiments of the present technology.

FIG. 53 depicts an example output from performing a process for heuristically redefining the ground plane, in accordance with embodiments of the present technology.

FIG. 54 depicts an example output from performing mapping on RADAR and vision tracks, in accordance with embodiments of the present technology.

FIG. 55 depicts an example dynamic occupancy grid, in accordance with embodiments of the present technology.

FIG. 56 depicts an example path perception scenario, in accordance with embodiments of the present technology.

FIG. 57 depicts an example scenario for performing in-path determination, in accordance with embodiments of the present technology.

FIG. 58 depicts an example wait condition scenario, in accordance with embodiments of the present technology.

FIG. 59 depicts an example map perception scenario, in accordance with embodiments of the present technology.

FIG. 60 depicts an example directed graph with points and tangents at each node, in accordance with embodiments of the present technology.

FIG. 61 depicts an example directed graph with wait conditions, in accordance with embodiments of the present technology.

FIG. 62 depicts an example representational view of a schematic for displaying additional definitional information in a directed graph, in accordance with embodiments of the present technology.

FIG. 63 depicts an example planning hierarchy, in accordance with embodiments of the present technology.

FIG. 64 depicts an example output from mapping a planned trajectory from a forward prediction model to a trajectory achieved by a controller module, in accordance with embodiments of the present technology.

FIG. 65 depicts an example truck capable of autonomous driving, in accordance with embodiments of the present technology.

FIG. 66 depicts an example two-level bus capable of autonomous driving, in accordance with embodiments of the present technology.

FIG. 67 depicts an example articulated bus capable of autonomous driving, in accordance with embodiments of the present technology.

FIG. 68 depicts an

CLAIMS

Claims ( 20 )

1 . A system-on-a-chip for an autonomous vehicle including:

at least one central processing unit (CPU) cluster or CPU complex supporting virtualization, wherein the CPU cluster or CPU complex includes multiple CPU cores and associated caches, at least one graphics processing unit (GPU) providing multi-core parallel processing, an embedded hardware accelerator cluster, and at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing instructions that when executed by the system-on-the-chip, configure the system-on-a-chip to operate as an autonomous vehicle controller configured to receive optical sensor data, wherein the at least one CPU cluster or complex, the at least one GPU providing multi-core parallel processing, and the at least embedded hardware accelerator cluster comprising the system-on-a-chip, interoperate to process at least the received optical sensor data to perform autonomous driving.

2 . The system-on-a-chip of claim 1 wherein the system-on-a-chip is configured to enable the autonomous vehicle controller to be substantially compliant with level 5 full autonomous driving as defined by SAE specification J3016.

3 . The system-on-a-chip of claim 1 wherein the system-on-a-chip is configured to enable the autonomous vehicle controller to be substantially compliant with integrity level âDâ defined by ISO Standard 26262.

4 . A system-on-a-chip for use in an autonomous vehicle including:

at least one central processing unit (CPU), at least one programmable graphics processing unit (GPU) providing parallel processing and configured to use a tensor instruction set including mixed-precision processing cores partitioned into multiple processing blocks, at least one programmable vision accelerator and/or at least one deep learning accelerator, and at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing program code that when executed by system-on-the-chip, configures the system-on-a-chip to operate as an autonomous vehicle controller configured to receive optical sensor data, wherein the at least one CPU, the at least one GPU, and the at least one programmable vision accelerator and/or the at least one deep learning accelerator, interoperate to process the optical sensor data and a trajectory estimation and/or route plan to provide autonomous driving control of an automobile.

5 . The system-on-a-chip of claim 4 wherein the at least one CPU, the at least one GPU, and the at least one programmable vision accelerator and/or the at least one deep learning accelerator are structured and interconnected to be substantially compliant with integrity level âDâ defined by Standard 26262 of the International Organization for Standardization.

6 . The system-on-a-chip of claim 4 wherein the system-on-a-chip includes at least one memory device connected to the system-on-a-chip, the memory device storing instructions that when executed by the CPU and/or the GPU provides autonomous vehicle control that is substantially compliant with level 5 full autonomous driving as defined by SAE specification J3016.

7 . The system-on-a-chip of claim 4 wherein the at least one GPU providing parallel processing is programmable and the deep learning accelerator comprises a tensor processing unit configured to execute the neural networks based on a tensor instruction set.

8 . The system-on-a-chip of claim 4 wherein the at least one GPU is power-optimized for performance in automotive embedded use applications.

9 . The system-on-a-chip of claim 4 wherein the at least one GPU is fabricated on a FinFET (Fin field effect transistor) high-performance manufacturing process.

10 . A system-on-a-chip for an autonomous vehicle including:

at least one central processing unit (CPU), at least one graphics processing unit (GPU) providing parallel processing, a cache available to both the at least one CPU and the at least one GPU, an embedded hardware accelerator cluster comprising at least one hardware-based accelerator configured to accelerate neural networks and/or accelerate programmable vision; and at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing code that when executed by the system-on-the-chip, configures the system-on-a-chip to operate as an autonomous vehicle controller configured to receive sensor data, wherein the at least one CPU, the at least one GPU providing parallel processing, and the at least one hardware-based accelerator interoperate to process the received sensor data to perform autonomous driving.

11 . A system-on-a-chip for an autonomous vehicle including:

at least one central processing unit (CPU), at least one graphics processing unit (GPU) providing parallel processing cores, an embedded hardware accelerator cluster comprising at least one hardware-based accelerator, and at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing program instructions that when executed by the system-on-the-chip, configure the system-on-a-chip to operate as an autonomous vehicle controller configured to receive sensor data, wherein the at least one CPU, the at least one GPU providing parallel processing cores, and the at least one hardware-based accelerator interoperate to process at least the received sensor data to perform autonomous driving of an automobile.

12 . The system-on-a-chip of claim 11 wherein the at least hardware-based accelerator includes one or more tensor processing units.

13 . The system-on-a-chip of claim 12 wherein the one or more tensor processing units are configured for supporting INT8/INT16/FP16 data type for both features and weights.

14 . The system-on-a-chip of claim 11 wherein the at least one accelerator is configured to accelerate computer vision algorithms for autonomous driving.

15 . The system-on-a-chip of claim 11 wherein the at least one GPU is power-optimized for performance in automotive embedded use applications.

16 . The system-on-a-chip of claim 11 wherein the at least one GPU is fabricated on a FinFET (Fin field effect transistor) high-performance manufacturing process.

17 . A system-on-a-chip for an autonomous vehicle including:

at least one central processing unit (CPU) supporting virtualization, at least one graphics processing unit (GPU) providing parallel processing, a cache memory available to both the at least one CPU and the at least one GPU, at least one accelerator, and at least one memory device interface structured to connect the system-on-a-chip to at least one memory device storing instructions that when executed by the system-on-the-chip, configure the system-on-a-chip to operate as an autonomous vehicle controller configured to receive LIDAR sensor data, wherein the at least one CPU, the at least one GPU providing parallel processing, and the at least one accelerator interoperate to process the LIDAR sensor data and a trajectory estimation and/or route planning to provide autonomous driving.

18 . The system-on-a-chip of claim 17 wherein the accelerator is configured to accelerate computer vision algorithms for autonomous driving.

19 . The system-on-a-chip of claim 17 wherein the accelerator is configured for deep neural network acceleration.

20 . The system-on-a-chip of claim 17 wherein the system-on-a-chip is configured to comprise at least a part of an autonomous vehicle controller.

US18/143,360

2017-11-10

2023-05-04

Systems and methods for safe and reliable autonomous vehicles

Pending

US20240045426A1

( en )

Priority Applications (1)

Application Number

Priority Date

Filing Date

Title

US18/143,360

US20240045426A1

( en )

2017-11-10

2023-05-04

Systems and methods for safe and reliable autonomous vehicles

Applications Claiming Priority (3)

Application Number

Priority Date

Filing Date

Title

US201762584549P

2017-11-10

US16/186,473

US11644834B2

( en )

2017-11-10

2018-11-09

Systems and methods for safe and reliable autonomous vehicles

US18/143,360

US20240045426A1

( en )

2017-11-10

2023-05-04

Systems and methods for safe and reliable autonomous vehicles

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US16/186,473

Continuation

US11644834B2

( en )

2017-11-10

2018-11-09

Systems and methods for safe and reliable autonomous vehicles

Publications (1)

Publication Number

Publication Date

US20240045426A1

true

US20240045426A1 ( en )

2024-02-08

Family

ID=64664395

Family Applications (3)

Application Number

Title

Priority Date

Filing Date

US16/186,473

Active

2040-03-11

US11644834B2

( en )

2017-11-10

2018-11-09

Systems and methods for safe and reliable autonomous vehicles

US18/076,543

Pending

US20230176577A1

( en )

2017-11-10

2022-12-07

Systems and methods for safe and reliable autonomous vehicles

US18/143,360

Pending

US20240045426A1

( en )

2017-11-10

2023-05-04

Systems and methods for safe and reliable autonomous vehicles

Family Applications Before (2)

Application Number

Title

Priority Date

Filing Date

US16/186,473

Active

2040-03-11

US11644834B2

( en )

2017-11-10

2018-11-09

Systems and methods for safe and reliable autonomous vehicles

US18/076,543

Pending

US20230176577A1

( en )

2017-11-10

2022-12-07

Systems and methods for safe and reliable autonomous vehicles

Country Status (5)

Country

Link

( 3 )

US11644834B2

( en )

( 1 )

EP3707572B1

( en )

( 1 )

JP7346401B2

( en )

( 1 )

CN111587407B

( en )

( 1 )

WO2019094843A1

( en )

Cited By (14)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US20220227340A1

( en )

2019-06-07

2022-07-21

Mando Corporation

Control device of brake system

US20230085098A1

( en )

2021-09-10

2023-03-16

Transportation Ip Holdings, Llc

Vehicle Network Monitoring System

US20240192379A1

( en )

2018-04-23

2024-06-13

Aurora Operations, Inc.

Lidar system for autonomous vehicle

US20240416931A1

( en )

2023-06-15

2024-12-19

Volkswagen Aktiengesellschaft

Method for checking an automated driving vehicle prior to starting a drive, and automated driving vehicle

US12282528B1

( en )

2024-12-19

2025-04-22

Digital Global Systems, Inc.

Systems and methods of sensor data fusion

US12461239B2

( en )

2016-11-30

2025-11-04

Aurora Operations, Inc.

Method and system for doppler detection and doppler correction of optical chirped range detection

US12461203B2

( en )

2019-01-04

2025-11-04

Aurora Operations, Inc.

LIDAR system

WO2025232996A1

( en )

2024-05-07

2025-11-13

Bayerische Motoren Werke Aktiengesellschaft

Device and method for checking the plausibility of a position of at least one object, said position being based on the image data of an interior camera provided in the interior of a vehicle

US12481032B2

( en )

2019-01-04

2025-11-25

Aurora Operations, Inc.

Systems and methods for refractive beam-steering

US12479105B2

( en )

2024-12-19

2025-11-25

Digital Global Systems, Inc

Systems and methods of sensor data fusion

US12487564B2

( en )

2024-12-19

2025-12-02

Digital Global Systems, Inc.

Systems and methods of sensor data fusion

EP4685516A1

( en )

2024-07-24

2026-01-28

Iveco S.P.A.

System and method for dimensional control of vehicular systems on a working vehicle and a related working vehicle

WO2026039520A1

( en )

2024-08-13

2026-02-19

Demetrius Thompson

Safe driving system generating map points

KR102948046B1

( en )

2024-09-26

2026-04-03

ë§ì´í¬ë¡ë ìì¤íì£¼ìíì¬

A Smart Distributed Emergency Alert Broadcasting System and Method Thereof

Families Citing this family (722)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US10514837B1

( en )

2014-01-17

2019-12-24

Knightscope, Inc.

Systems and methods for security data analysis and display

US10279488B2

( en )

2014-01-17

2019-05-07

Knightscope, Inc.

Autonomous data machines and systems

JP6524943B2

( en )

2016-03-17

2019-06-05

æ ªå¼ä¼ç¤¾ãã³ã½ã¼

Driving support device

GB2551516B

( en )

2016-06-20

2019-03-20

Jaguar Land Rover Ltd

Activity monitor

DE112016007448T5

( en )

2016-11-17

2019-08-14

Mitsubishi Electric Corporation

Vehicle interior device, mobile terminal device, recognition support system, recognition support method, and recognition support program

EP3548840B1

( en )

2016-11-29

2023-10-11

Blackmore Sensors & Analytics, LLC

Method and system for classification of an object in a point cloud data set

KR102254466B1

( en )

2016-11-30

2021-05-20

ë¸ëëª¨ì´ ì¼ìì¤ ì¤ë ì ëë¦¬í±ì¤, ììì¨

Automatic real-time adaptive scanning method and system using optical distance measurement system

US11624828B2

( en )

2016-11-30

2023-04-11

Blackmore Sensors & Analytics, Llc

Method and system for adaptive scanning with optical ranging systems

US10422880B2

( en )

2017-02-03

2019-09-24

Blackmore Sensors and Analytics Inc.

Method and system for doppler detection and doppler correction of optical phase-encoded range detection

DE102017204691B3

( en )

2017-03-21

2018-06-28

Audi Ag

Control device for redundantly performing an operating function and motor vehicle

JP7290571B2

( en )

2017-03-31

2023-06-13

ãããã¤ã³ ã©ã¤ãã¼ ã¦ã¼ã¨ã¹ã¨ã¼ï¼ã¤ã³ã³ã¼ãã¬ã¤ããã

Integrated LIDAR lighting output control

WO2018195869A1

( en )

2017-04-27

2018-11-01

SZ DJI Technology Co., Ltd.

Systems and methods for generating real-time map using movable object

US10401495B2

( en )

2017-07-10

2019-09-03

Blackmore Sensors and Analytics Inc.

Method and system for time separated quadrature detection of doppler effects in optical range measurements

KR20200016386A

( en )

2017-07-20

2020-02-14

ëì° ì§ëì°ì¤ ê°ë¶ìí¤ê°ì´ì¤

Vehicle driving control method and vehicle driving control device

US20190033875A1

( en )

2017-07-31

2019-01-31

Ford Global Technologies, Llc

Occupancy-based vehicle collision management

CN110892281B

( en )

2017-08-28

2023-08-18

é»ææéè´£ä»»ä¸¤åå¬å¸

Method for operation of radar system

DE102017215718B4

( en )

2017-09-07

2019-06-13

Audi Ag

Method for evaluating an optical appearance in a vehicle environment and vehicle

US20190079526A1

( en )

2017-09-08

2019-03-14

Uber Technologies, Inc.

Orientation Determination in Object Detection and Tracking for Autonomous Vehicles

US11599795B2

( en )

2017-11-08

2023-03-07

International Business Machines Corporation

Reducing the cost of n modular redundancy for neural networks

WO2019094863A1

( en )

2017-11-13

2019-05-16

Smart Ag, Inc.

Safety system for autonomous operation of off-road and agricultural vehicles using machine learning for detection and identification of obstacles

EP3688718A1

( en )

2017-11-15

2020-08-05

Google LLC

Unsupervised learning of image depth and ego-motion prediction neural networks

US20190155283A1

( en )

2017-11-17

2019-05-23

Waymo Llc

Determining pullover locations for autonomous vehicles

US11163309B2

( en )

2017-11-30

2021-11-02

Direct Current Capital LLC

Method for autonomous navigation

US11360475B2

( en )

2017-12-05

2022-06-14

Waymo Llc

Real-time lane change selection for autonomous vehicles

KR102030462B1

( en )

2017-12-08

2019-10-10

íëì¤í¸ë¡ ì£¼ìíì¬

An Apparatus and a Method for Detecting Errors On A Plurality of Multi-core Processors for Vehicles

US11130497B2

( en )

2017-12-18

2021-09-28

Plusai Limited

Method and system for ensemble vehicle control prediction in autonomous driving vehicles

US11273836B2

( en )

2017-12-18

2022-03-15

Plusai, Inc.

Method and system for human-like driving lane planning in autonomous driving vehicles

US20190185012A1

( en )

2017-12-18

2019-06-20

PlusAI Corp

Method and system for personalized motion planning in autonomous driving vehicles

US10852731B1

( en )

2017-12-28

2020-12-01

Waymo Llc

Method and system for calibrating a plurality of detection systems in a vehicle

US11328210B2

( en )

2017-12-29

2022-05-10

Micron Technology, Inc.

Self-learning in distributed architecture for enhancing artificial neural network

CN109993300B

( en )

2017-12-29

2021-01-29

åä¸ºææ¯æéå¬å¸

Training method and device of neural network model

US10824162B2

( en )

2018-01-03

2020-11-03

Uatc, Llc

Low quality pose lane associator

CN110248861B

( en )

2018-01-07

2023-05-30

è¾è¾¾å¬å¸

Guiding a vehicle using a machine learning model during vehicle maneuvers

JP7113337B2

( en )

2018-01-12

2022-08-05

ããã½ããã¯ï¼©ï½ããã¸ã¡ã³ãæ ªå¼ä¼ç¤¾

Server device, vehicle device, vehicle system, and information processing method

US20190220016A1

( en )

2018-01-15

2019-07-18

Uber Technologies, Inc.

Discrete Decision Architecture for Motion Planning System of an Autonomous Vehicle

JP7204326B2

( en )

2018-01-15

2023-01-16

ãã¤ãã³æ ªå¼ä¼ç¤¾

Information processing device, its control method and program, and vehicle driving support system

DE102018200982A1

( en )

2018-01-23

2019-08-08

Volkswagen Aktiengesellschaft

Method for processing sensor data in a number of control units, appropriately designed preprocessing unit and vehicle

FR3077382B1

( en )

2018-01-30

2020-02-21

Transdev Group

ELECTRONIC METHOD AND DEVICE FOR CONTROLLING THE SPEED OF AN AUTONOMOUS VEHICLE, COMPUTER PROGRAM, AUTONOMOUS VEHICLE AND ASSOCIATED MONITORING PLATFORM

US11091162B2

( en )

2018-01-30

2021-08-17

Toyota Motor Engineering & Manufacturing North America, Inc.

Fusion of front vehicle sensor data for detection and ranging of preceding objects

DE112019000065B4

( en )

2018-02-02

2025-01-09

Nvidia Corporation

SAFETY PROCEDURE ANALYSIS FOR OBSTACLE AVOIDANCE IN AN AUTONOMOUS VEHICLE

KR102066219B1

( en )

2018-02-05

2020-01-14

ì£¼ìíì¬ ë§ë

Apparatus and method for controlling vehicle based on redundant architecture

KR102541561B1

( en )

2018-02-12

2023-06-08

ì¼ì±ì ìì£¼ìíì¬

Method of providing information for driving vehicle and apparatus thereof

US11995551B2

( en )

2018-02-14

2024-05-28

Nvidia Corporation

Pruning convolutional neural networks

DE102018202296A1

( en )

2018-02-15

2019-08-22

Robert Bosch Gmbh

Radar sensor system and method for operating a radar sensor system

US10769840B2

( en )

2018-02-27

2020-09-08

Nvidia Corporation

Analysis of point cloud data using polar depth maps and planarization techniques

JP6607272B2

( en )

2018-03-02

2019-11-20

æ ªå¼ä¼ç¤¾ï¼ªï½ï½ã±ã³ã¦ãã

VEHICLE RECORDING DEVICE, VEHICLE RECORDING METHOD, AND PROGRAM

US10605897B2

( en )

2018-03-06

2020-03-31

Veoneer Us, Inc.

Vehicle lane alignment correction improvements

US20220035371A1

( en )

2018-03-09

2022-02-03

State Farm Mutual Automobile Insurance Company

Backup control systems and methods for autonomous vehicles

US10884115B2

( en )

2018-03-09

2021-01-05

Waymo Llc

Tailoring sensor emission power to map, vehicle state, and environment

EP3540710A1

( en )

2018-03-14

2019-09-18

Honda Research Institute Europe GmbH

Method for assisting operation of an ego-vehicle, method for assisting other traffic participants and corresponding assistance systems and vehicles

WO2019178548A1

( en )

2018-03-15

2019-09-19

Nvidia Corporation

Determining drivable free-space for autonomous vehicles

WO2019182974A2

( en )

2018-03-21

2019-09-26

Nvidia Corporation

Stereo depth estimation using deep neural networks

RU2716322C2

( en )

2018-03-23

2020-03-11

Reproducing augmentation of image data

WO2019191306A1

( en )

2018-03-27

2019-10-03

Nvidia Corporation

Training, testing, and verifying autonomous machines using simulated environments

US10946868B2

( en )

2018-03-28

2021-03-16

Nio Usa, Inc.

Methods and devices for autonomous vehicle operation

US10676085B2

( en )

2018-04-11

2020-06-09

Aurora Innovation, Inc.

Training machine learning model based on training instances with: training instance input based on autonomous vehicle sensor data, and training instance output based on additional vehicle sensor data

US10522038B2

( en )

2018-04-19

2019-12-31

Micron Technology, Inc.

Systems and methods for automatically warning nearby vehicles of potential hazards

CN111971533B

( en )

2018-04-24

2022-05-31

ä¸è±çµæºæ ªå¼ä¼ç¤¾

Attack detection device, computer-readable recording medium, and attack detection method

DE102018109851A1

( en )

2018-04-24

2019-10-24

Albert-Ludwigs-UniversitÃ¤t Freiburg

Method and device for determining a network configuration of a neural network

US11593119B2

( en )

2018-04-27

2023-02-28

Tesla, Inc.

Autonomous driving controller parallel processor boot order

EP3794421A4

( en )

2018-05-09

2022-12-21

Cavh Llc

INTELLIGENCE ALLOCATION TRAINING SYSTEMS AND METHODS BETWEEN VEHICLES AND HIGHWAYS

US11042156B2

( en )

2018-05-14

2021-06-22

Honda Motor Co., Ltd.

System and method for learning and executing naturalistic driving behavior

US11231715B2

( en )

2018-05-22

2022-01-25

King Fahd University Of Petroleum And Minerals

Method and system for controlling a vehicle

US11372115B2

( en )

2018-05-24

2022-06-28

Cyngn, Inc.

Vehicle localization

EP3572939A1

( en )

2018-05-25

2019-11-27

TTTech Auto AG

Method, device and real-time network for highly-integrated automotive systems

FR3082162B1

( en )

2018-06-11

2020-06-05

Renault S.A.S

METHOD AND DEVICE FOR DEVELOPING A CLOSED LOOP OF AN ADVANCED DRIVING AID DEVICE

DE102019113114A1

( en )

2018-06-19

2019-12-19

Nvidia Corporation

BEHAVIOR-CONTROLLED ROUTE PLANNING IN AUTONOMOUS MACHINE APPLICATIONS

US11966838B2

( en )

2018-06-19

2024-04-23

Nvidia Corporation

Behavior-guided path planning in autonomous machine applications

KR102563739B1

( en )

2018-06-20

2023-08-17

íì¨ìì¤í ì£¼ìíì¬

Vehicle electrical equipment cooling system

US11296999B2

( en )

2018-06-26

2022-04-05

Telefonaktiebolaget Lm Ericsson (Publ)

Sliding window based non-busy looping mode in cloud computing

US11354406B2

( en )

2018-06-28

2022-06-07

Intel Corporation

Physics-based approach for attack detection and localization in closed-loop controls for autonomous vehicles

US11119478B2

( en )

2018-07-13

2021-09-14

Waymo Llc

Vehicle sensor verification and calibration

WO2020014683A1

( en )

2018-07-13

2020-01-16

Kache.AI

Systems and methods for autonomous object detection and vehicle following

US11216007B2

( en )

2018-07-16

2022-01-04

Phantom Auto Inc.

Normalization of intelligent transport system handling characteristics

KR102077201B1

( en )

2018-07-20

2020-02-13

íëëª¨ë¹ì¤ ì£¼ìíì¬

Integrated control apparatus of vehicle method thereof

JP7199545B2

( en )

2018-07-20

2023-01-05

ã¡ã¤ ã¢ããªãã£ã¼ï¼ã¤ã³ã³ã¼ãã¬ã¤ããã

A Multi-view System and Method for Action Policy Selection by Autonomous Agents

US10614709B2

( en )

2018-07-24

2020-04-07

May Mobility, Inc.

Systems and methods for implementing multimodal safety operations with an autonomous agent

FR3084631B1

( en )

2018-07-31

2021-01-08

Valeo Schalter & Sensoren Gmbh

DRIVING ASSISTANCE FOR THE LONGITUDINAL AND / OR SIDE CHECKS OF A MOTOR VEHICLE

US11204605B1

( en )

2018-08-03

2021-12-21

GM Global Technology Operations LLC

Autonomous vehicle controlled based upon a LIDAR data segmentation system

US10479356B1

( en )

2018-08-17

2019-11-19

Lyft, Inc.

Road segment similarity determination

US10942030B2

( en )

2018-08-17

2021-03-09

Lyft, Inc.

Road segment similarity determination

US11129024B2

( en )

2018-08-21

2021-09-21

Continental Teves Ag & Co. Ohg

Vehicle-to-X communication device and method for realizing a safety integrity level in vehicle-to-X communication

US10816979B2

( en )

2018-08-24

2020-10-27

Baidu Usa Llc

Image data acquisition logic of an autonomous driving vehicle for capturing image data using cameras

WO2020045978A1

( en )

2018-08-29

2020-03-05

íêµê³¼íê¸°ì ì

Method and apparatus for estimating road surface type by using ultrasonic signal

US11800827B2

( en )

2018-09-14

2023-10-31

Agjunction Llc

Using non-real-time computers for agricultural guidance systems

US10712434B2

( en )

2018-09-18

2020-07-14

Velodyne Lidar, Inc.

Multi-channel LIDAR illumination driver

DE102018216082A1

( en )

2018-09-20

2018-12-13

Robert Bosch Gmbh

Method for cooperative maneuvering

US11199847B2

( en )

2018-09-26

2021-12-14

Baidu Usa Llc

Curvature corrected path sampling system for autonomous driving vehicles

US11403517B2

( en )

2018-09-27

2022-08-02

Intel Corporation

Proximity-based distributed sensor processing

KR102233260B1

( en )

2018-10-02

2021-03-29

ìì¤ì¼ì´íë ì½¤ ì£¼ìíì¬

Apparatus and method for updating high definition map

US11138348B2

( en )

2018-10-09

2021-10-05

Intel Corporation

Heterogeneous compute architecture hardware/software co-design for autonomous driving

US10627819B1

( en )

2018-10-11

2020-04-21

Pony Ai Inc.

On-site notification from autonomous vehicle for traffic safety

JP7014129B2

( en )

2018-10-29

2022-02-01

ãªã ãã³æ ªå¼ä¼ç¤¾

Estimator generator, monitoring device, estimator generator method and estimator generator

CN109543245B

( en )

2018-10-31

2021-08-10

ç¾åº¦å¨çº¿ç½ç»ææ¯ï¼åäº¬ï¼æéå¬å¸

Unmanned vehicle response capability boundary information determining method and device and electronic equipment

US11256263B2

( en )

2018-11-02

2022-02-22

Aurora Operations, Inc.

Generating targeted training instances for autonomous vehicles

US11403492B2

( en )

2018-11-02

2022-08-02

Aurora Operations, Inc.

Generating labeled training instances for autonomous vehicles

US11163312B2

( en )

2018-11-02

2021-11-02

Aurora Operations, Inc.

Removable automotive LIDAR data collection POD

US20200153926A1

( en )

2018-11-09

2020-05-14

Toyota Motor North America, Inc.

Scalable vehicle data compression systems and methods

US11032370B2

( en )

2018-11-14

2021-06-08

Toyota Jidosha Kabushiki Kaisha

Wireless communications in a vehicular macro cloud

US10789728B2

( en )

2018-11-15

2020-09-29

<td

Systems and methods for safe and reliable autonomous vehicles — Nvidia Corporation (US20240045426A1)

Related documents