ABSTRACT
Abstract
Systems and methods are disclosed for virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine and an operational flight program (OFP) running on a virtual flight management computer (FMC) by using a communication channels that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment core components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high-quality, realistic testing and training capability.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of and priority to and is a Continuation of U.S. patent application Ser. No. 16/195,117 filed Nov. 19, 2018, the contents of which are incorporated by reference in its entirety.
BACKGROUND
As computer processing capabilities have increased over the years, it has become possible to create compelling augmented reality (AR), mixed reality (MR) and virtual reality (VR) platforms using low cost components. For example, AR, MR, and VR platforms (collectively VR) may often use game engines that provide an application programming interface (API) for graphics and user controls. However, these game engines have limited fidelity with respect to interactions with objects in the environment.
An example of the limited fidelity of a VR game engine is that a control panel in a virtual environment is often displayed as a polygon having a texture bitmapped onto the polygon so that it resembles a physical control panel, but yet does not provide a realistic interactive experience across a full range of potential inputs. In some situations, the texture bitmaps are static, or the simulated control panel has only limited functionality. Thus, a need exists for more realistic virtual reality with virtualization in trainers and test environments.
SUMMARY
The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below. The following summary is provided to illustrate some examples disclosed herein. It is not meant, however, to limit all examples to any particular configuration or sequence of operations.
Some aspects and examples disclosed herein are directed to virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine (possibly a VR game engine) and an operational flight program (OFP) running on a virtual flight management computer (FMC) implemented on a virtual machine by using a communication channel that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high quality, realistic testing and training capability.
Some aspects and examples disclosed herein are directed to a VR environment platform with an immersive VR environment engine having aircraft control and display units and providing an immersive VR environment of an aircraft; a virtual FMC implemented on a virtual machine running an OFP; and a communication channel coupling the immersive VR environment engine with the virtual FMC, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.
BRIEF DESCRIPTION OF THE DRAWINGS
The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below:
FIG. 1 is a block diagram illustrating a virtual reality (VR) environment platform for providing a test environment in accordance with aspects of the present disclosure.
FIG. 2 is a block diagram illustrating a VR environment platform for providing a training environment in accordance with aspects of the present disclosure.
FIG. 3 is an illustration of a virtual flight management computer (FMC) implemented on a virtual machine in accordance with aspects of the present disclosure.
FIG. 4 is a block diagram illustrating a virtualization system in accordance with aspects of the present disclosure.
FIG. 5 is a flow diagram illustrating a process of providing virtual reality with virtualization in trainers and test environments in accordance with aspects of the present disclosure.
FIG. 6 A is an illustration of server consolidation in accordance with aspects of the present disclosure.
FIG. 6 B is an illustration of virtualization in accordance with aspects of the present disclosure.
FIG. 7 is an illustration of an aircraft flight deck that can be virtualized in accordance with aspects of the present disclosure.
Corresponding reference characters indicate corresponding parts throughout the drawings.
DETAILED DESCRIPTION
The various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made throughout this disclosure relating to specific examples and implementations are provided solely for illustrative purposes but, unless indicated to the contrary, are not meant to limit all examples.
In contrast with game engines, full-featured flight simulators have high fidelity models of hardware such as flight management computers (FMC) that communicate with a multipurpose computer display unit (MCDU), also known as a multipurpose control and display unit). In order to achieve the desired level of realism, the MCDU and FMCs are often simulated with intricate models or achieved through the use of actual flight hardware or proxy flight hardware. However, these options are costly to produce and are reliant on the skills of the developer. Meanwhile, existing low level training devices are model-based and sometime supplemented with physical structures that mock up the flight deck, flight deck, ground station, etc. When present, these physical mock ups are close to the layout of the actual device being trained for, but may not match the actual layout in most other cases. Additionally, the subsystem models do not have as much fidelity as virtual machines (VMs). A model is typically created by looking at the input/output for a subsystem and creating software to mimic the behavior of this subsystem from an input/output perspective.
VMs also generate input/output behavior, thus may appear to be similar, although for VMs, the behavior is driven by actual unmodified or minimally modified flight software binary executables, such as an operational flight program (OFP). For example, the FMC OFP can be run on the VM. VMs, that run the FMC's OFP, also referred to herein as a âvirtual FMC,â can provide considerably higher fidelity in a training environment, compared with simpler models and can also provide confidence that the behavior observed in the trainer will be the same as the physical device that the trainer is representing. It may then be possible to reduce the need for expensive full motion trainers (often costing in excess of $10M), by instead using a lower-cost compelling, immersive, high fidelity training platform.
Accordingly, examples of the present disclosure provide for virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine (possibly a VR game engine) and an OFP running on a virtual FMC by using a communication channel that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high quality, realistic testing and training capability.
Recent improvements in computer processing capabilities and advancements in VR and virtualization technologies are leveraged to create a new kind of training environment. A front end VR world provides an immersive and realistic visual experience for users, while the combinations of VR and real-time computer graphics allows users to perceive and interact with a virtual world in the same way they would with real devices. The integration of the VR world with a high fidelity back end consisting of models and VMs provides dynamic behavior that is nearly equivalent to physical hardware. The VMs allow for the actual unmodified or minimally modified flight software binaries and databases to be used in the training device, providing confidence that what is observed in the trainer will be the same as the actual aircraft. Additionally, testing environments are feasible with early creation of human-in-the-loop testing. Since the flight deck is created in the VR world, the flight deck can be prototyped and modified relatively inexpensively, and ultimately used as a design input for a physical flight deck. In general, when requirements are developed for the physical systems, models and virtual machines can be created and tested relatively quickly.
Various examples solve one or more problems related to the use of different protocols by different FMCs/MCDU's to communicate (e.g., ARINC 429, MIL-STD-1553, Ethernet). And, the messages can be packaged differently depending on the remote terminal (RT) and aircraft type. One or more examples disclosed herein use communication channels that allow for a uniform way to pack/unpack the messages. As a result, the communication channels can be incorporated in different processing engines, for example, a game engine, such as Unreal Engine 4. As such, with the one or more examples of the present disclosure, coding is performed once, and can be used on different FMCs with fewer changes, and more consistency. This reduces errors/debugging time and expense. Examples of various embodiments will now be described.
It should be appreciated that while the various examples described herein relate to an aircraft environment, the present disclosure can be implemented in other environments. For example, the herein described examples can be implemented in connection with any complex system that requires training, such as spacecraft, cars, boats, medical devices, etc. Thus, the various components described herein are operable in many other systems.
FIG. 1 is a block diagram illustrating an exemplary VR environment platform 100 for providing a test environment in accordance with aspects of the present disclosure. VR environment platform 100 includes an immersive VR environment engine 102 comprising aircraft control and display units and providing an immersive VR environment of an aircraft flight deck. For example, immersive VR environment engine 102 includes an MCDU 104 , a mode control panel (MCP) 106 , a display system (DS) 108 , such as a MAX® display system (MDS) available from Rockwell Collins of Cedar Rapids, Iowa, a radio tuning panel (RTP) 110 , an overhead panel 112 , and an electronic flight instrument system (EFIS) 114 . The MCDU 104 provides for navigation planning, and can display an electronic compass and flight information. The MCP 106 displays information such as airspeed and warnings for the flight crew. The DS 108 displays information such as the Primary Flight Display (PFD), Navigation Display (ND), and systems status. The overhead panel 112 provides interaction points, such as switches. In the VR environment, the overhead panel 112 can receive aircraft flight deck control inputs through virtual interaction points. The EFIS 114 is a (virtual version of a) flight deck instrument display system that displays flight data electronically rather than electromechanically. In some examples, the EFIS 114 is incorporated into the DS 108 .
A virtualization system 170 comprises one or more computer storage devices having computer-executable instructions stored thereon for operating the VR environment platform 100 , which, on execution by a computer, cause the computer to perform operations including operating the immersive VR environment engine 102 . A physical interaction point 172 , coupled to the virtualization system 170 and illustrated as a control stick, simulates an aircraft control and provides physical feedback to a user 180 of the VR environment platform 100 . It should be understood that other physical interaction points can be provided, for example, rudder pedals that also simulate an aircraft control and provide physical feedback to the user 180 . Also shown as coupled to the virtualization system 170 is a head mounted display (HMD) 182 , worn by the user 180 , for displaying a view of an immersive VR environment (as generated by the VR environment platform 100 ) to the user 180 .
In the VR environment platform 100 , as shown in FIG. 1 ,
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of and priority to and is a Continuation of U.S. patent application Ser. No. 16/195,117 filed Nov. 19, 2018, the contents of which are incorporated by reference in its entirety.
BACKGROUND
As computer processing capabilities have increased over the years, it has become possible to create compelling augmented reality (AR), mixed reality (MR) and virtual reality (VR) platforms using low cost components. For example, AR, MR, and VR platforms (collectively VR) may often use game engines that provide an application programming interface (API) for graphics and user controls. However, these game engines have limited fidelity with respect to interactions with objects in the environment.
An example of the limited fidelity of a VR game engine is that a control panel in a virtual environment is often displayed as a polygon having a texture bitmapped onto the polygon so that it resembles a physical control panel, but yet does not provide a realistic interactive experience across a full range of potential inputs. In some situations, the texture bitmaps are static, or the simulated control panel has only limited functionality. Thus, a need exists for more realistic virtual reality with virtualization in trainers and test environments.
SUMMARY
The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below. The following summary is provided to illustrate some examples disclosed herein. It is not meant, however, to limit all examples to any particular configuration or sequence of operations.
Some aspects and examples disclosed herein are directed to virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine (possibly a VR game engine) and an operational flight program (OFP) running on a virtual flight management computer (FMC) implemented on a virtual machine by using a communication channel that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high quality, realistic testing and training capability.
Some aspects and examples disclosed herein are directed to a VR environment platform with an immersive VR environment engine having aircraft control and display units and providing an immersive VR environment of an aircraft; a virtual FMC implemented on a virtual machine running an OFP; and a communication channel coupling the immersive VR environment engine with the virtual FMC, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.
BRIEF DESCRIPTION OF THE DRAWINGS
The disclosed examples are described in detail below with reference to the accompanying drawing figures listed below:
FIG. 1 is a block diagram illustrating a virtual reality (VR) environment platform for providing a test environment in accordance with aspects of the present disclosure.
FIG. 2 is a block diagram illustrating a VR environment platform for providing a training environment in accordance with aspects of the present disclosure.
FIG. 3 is an illustration of a virtual flight management computer (FMC) implemented on a virtual machine in accordance with aspects of the present disclosure.
FIG. 4 is a block diagram illustrating a virtualization system in accordance with aspects of the present disclosure.
FIG. 5 is a flow diagram illustrating a process of providing virtual reality with virtualization in trainers and test environments in accordance with aspects of the present disclosure.
FIG. 6 A is an illustration of server consolidation in accordance with aspects of the present disclosure.
FIG. 6 B is an illustration of virtualization in accordance with aspects of the present disclosure.
FIG. 7 is an illustration of an aircraft flight deck that can be virtualized in accordance with aspects of the present disclosure.
Corresponding reference characters indicate corresponding parts throughout the drawings.
DETAILED DESCRIPTION
The various embodiments will be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made throughout this disclosure relating to specific examples and implementations are provided solely for illustrative purposes but, unless indicated to the contrary, are not meant to limit all examples.
In contrast with game engines, full-featured flight simulators have high fidelity models of hardware such as flight management computers (FMC) that communicate with a multipurpose computer display unit (MCDU), also known as a multipurpose control and display unit). In order to achieve the desired level of realism, the MCDU and FMCs are often simulated with intricate models or achieved through the use of actual flight hardware or proxy flight hardware. However, these options are costly to produce and are reliant on the skills of the developer. Meanwhile, existing low level training devices are model-based and sometime supplemented with physical structures that mock up the flight deck, flight deck, ground station, etc. When present, these physical mock ups are close to the layout of the actual device being trained for, but may not match the actual layout in most other cases. Additionally, the subsystem models do not have as much fidelity as virtual machines (VMs). A model is typically created by looking at the input/output for a subsystem and creating software to mimic the behavior of this subsystem from an input/output perspective.
VMs also generate input/output behavior, thus may appear to be similar, although for VMs, the behavior is driven by actual unmodified or minimally modified flight software binary executables, such as an operational flight program (OFP). For example, the FMC OFP can be run on the VM. VMs, that run the FMC's OFP, also referred to herein as a âvirtual FMC,â can provide considerably higher fidelity in a training environment, compared with simpler models and can also provide confidence that the behavior observed in the trainer will be the same as the physical device that the trainer is representing. It may then be possible to reduce the need for expensive full motion trainers (often costing in excess of $10M), by instead using a lower-cost compelling, immersive, high fidelity training platform.
Accordingly, examples of the present disclosure provide for virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine (possibly a VR game engine) and an OFP running on a virtual FMC by using a communication channel that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high quality, realistic testing and training capability.
Recent improvements in computer processing capabilities and advancements in VR and virtualization technologies are leveraged to create a new kind of training environment. A front end VR world provides an immersive and realistic visual experience for users, while the combinations of VR and real-time computer graphics allows users to perceive and interact with a virtual world in the same way they would with real devices. The integration of the VR world with a high fidelity back end consisting of models and VMs provides dynamic behavior that is nearly equivalent to physical hardware. The VMs allow for the actual unmodified or minimally modified flight software binaries and databases to be used in the training device, providing confidence that what is observed in the trainer will be the same as the actual aircraft. Additionally, testing environments are feasible with early creation of human-in-the-loop testing. Since the flight deck is created in the VR world, the flight deck can be prototyped and modified relatively inexpensively, and ultimately used as a design input for a physical flight deck. In general, when requirements are developed for the physical systems, models and virtual machines can be created and tested relatively quickly.
Various examples solve one or more problems related to the use of different protocols by different FMCs/MCDU's to communicate (e.g., ARINC 429, MIL-STD-1553, Ethernet). And, the messages can be packaged differently depending on the remote terminal (RT) and aircraft type. One or more examples disclosed herein use communication channels that allow for a uniform way to pack/unpack the messages. As a result, the communication channels can be incorporated in different processing engines, for example, a game engine, such as Unreal Engine 4. As such, with the one or more examples of the present disclosure, coding is performed once, and can be used on different FMCs with fewer changes, and more consistency. This reduces errors/debugging time and expense. Examples of various embodiments will now be described.
It should be appreciated that while the various examples described herein relate to an aircraft environment, the present disclosure can be implemented in other environments. For example, the herein described examples can be implemented in connection with any complex system that requires training, such as spacecraft, cars, boats, medical devices, etc. Thus, the various components described herein are operable in many other systems.
FIG. 1 is a block diagram illustrating an exemplary VR environment platform 100 for providing a test environment in accordance with aspects of the present disclosure. VR environment platform 100 includes an immersive VR environment engine 102 comprising aircraft control and display units and providing an immersive VR environment of an aircraft flight deck. For example, immersive VR environment engine 102 includes an MCDU 104 , a mode control panel (MCP) 106 , a display system (DS) 108 , such as a MAX® display system (MDS) available from Rockwell Collins of Cedar Rapids, Iowa, a radio tuning panel (RTP) 110 , an overhead panel 112 , and an electronic flight instrument system (EFIS) 114 . The MCDU 104 provides for navigation planning, and can display an electronic compass and flight information. The MCP 106 displays information such as airspeed and warnings for the flight crew. The DS 108 displays information such as the Primary Flight Display (PFD), Navigation Display (ND), and systems status. The overhead panel 112 provides interaction points, such as switches. In the VR environment, the overhead panel 112 can receive aircraft flight deck control inputs through virtual interaction points. The EFIS 114 is a (virtual version of a) flight deck instrument display system that displays flight data electronically rather than electromechanically. In some examples, the EFIS 114 is incorporated into the DS 108 .
A virtualization system 170 comprises one or more computer storage devices having computer-executable instructions stored thereon for operating the VR environment platform 100 , which, on execution by a computer, cause the computer to perform operations including operating the immersive VR environment engine 102 . A physical interaction point 172 , coupled to the virtualization system 170 and illustrated as a control stick, simulates an aircraft control and provides physical feedback to a user 180 of the VR environment platform 100 . It should be understood that other physical interaction points can be provided, for example, rudder pedals that also simulate an aircraft control and provide physical feedback to the user 180 . Also shown as coupled to the virtualization system 170 is a head mounted display (HMD) 182 , worn by the user 180 , for displaying a view of an immersive VR environment (as generated by the VR environment platform 100 ) to the user 180 .
In the VR environment platform 100 , as shown in FIG. 1 , a test environment core component 130 interfaces with a flight simulator component 150 , the immersive VR environment engine 102 , and a virtual FMC 160 . The test environment core component 130 further interfaces with a navigation simulation 132 , a data link simulation 134 , an air traffic control simulation 136 , and a flight visualization 138 , in order to provide additional realism for the operation of the aircraft flight simulator component 150 . In other examples, other arrangements are possible with more or fewer components as necessary for the desired level of fidelity. In operation, the test environment core component 130 can be used, for example, to interject simulated failures and monitor subsystem operations. The test environment core component 130 is able to accomplish these tasks by the signaling among each of the flight simulator component 150 , the virtual FMC 160 , the navigation simulation 132 , the data link simulation 134 , the air traffic control simulation 136 , and the flight visualization 138 passing through the test environment core component 130 . Also shown as interfacing with the test environment core component 130 is a file server 140 , for storing data used by and generated by operations of the VR environment platform 100 .
In some examples, the flight simulator component 150 is the same binary executable that is used in high-end flight simulators. As such, the flight simulator component 150 can be interchangeable based on the configuration of the system. For example, the flight simulator component 150 can be configured to operate with or without a motion base (not shown), as needed for the desired level of fidelity. In the VR environment platform 100 , the flight simulator component 150 interfaces with the virtual FMC 160 through the test environment core component 130 , so that the test environment core component 130 can monitor the data traffic and inject simulated failures. The virtual FMC 160 runs an OFP, which, in some examples, is a copy of the same binary executable that is used in hardware FMCs on operational aircraft. The above-described arrangement provides a level of realism otherwise unattainable by typical game engines. The use of a communication channel, as described herein, also allows a game engine, in various examples, to interact with an OFP more easily than traditionally achievable by a game engine alone. The communication channel 122 couples the immersive VR environment engine 102 with the virtual FMC 160 through the test environment core component 130 . That is, data traffic is routed through the test environment core component 130 , which allows for example, test scripts and manual interaction to override nominal system behavior for testing purposes. The communication channel 122 emulates FMC communication protocols to permit the immersive VR environment engine 102 to interact with the virtual FMC 160 using the actual packed input/output (I/O) of the aircraft. In various examples, the communication channel 122 (with emulated FMC protocols) is also between the data link simulation 134 and the test environment core component 130 , and the air traffic control simulation 136 and the test environment core component 130 . The remaining connections illustrated in FIG. 1 do not use FMC communication protocols, and interconnect components that support the primary aircraft simulations and VMs. In some examples, the communication channel 122 is implemented using protocol buffers as known in the data communication technology. However, any suitable type of communication channel can be used.
The communication channel 122 supports any kind of packed input/output (I/O) to an aircraft avionics box, such as ARINC 429, ARINC 664, CAN bus, RS-485, MIL-STD-1553, MIL-STD-6016, ACARS, ATN, Ethernet, and so on. The I/O is transferred over the communication channel 122 using a form of interprocess communications if the link is virtual, e.g. sockets and UDP packets with aircraft I/O encapsulated in the packets. As another example, the packed I/O is transferred over physical I/O links by installing an ARINC 429 I/O card in a PC. The I/O is not stored in a file, but is dynamic information being shared between systems, just like in a real aircraft.
FIG. 2 is a block diagram illustrating an exemplary VR environment platform 200 for providing a training environment in accordance with aspects of the present disclosure. The VR environment platform 200 shares some components with the VR environment platform 100 of FIG. 1 , although whereas the VR environment platform 100 is used for testing, the VR environment platform 200 is used for training, such as training flight crews. In the VR environment platform 200 , the immersive VR environment engine 102 interfaces with the flight simulator component 150 and the virtual FMC 160 , directly. Thus, a communication channel 220 couples the immersive VR environment engine 102 with the virtual FMC 160 . The communication channel 220 emulates FMC communication protocols to permit the immersive VR environment engine 102 to interact with the virtual FMC 160 . In some configuration of FIG. 2 , the aircraft flight simulator 150 is located and connected to the other components similar to FIG. 1 .
Additionally, the flight simulator component 150 and the virtual FMC 160 are coupled without the test environment core component 130 as an intermediary, although in some examples, a component similar to the test environment core component 130 can be used. The flight simulator component 150 interfaces with both the immersive VR environment engine 102 and the virtual FMC 160 . The file server 140 is shown as coupled to the virtual FMC 160 , although in some examples, the file server 140 can be coupled to the flight simulator component 150 . Additionally, since the VR environment platform 200 is used for training, an instructor operator station component 230 interfaces with the flight simulator component 150 . The instructor operator station component 230 permits an instructor to monitor and possibly participate in training activities conducted with the VR environment platform 200 .
The VR environment platform 200 can be used as a flight management system (FMS) trainer. In some examples, a two-dimensional (2D) tablet-based application facilitates learning and practicing the operation of the MCDU 104 , the MCP 106 , and other displays (e.g., the DS 108 ) prior to and during various phases of flight. Students can learn and to practice FMS data entry (flight plan, etc.), as well as autoflight and flight path management as part of an instructor-led curriculum or for standalone self-guided training.
The VR environment platform 200 can be used as an interactive, fully-simulated virtual aircraft flight deck. Students will be able to learn and practice procedures in a fully immersive realistic virtual environment as part of an instructor-led curriculum or for standalone self-guided training. 3D interactive VR applications with high fidelity components as described in the VR environment platform 200 have the potential to greatly reduce the cost of training, as well as improve the learning outcome through immersive experiences. It can also enable self and distance learning, reducing and potentially eliminating the need to travel to an onsite training location.
FIG. 3 is an illustration of the exemplary virtual FMC 160 , which can be executed as a virtual machine (VM) on the virtualization system 170 , shown in more detail in FIG. 4 . As illustrated, the virtual FMC 160 comprises a processor 302 , which can be a PowerPC processor, an ARINC bus 304 (e.g., ARINC 429 or ARINC 664), a universal asynchronous receiver-transmitter (UART) 306 , an Ethernet module 308 , and an OFP 310 . The DITS 304 is the predominant avionics data bus used on many higher-end commercial and transport aircraft. It defines the physical and electrical interfaces of a two-wire data bus and a data protocol to support an aircraft's avionics local area network The UART 306 is a device for asynchronous serial communication in which the data format and transmission speeds are configurable.
The OFP 310 is the embedded software that runs on an avionics device, namely, the hardware FMC. The physical platform that the test environment can be used to develop and that the trainer is used to train to operate, as described herein, emulates an aircraft flight deck 700 as shown in FIG. 7 . That is, various examples emulate a physical system corresponding to the aircraft flight deck 700 .
The OFP 310 contains software logic that perform functions necessary for the operation of the FMC. In the example of the hardware FMC, the OFP 310 performs the route planning, calculations, logic, input/out, and the like. The OFP 310 typically executes on top of an operating system software (e.g., an embedded real-time operating system) and functional components. The OFP 310 can be specifically tailored to the hardware of the avionics, the aircraft, and the aircraft type. The hardware FMC OFP communicates with other aircraft systems using Avionics buses, such as ARINC 429, examples of other aircraft systems include MCDU 104 , MCP 106 , DS 108 , Overhead Panel 112 , and EFIS 114 . It should be appreciated that protocol buffers are not typically used for communication on an actual aircraft. Standard Ethernet for which the protocol buffers are designed lack the guaranteed message delivery required for aircraft system operation.
FIG. 4 is a block diagram illustrating additional detail for the exemplary virtualization system 170 . The virtualization system 170 can be a computing system comprised of hardware and software that can emulate other devices. In implementations, the virtualization system 170 can be one or more general purpose computers, such as a server or a desktop computer that hosts one or more VMs that emulate physical hardware systems, such as a hardware FMC, as well as their hardware and software interfaces (e.g., peripherals, data links, interrupt behavior, and timing requirements). Additionally, a VM, such as the virtual FMC 160 execute an exact or substantially exact copy (e.g., an image) of the application software executed by the corresponding physical hardware system (for example the OFP 310 ). In implementations, the virtualization system 170 can include, e.g., a hypervisor or virtual machine monitor software. For example, the virtualization system 170 can use QUICK EMULATOR (âQEMUâ), which is an open source software application that performs hardware virtualization. Thus, the virtual FMC 160 can be an emulation of a physical hardware FMC within the virtualization system 170 .
The virtualization system 170 includes a computing device 403 , an input/output (I/O) device 413 , and a storage device 415 . The I/ O device 413 can include any device that enables an individual to interact with the computing device 403 (e.g., a user interface) and/or any device that enables the computing device 403 to communicate with one or more other computing devices using any type of communications link. The I/ O device 413 can be, for example, a touchscreen display, pointer device, keyboard, etc.
The storage device 415 can comprise a computer-readable, non-volatile hardware storage device that stores information and program instructions. For example, the storage device 415 can be one or more flash drives and/or hard disk drives. In accordance with aspects of the present disclosure, the storage device 415 can store hardware system program code 417 , one or more memory maps 419 , interrupt logic 421 , and a device library 423 . The program code 417 can be application software of a physical hardware system (e.g., a physical FMC). In implementations, the program code 417 substantially mirrors that of the physical hardware system. The memory map 419 describes connections between components of the physical hardware systems from a memory interface perspective. For example, the memory map 419 can represent locations of information for memory registers of hardware components of the physical hardware system as an offset from a starting memory address. The interrupt logic 421 can be information describing the interrupt functionality of the physical hardware system, as detailed below. The device library 423 can be a repository of computer-readable information and instructions describing emulations of one or more physical hardware systems, which can be previously created and stored for future use.
In some examples, the computing device 403 includes one or more processors 439 (e.g., microprocessor, microchip, or application-specific integrated circuit), one or more memory devices 441 (e.g., RAM, read-only memory (ROM)), one or more I/O interfaces 443 , and one or more network interface devices 445 . The memory devices 441 can include a local memory (e.g., a random access memory and a cache memory) employed during execution of program instructions. Additionally, the computing device 403 includes at least one communication channel 444 (e.g., a data bus) by which it communicates with the I/ O device 413 and the storage device 415 . The processor 439 executes computer program instructions (e.g., an operating system and/or application programs), which can be stored in the memory device 441 and/or storage device 415 .
The processor 439 can also execute computer program instructions of a virtualization application 451 (e.g., QEMU) and test and/or evaluation software 455 . The virtualization application 451 can be the same or similar to that previously described. For example, the virtualization application 451 can include a hypervisor or VM monitor software. In accordance with aspects of the present disclosure, the virtualization application 451 can provide a VM (e.g., the virtual FMC 160 ) using the program code 417 , the memory map 419 , the interrupt logic 421 , and/or the device library 423 .
The training, test and/or evaluation software 455 can be an application or program including computer-readable instructions and information configured to train, test, evaluate, and/or validate software. For example, training, test and/or evaluation software 455 can execute test routines that verify program code of a hardware device behaves as expected in response to a predetermined scenario. Additionally, the training, test and evaluation software 455 can execute cybersecurity routines that test attack vectors of malicious software. For example, the training, test and/or evaluation software 455 can record complete state (registers, memory, hardware states, etc.) of a virtual machine as instructions execute to allow observation and analysis of a compromised system. The training, test and/or evaluation software 455 can be used to provide a trainer for users of the VR environment platform 200 (e.g. flight crew or maintenance trainers).
It is noted that the computing device 403 is representative of various possible equivalent-computing devices that can perform the processes described herein. To this extent, in embodiments, the functionality provided by the computing device 403 can be any combination of general and/or specific purpose hardware and/or computer program instructions. In the disclosed embodiments, the program instructions and hardware can be created using standard programming and engineering techniques, respectively.
FIG. 5 is a flow diagram 500 illustrating an exemplary process of providing an immersive VR environment with virtualization in trainers and test environments. Operation 502 includes initiating the immersive VR environment engine 102 comprising aircraft control and display units (e.g., the MCDU 104 , the MCP 106 , the <figure-callout id="108" label="DS" filenames="US12027072-20240702
CLAIMS
Claims ( 20 )
What is claimed is:
1. A virtual reality (VR) environment platform comprising:
one or more virtual machines (VMs);
a host server, wherein each of the one or more VMs are operable on the host server to emulate other systems;
a virtualization system operable on one of the one or more VMs, the virtualization system having a hardware device including one or more computer storage devices including computer-executable instructions stored thereon for operating the VR environment platform, which, on execution by the hardware device, cause the device to:
provide an immersive VR environment of an aircraft;
run, via a virtual flight management computer (FMC), an operational flight program (OFP); and
couple the immersive VR environment with the virtual FMC to permit the immersive VR environment to interact with the virtual FMC using actual packed input/output (I/O) of the aircraft.
2. The VR environment platform of claim 1 , wherein the immersive VR environment and the virtual FMC are to be coupled via a protocol buffer.
3. The VR environment platform of claim 1 , further comprising:
simulating one or more line replaceable units (LRUs) that represent specialized hardware units used in an aircraft, wherein each of the LRUs is to be simulated via a different one of the one or more VMs.
4. The VR environment platform of claim 3 , wherein simulating the one or more LRUs is to permit other components of an aircraft to be virtualized to be used with the VR environment platform.
5. The VR environment platform of claim 1 , further comprising:
a head mounted display (HMD) for displaying a view of an immersive VR environment.
6. The VR environment platform of claim 1 , further comprising:
virtual interaction points providing aircraft flight deck control inputs for the immersive VR environment.
7. The VR environment platform of claim 1 , further comprising:
a physical interaction point simulating an aircraft control and providing physical feedback to a user of the VR environment platform.
8. The VR environment platform of claim 1 , further comprising:
a flight simulator component interfacing with the immersive VR environment engine and the virtual FMC.
9. A method of operating an immersive virtual reality (VR) environment platform, the method comprising:
provide an immersive VR environment of an aircraft;
run, via a virtual flight management computer (FMC), an operational flight program (OFP); and
couple the immersive VR environment with the virtual FMC to permit the immersive VR environment to interact with the virtual FMC using actual packed input/output (I/O) of the aircraft.
10. The method of claim 9 , wherein the immersive VR environment and the virtual FMC are to be coupled via a protocol buffer.
11. The method of claim 9 , further comprising:
simulating one or more line replaceable units (LRUs) that represent specialized hardware units used in an aircraft, wherein each of the LRUs is to be simulated via a different one of the one or more VMs.
12. The method of claim 11 , wherein simulating the one or more LRUs is to permit other components of an aircraft to be virtualized to be used with the VR environment platform.
13. The method of claim 9 , further comprising:
displaying a view of an immersive VR environment on a head mounted display (HMD).
14. The method of claim 9 , further comprising:
receiving aircraft flight deck control inputs for the immersive VR environment through virtual interaction points.
15. The method of claim 9 , further comprising:
providing physical feedback to a user of the VR environment platform through a physical interaction point simulating an aircraft control.
16. The method of claim 9 , further comprising:
operating a flight simulator component interfacing with the immersive VR environment and the virtual FMC.
17. One or more computer storage devices having computer- executable instructions stored thereon for operating an immersive virtual reality (VR) environment platform, which, on execution by a computer, cause the computer to perform operations comprising:
provide an immersive VR environment of an aircraft;
run, via a virtual flight management computer (FMC), an operational flight program (OFP); and
couple the immersive VR environment with the virtual FMC to permit the immersive VR environment to interact with the virtual FMC using actual packed input/output (I/O) of the aircraft.
18. The one or more computer storage devices of claim 17 , wherein the immersive VR environment and the virtual FMC are to be coupled via a protocol buffer.
19. The one or more computer storage devices of claim 17 , simulating one or more line replaceable units (LRUs) that represent specialized hardware units used in an aircraft, wherein each of the LRUs is to be simulated via a different one of the one or more VMs.
20. The one or more computer storage devices of claim 19 , wherein simulating the one or more LRUs is to permit other components of an aircraft to be virtualized to be used with the VR environment platform.
US18/299,389
2018-11-19
2023-04-12
Virtual reality with virtualization in trainers and test environments
Active
US12027072B2
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US18/299,389
US12027072B2
( en )
2018-11-19
2023-04-12
Virtual reality with virtualization in trainers and test environments
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US16/195,117
US11688297B2
( en )
2018-11-19
2018-11-19
Virtual reality with virtualization in trainers and test environments
US18/299,389
US12027072B2
( en )
2018-11-19
2023-04-12
Virtual reality with virtualization in trainers and test environments
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/195,117
Continuation
US11688297B2
( en )
2018-11-19
2018-11-19
Virtual reality with virtualization in trainers and test environments
Publications (2)
Publication Number
Publication Date
US20230245585A1
US20230245585A1 ( en )
2023-08-03
US12027072B2
true
US12027072B2 ( en )
2024-07-02
Family
ID=68066612
Family Applications (2)
Application Number
Title
Priority Date
Filing Date
US16/195,117
Active
2041-12-28
US11688297B2
( en )
2018-11-19
2018-11-19
Virtual reality with virtualization in trainers and test environments
US18/299,389
Active
US12027072B2
( en )
2018-11-19
2023-04-12
Virtual reality with virtualization in trainers and test environments
Family Applications Before (1)
Application Number
Title
Priority Date
Filing Date
US16/195,117
Active
2041-12-28
US11688297B2
( en )
2018-11-19
2018-11-19
Virtual reality with virtualization in trainers and test environments
Country Status (5)
Country
Link
US
( 2 )
US11688297B2
( en )
EP
( 1 )
EP3654315B1
( en )
JP
( 1 )
JP7393922B2
( en )
KR
( 1 )
KR102831890B1
( en )
CN
( 1 )
CN111195433B
( en )
Families Citing this family (16)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11442829B2
( en )
*
2020-03-16
2022-09-13
International Business Machines Corporation
Packeted protocol device test system
CN111653141A
( en )
*
2020-06-08
2020-09-11
æ¹åç¿°å¤å®ä¸æéå ¬å¸
VR-based children traffic safety education simulation system and method
US12579344B2
( en )
*
2020-09-15
2026-03-17
The Boeing Company
Hosting pre-certified systems, remote activation of customer options, and optimization of flight algorithms in an emulated environment with real world operational conditions and data
CN112133152A
( en )
*
2020-10-22
2020-12-25
æ±èæ å½çèªç©ºææ¯æéå ¬å¸
PFCU comprehensive practical training platform
CN112102682B
( en )
*
2020-11-09
2021-05-18
ä¸çµç§èæ¹éç¨èªç©ºäº§ä¸ææ¯ç 究é¢æéå ¬å¸
Aircraft driving training system and method based on 5G communication
US11939085B2
( en )
2021-06-16
2024-03-26
Beta Air, Llc
Methods and systems for wrapping simulated intra-aircraft communication to a physical controller area network
TWI832084B
( en )
*
2021-09-03
2024-02-11
åéç§æ大å¸
Game interactive system with story-like and extendable content and method thereof
WO2023084456A1
( en )
*
2021-11-12
2023-05-19
Ahmad Fareed Aldarwish
System for simulating the operation of a vehicle, associated components and methods
JP2024024304A
( en )
*
2022-08-09
2024-02-22
æ¥æ¬ç¡ç·æ ªå¼ä¼ç¤¾
Ship handling training system and ship handling training program
JP2024072656A
( en )
*
2022-11-16
2024-05-28
ä¸è±éå·¥æ¥æ ªå¼ä¼ç¤¾
Operation display system
EP4545409A1
( en )
*
2023-10-26
2025-04-30
Airbus Operations GmbH
Method and system for surveillance of a passenger cabin of an aircraft
US20250191491A1
( en )
*
2023-11-20
2025-06-12
Vrgineers, Inc.
Modular reconfigurable simulator
CN117576980B
( en )
*
2024-01-19
2024-03-22
ä¸å½æ°ç¨èªç©ºé£è¡å¦é¢
Flight simulation cabin data complement method and system
CN118982938B
( en )
*
2024-09-03
2025-09-26
å®å¾½åçä¸å¤©çµåçµåæéå ¬å¸
A multi-channel 485 area networking and fault simulation module and system
KR102872488B1
( en )
*
2024-11-04
2025-10-20
ííìì¤í 주ìíì¬
Apparatus and method for monitoring flight control computers
CN121683561A
( en )
*
2026-02-11
2026-03-17
æ±è¥¿æ±æ¯é«çç 究é¢
Model control protocol-based illusion engine interaction and control method and system
Citations (23)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US6478581B1
( en )
1999-06-07
2002-11-12
American Gnc Corporation
External method for employing CNDU in trainer system
US20150097861A1
( en )
2013-10-03
2015-04-09
Honda Motor Co., Ltd.
System and method for dynamic in-vehicle virtual reality
US20150260474A1
( en )
2014-03-14
2015-09-17
Lineweight Llc
Augmented Reality Simulator
US20160091894A1
( en )
2014-09-30
2016-03-31
SZ DJI Technology Co., Ltd
Systems and methods for flight simulation
US20160236088A1
( en )
2013-12-23
2016-08-18
Hong C. Li
Provision of a virtual environment based on real time data
US20170061813A1
( en )
2014-09-30
2017-03-02
SZ DJI Technology Co., Ltd.
System and method for supporting simulated movement
US20170148340A1
( en )
2015-11-22
2017-05-25
Victor Popa-Simil
Method and device to improve the flying abilities of the airborne devices operator
US20180082477A1
( en )
*
2016-09-22
2018-03-22
Navitaire Llc
Systems and Methods for Improved Data Integration in Virtual Reality Architectures
US20180373411A1
( en )
*
2017-06-21
2018-12-27
Navitaire Llc
Systems and methods for seat selection in virtual reality
US20190063881A1
( en )
2017-08-25
2019-02-28
Aurora Flight Sciences Corporation
Aerial Vehicle Interception System
US20190130783A1
( en )
2017-11-02
2019-05-02
Bell Helicopter Textron Inc.
Vr emulator using galvanic vestibular stimulation devices
US20190244537A1
( en )
2018-02-02
2019-08-08
Access Virtual, LLC
Virtual reality based pilot training system
US20190355272A1
( en )
*
2018-05-17
2019-11-21
Gulfstream Aerospace Corporation
Tactile responsive surface for virtual reality training device
US20200066177A1
( en )
2018-08-24
2020-02-27
Industrial Technology Research Institute
Multi-view display device and manipulation simulation device
US20200134705A1
( en )
2018-10-31
2020-04-30
The Boeing Company
Aircraft production marketplace
US10924679B2
( en )
*
2015-12-24
2021-02-16
Lg Electronics Inc.
Display device for vehicle and control method thereof
US20210049925A1
( en )
2018-04-27
2021-02-18
Red 6 Inc.
Augmented reality for vehicle operations
US20210104099A1
( en )
*
2019-10-08
2021-04-08
Panasonic Avionics Corporation
Utilizing virtual reality and hi-definition camera technology to allow passengers to experience flight path
US11049318B2
( en )
*
2018-11-29
2021-06-29
Honda Motor Co., Ltd.
Content providing device, content providing method, and storage medium
US20210381639A1
( en )
*
2020-06-04
2021-12-09
Facebook Technologies, Llc
Pass-through ratcheting mechanism
US20230267849A1
( en )
*
2022-02-23
2023-08-24
The Boeing Company
Recorded physical interaction playback
US20230333552A1
( en )
*
2022-04-13
2023-10-19
Rotor Technologies, Inc.
Methods and systems for human-in-the-loop vehicular command and control using immersive synthetic vision
US20230360539A1
( en )
*
2022-01-20
2023-11-09
U.S. Army DEVCOM, Army Research Laboratory
System and method for calculation and display of formation flight information on augmented reality display device
Family Cites Families (12)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
JP2001100628A
( en )
*
1999-09-28
2001-04-13
Toshiba Corp
Flight training equipment
JP2002229433A
( en )
*
2001-01-31
2002-08-14
Shimadzu Corp
Flight simulator
JP3665058B2
( en )
*
2003-02-26
2005-06-29
ä¸è±éå·¥æ¥æ ªå¼ä¼ç¤¾
Simulator scenario production support program and simulator device
CN100377756C
( en )
*
2004-04-19
2008-04-02
æ·±å³å强æºè½ææ¯æéå ¬å¸
Virtual flight system and method thereof
US8734250B2
( en )
*
2012-10-05
2014-05-27
Wargaming.Net Llp
Control of vehicles in three dimensional space
KR101392266B1
( en )
*
2013-01-18
2014-05-07
íêµíê³µì°ì£¼ì°ì 주ìíì¬
Flight simulator for kuh and controlling method
US10173129B2
( en )
*
2013-06-09
2019-01-08
Sony Interactive Entertainment Inc.
Methods for rendering interactive content to a head mounted display
JP2016184244A
( en )
*
2015-03-25
2016-10-20
ä¸è±éå·¥æ¥æ ªå¼ä¼ç¤¾
Simulator device and image formation method
JP6769697B2
( en )
*
2015-09-16
2020-10-14
æ ªå¼ä¼ç¤¾ãã³ãã¤ãã ã³ã¨ã³ã¿ã¼ãã¤ã³ã¡ã³ã
Program and game equipment
CN108295464A
( en )
*
2017-01-11
2018-07-20
ç¿åï¼ä¸æµ·ï¼çµåææ¯æéå ¬å¸
A kind of simulated training method and system
US10395551B2
( en )
*
2017-03-29
2019-08-27
The Boeing Company
Systems and methods for an immersive simulator
CN107694090A
( en )
*
2017-09-29
2018-02-16
广å·äºåç½ç»ç§ææéå ¬å¸
A Simulated Flight System Combining Virtual Scene with Manipulator
2018
2018-11-19
US
US16/195,117
patent/US11688297B2/en
active
Active
2019
2019-09-24
EP
EP19199207.2A
patent/EP3654315B1/en
active
Active
2019-09-30
KR
KR1020190120470A
patent/KR102831890B1/en
active
Active
2019-10-17
CN
CN201910987554.8A
patent/CN111195433B/en
active
Active
2019-11-14
JP
JP2019205990A
patent/JP7393922B2/en
active
Active
2023
2023-04-12
US
US18/299,389
patent/US12027072B2/en
active
Active
Patent Citations (24)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US6478581B1
( en )
1999-06-07
2002-11-12
American Gnc Corporation
External method for employing CNDU in trainer system
US20150097861A1
( en )
2013-10-03
2015-04-09
Honda Motor Co., Ltd.
System and method for dynamic in-vehicle virtual reality
US20150097860A1
( en )
2013-10-03
2015-04-09
Honda Motor Co., Ltd.
System and method for dynamic in-vehicle virtual reality
US20160236088A1
( en )
2013-12-23
2016-08-18
Hong C. Li
Provision of a virtual environment based on real time data
US20150260474A1
( en )
2014-03-14
2015-09-17
Lineweight Llc
Augmented Reality Simulator
US20170061813A1
( en )
2014-09-30
2017-03-02
SZ DJI Technology Co., Ltd.
System and method for supporting simulated movement
US20160091894A1
( en )
2014-09-30
2016-03-31
SZ DJI Technology Co., Ltd
Systems and methods for flight simulation
US20170148340A1
( en )
2015-11-22
2017-05-25
Victor Popa-Simil
Method and device to improve the flying abilities of the airborne devices operator
US10924679B2
( en )
*
2015-12-24
2021-02-16
Lg Electronics Inc.
Display device for vehicle and control method thereof
US20180082477A1
( en )
*
2016-09-22
2018-03-22
Navitaire Llc
Systems and Methods for Improved Data Integration in Virtual Reality Architectures
US20180373411A1
( en )
*
2017-06-21
2018-12-27
Navitaire Llc
Systems and methods for seat selection in virtual reality
US20190063881A1
( en )
2017-08-25
2019-02-28
Aurora Flight Sciences Corporation
Aerial Vehicle Interception System
US20190130783A1
( en )
2017-11-02
2019-05-02
Bell Helicopter Textron Inc.
Vr emulator using galvanic vestibular stimulation devices
US20190244537A1
( en )
2018-02-02
2019-08-08
Access Virtual, LLC
Virtual reality based pilot training system
US20210049925A1
( en )
2018-04-27
2021-02-18
Red 6 Inc.
Augmented reality for vehicle operations
US20190355272A1
( en )
*
2018-05-17
2019-11-21
Gulfstream Aerospace Corporation
Tactile responsive surface for virtual reality training device
US20200066177A1
( en )
2018-08-24
2020-02-27
Industrial Technology Research Institute
Multi-view display device and manipulation simulation device
US20200134705A1
( en )
2018-10-31
2020-04-30
The Boeing Company
Aircraft production marketplace
US11049318B2
( en )
*
2018-11-29
2021-06-29
Honda Motor Co., Ltd.
Content providing device, content providing method, and storage medium
US20210104099A1
( en )
*
2019-10-08
2021-04-08
Panasonic Avionics Corporation
Utilizing virtual reality and hi-definition camera technology to allow passengers to experience flight path
US20210381639A1
( en )
*
2020-06-04
2021-12-09
Facebook Technologies, Llc
Pass-through ratcheting mechanism
US20230360539A1
( en )
*
2022-01-20
2023-11-09
U.S. Army DEVCOM, Army Research Laboratory
System and method for calculation and display of formation flight information on augmented reality display device
US20230267849A1
( en )
*
2022-02-23
2023-08-24
The Boeing Company
Recorded physical interaction playback
US20230333552A1
( en )
*
2022-04-13
2023-10-19
Rotor Technologies, Inc.
Methods and systems for human-in-the-loop vehicular command and control using immersive synthetic vision
Non-Patent Citations (6)
* Cited by examiner, â Cited by third party
Title
Aslandere, Turgay, et al., " A Generic Virtual Reality Flight Simulator, " pp. 1-12, 2014.
Batista Da Silva et al., " Unmanned Aircraft System Coordination for Persistent Surveillance With Different Priorities, " IEEE 26th International Symposium on Industrial Electronics (ISIE), Jun. 19, 2017, 6 Pages.
Chinese Office Action, Application No. 201910987554.8, dated Jun. 28, 2023, 8 Pages.
Communication Pursuant to Article 94(3) EPC, Application No. 19199207.2-1218, dated May 5, 2023, 7 Pages.
Extended European Search Report received for Application No. 19199207.2-1222, dated Feb. 5, 2020, 8 pages.
Oberhauser, Matthias, et al., " Bridging the Gap Between Desktop Research and Full Flight Simulators for Human Factors Research, " EPCE 2015, LNAI 9174, pp. 460-471, 2015.
Also Published As
Publication number
Publication date
KR20200059142A
( en )
2020-05-28
KR102831890B1
( en )
2025-07-08
EP3654315A1
( en )
2020-05-20
EP3654315B1
( en )
2025-05-07
CN111195433B
( en )
2024-03-22
US11688297B2
( en )
2023-06-27
JP2020112780A
( en )
2020-07-27
JP7393922B2
( en )
2023-12-07
US20200160742A1
( en )
2020-05-21
US20230245585A1
( en )
2023-08-03
CN111195433A
( en )
2020-05-26
Similar Documents
Publication
Publication Date
Title
US20230245585A1
( en )
2023-08-03
Virtual reality with virtualization in trainers and test environments
EP3647956B1
( en )
2023-02-01
Generalized virtualization platform for systems using hardware abstraction software layers
CN104064071A
( en )
2014-09-24
Small fixed flight training device system
WO2016154721A1
( en )
2016-10-06
Method and system for customizing a recorded real time simulation based on simulation metadata
US20050017977A1
( en )
2005-01-27
Method and apparatus for integrating virtual environments with functional simulations via HLA protocol
EP4407587A1
( en )
2024-07-31
Master data model for aviation training
EP4398220A1
( en )
2024-07-10
Training lesson authoring and execution
Morrison et al.
2023
HUD training and extended reality solutions
KR20200099229A
( en )
2020-08-24
Avionics simulation system and method
Perz-Osowska et al.
2013
Usage of virtual technology in training given to ground engineering crew
Richards et al.
2022
OMIA: MH-60R Helicopter Desktop Crew Trainer & Software Change Experimentation Tool
Wallström et al.
2016
Emulation-based software development for embedded systems
Kenney et al.
1998
Using abstration to isolate hardware in an object-oriented simulation
Negro et al.
1999
V-22 Dynamic Display Simulation (VDDS)-avionics program risk reduction and life cycle support through the use of commercial off the shelf (COTS) equipment
Bendall et al.
1999
Rapid prototyping tools applicable to avionics requirements development and real time manned simulation
Bezdek et al.
2003
A real time simulation using linux converted from a unix application
Bendall et al.
2001
Reusable prototypes for avionics development, testing, training, and real time manned simulation
Bezdek et al.
0
Weapon System Open Architecture (WSOA) Avionics Development, Simulation, and Flight Test
CN120779771A
( en )
2025-10-14
Verification and evaluation system and method for airborne equipment
Bezdek et al.
0
Open Systems Architecture for Avionics Development, Testing, and Real Time Manned Simulation
Beavin
1996
Scaleable design for Reconfigurable Manned Interactive Crewstations
Blaser et al.
2004
Tool for Rapid Advanced Cockpit Simulation (TRACS) for next generation human space vehicles
PERZ-OSOWSKA et al.
2012
Diagnostic Simulator for the Ground Engineering Crew in Virtual Technology
Connery Jr
2012
Leveraging Virtualization Technology for Command and Control Systems Training
Hammar et al.
2012
Integrating X-Plane
Legal Events
Date
Code
Title
Description
2023-04-13
AS
Assignment
Owner name : THE BOEING COMPANY, ILLINOIS
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HOTRA, JONATHAN N.;CARR, LEIGHTON J.;HUNT, ANTONY R.;SIGNING DATES FROM 20181112 TO 20181116;REEL/FRAME:063309/0916
2023-06-07
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2024-02-29
STPP
Information on status: patent application and granting procedure in general
Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS
2024-03-08
STPP
Information on status: patent application and granting procedure in general
Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS
Free format text : AWAITING TC RESP., ISSUE FEE NOT PAID
2024-05-31
STPP
Information on status: patent application and granting procedure in general