ABSTRACT
Abstract
Techniques described herein include graphically representing mathematical functions or operators in a mixed or virtual reality display device. The graphical representations may be moved around in a virtual reality space that includes displayed source material, such as data. If the graphical representations are located along a line of sight from a virtual location of a user of the mixed or virtual reality display device to the displayed source data, then the mathematical functions or operators represented by the graphical representations are applied to the displayed source data. Results may be displayed on the graphical representation virtually located closest to the virtual location of the user. Techniques described herein allow for simultaneous collaboration among mixed or virtual reality display devices of multiple users interacting with the graphical representations in the virtual reality space.
Description
PRIORITY APPLICATION
This application claims the benefit of and priority to U.S. Provisional Patent Application No. 62/380,476, filed on Aug. 28, 2016, titled âHolomathâ, which is incorporated herein by reference.
BACKGROUND
Computing systems can help generate new environments including virtual reality environments and/or mixed reality environments. Virtual reality is an immersive experience, which simulates physical presence in a real or imagined environment. For example, a virtual reality environment can immerse a physical, real-world person with computer-generated graphics in a computer-generated, virtual scene via a virtual reality display device. Mixed reality, which can also be known as augmented reality, is a hybrid reality experience, which merges real worlds and virtual worlds. Mixed reality is a technology that produces mixed reality environments where a physical, real-world person and/or objects in physical, real-world scenes co-exist with virtual, computer-generated people and/or objects in real time. For example, a mixed reality environment can augment a physical, real-world scene and/or a physical, real-world person with computer-generated graphics in the physical, real-world scene viewed via a mixed reality display device.
Co-located and/or remotely located users can communicate via virtual reality or mixed reality technologies. Various additional and/or alternative technologies are available to enable remotely located users to communicate with one another. For instance, remotely located users can communicate via visual communication service providers that leverage online video chat, online voice calls, online video conferencing, remote desktop sharing, etc.
SUMMARY
Techniques described herein include graphically representing mathematical functions or operators in a mixed reality display device. The graphical representations may be moved around in a virtual reality space that includes displayed source material, such as data. If the graphical representations are located along a line of sight from a virtual location of a user of the mixed reality display device to the displayed source data, then the mathematical functions or operators represented by the graphical representations are applied to the displayed source data. Results may be displayed on the graphical representation virtually located closest to the virtual location of the user. Techniques described herein allow for simultaneous collaboration among mixed reality display devices of multiple users interacting with the graphical representations in the virtual reality space.
It should be appreciated that the above-described subject matter can be implemented as a computer-controlled apparatus, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. These and various other features will be apparent from a reading of the following Detailed Description and a review of the associated drawings.
This Summary is provided to introduce a selection of techniques in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended that this Summary be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
The Detailed Description is set forth with reference to the accompanying figures, in which the left-most digit of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in the same or different figures indicates similar or identical items or features.
FIG. 1 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 2 is a schematic diagram showing an example of a head mounted mixed reality display device.
FIG. 3 is a schematic diagram showing an example of a view of a mixed reality environment wherein two or more users can interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 4 is a schematic view depicting a spatial region in which a mixed reality environment is visible to a user via a display of a corresponding device, according to some examples.
FIG. 5 illustrates an example lens.
FIG. 6 is a schematic sequence of arrangements of example lenses along a sight line.
FIG. 7 is a schematic sequence of arrangements of example lenses along a sight line.
FIG. 8 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 9 is a flow diagram that illustrates an example process to cause virtual content to be presented in the mixed reality environment.
FIG. 10 is a flow diagram that illustrates an example process to cause virtual content to be presented in the mixed reality environment in different modes (e.g., presenter mode or sharing mode).
FIG. 11 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 12 is a flow diagram that illustrates an example process to cause virtual content to be presented in the mixed reality environment.
FIG. 13 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 14 is a flow diagram that illustrates an example process to cause the visibility of virtual content to be modified in a mixed reality environment.
FIG. 15 is a flow diagram that illustrates an example process to cause an interaction associated with the virtual content to be performed via one or more devices in a mixed reality environment.
FIG. 16 is a flow diagram that illustrates an example process of performing mathematical operations in a virtual or mixed reality environment.
FIG. 17 is a flow diagram that illustrates another example process of performing mathematical operations in a virtual or mixed reality environment.
DETAILED DESCRIPTION
This disclosure describes techniques for enabling two or more users in a mixed reality environment to collaborate with one another and/or with virtual content that is presented in the mixed reality environment. The techniques described herein can enhance mixed reality collaborations between users in mixed reality environments. In at least one example, the techniques are directed to mixed reality social collaborations between two or more users who are physically located in a same real scene, as described below, and the real scene is unmarked (i.e., lacking predetermined and/or machine vision-specific markings for directing interactions between the two or more users). The techniques described herein can have various applications, including but not limited to, enabling users that are located in a same real scene to share virtual content and/or interact with the virtual content in a mixed reality environment via mixed reality display devices. The techniques described herein enable enhanced user interfaces to be presented on displays of mixed reality devices thereby enhancing mixed reality collaborations between users and the mixed reality experience.
For the purposes of this discussion, physical, real-world objects (âreal objectsâ) or physical, real-world people (âreal peopleâ and/or âreal personâ) describe objects or people, respectively, that physically exist in a physical, real-world scene (âreal sceneâ) associated with a mixed reality display. Real objects and/or real people can move in and out of a field of view based on movement patterns of the real objects and/or movement of a user and/or user device. Virtual, computer-generated content (âvirtual contentâ and/or âcontent itemsâ) can describe content that is generated by one or more computing devices to supplement the real scene in a user's field of view. In at least one example, virtual content can include one or more pixels each having a respective color or brightness that are collectively presented on a display such to represent a person, object, etc. that is not physically present in a real scene. That is, in at least one example, virtual content can include graphics that are representative of objects (âvirtual objectsâ), people (âvirtual peopleâ and/or âvirtual personâ), biometric data, effects, etc. Virtual content can include two dimensional graphics, three dimensional objects, content associated with applications, etc. Virtual content can be rendered into the mixed reality environment via techniques described herein. In additional and/or alternative examples, virtual content can include computer-generated content such as sound, video, global positioning system (UPS), etc.
Mixed reality experiences offer different opportunities to affect self-perception and new ways for communication to occur. The techniques described herein enable users to interact with one another and/or with virtual content in mixed reality environments using mixed reality devices. In at least one example, the techniques described herein can enable conversational partners to share virtual content and/or interact with virtual content in mixed reality environments. While the techniques described herein are directed to mixed reality environments, as described above, mixed reality may also be known as augmented reality. Accordingly, the techniques described herein should not be construed to exclude augmented reality environments.
In various examples described herein, a mixed or virtual reality system may incorporate any of a variety of math programs or applications that host math functions or operators, such as MAPLE®, WOLFRAM MATHEMATICA® (hereinafter âMathematicaâ) or any symbolic mathematical computation program, and EXCEL®, just to name a few examples. Such examples may be described by the term âHoloMath,â which may be considered a framework for mathematical collaboration in a mixed or virtual reality. Math functions or operators may be visually (e.g., virtually) represented in a display of a mixed or virtual reality system. Moreover, the functions or operators may be visually applied to source material, such as tabulated or graphical data, and results of such application may be displayed in the mixed or virtual reality system. Such a system may allow multiple collaborators, who may be located remotely from one another, to simultaneously look at the same results and explore structure of various similar functions or operators collaboratively.
In particular examples, HoloMath may be capable of allowing visualization of discrete two-dimensional (2D) data sets and functions, mathematical operations on the above, represented by lenses that users can look through to obtain results. Additionally, HoloMath may be capable of combining two or more functions to obtain a result depending on a chosen algebraic operation, for example, and if an operation is non-commutative, HoloMath may allow for looking from opposing sides to give opposing results, and creation of three-dimensional (3D) tensor networks and their contractions.
In a 3D world of a display of a mixed or virtual reality system, elements of a mathematical expression (e.g., one or more functions and/or operators (including tensors)) may be represented by 3D virtual objects. In some examples, the 3D virtual objects are virtual lenses, which may be positioned so that as a user of the mixed or virtual reality system looks along a sight line through one or more of the virtual lenses toward displayed source material, the user may observe the source material modified by the virtual lenses. In this case, each of the virtual lenses may represent a respective function or operator. Accordingly, these functions or operators are applied to the source material if the sight line intersects these virtual lenses.
As a user virtually moves around in the 3D world, the user's sight line to the source material changes so that various virtual lenses move in or out of the sight line. As this occurs, different functions or operators may be alternately applied to the source material, based on which virtual lenses are intersected by the sight line at a particular time. Multiple users, which may be located remotely from one another, may simultaneously interact with such virtual lenses in the 3D world. Thus, individual users may have respective lines of sight so that each user may see results of different (or the same) functions or operators applied to the source material. In this fashion, collaborating users may explore a space of different alternatives in a meta-structure of mathematical expressions, for example.
In particular examples, the user interface in HoloMath may be a âToolkit,â which a user can make appear or disappear by saying âToolkit,â for example. The Toolkit may appear virtually by default at some distance (e.g., about four feet) in front of the user, and the user can reposition the Toolkit by gazing at a center cube and air tapping, just to described a specific example. Depending on the settings, these actions may make the Toolkit follow the user's gaze or hand position. To stop such movement or placement, the user may air tap again.
The Toolkit may have two panes of buttonsâthe left pane may have mathematical options, while the right pane may hold various input tools, as well as Help and Settings menu, for example.
The actions of different buttons on the right pane include, but are not limited to, Settings, Move Tool, Selection Tool, Remove Tool, Evaluation Tool, etc. A Settings tool (e.g., a cog wheel) may be used to bring up the Settings Menu, in which the user can change the HoloMath Server IP address and connect to this address. The user can also choose whether the Move tool uses the gaze or hand position. Move Tool (e.g., arrows) may enable movement of objects on air tap. Move Tool may follow either the user's gaze or hand position, depending on the option chosen in Settings menu, for example. Selection Tool may allow the selection of multiple objects. Remove Tool (e.g., trash bin) may be used to delete objects on air tap. Evaluation Tool (e.g., calculator) may be used to evaluate tensor contraction.
The buttons on the left pane may create mathematical objects. Transformation button (e.g., magnifying lens) may be used to create a virtual lens with a mathematical operation. Looking through the lens at a data set (e.g., source material), the user can see the data set transformed by the operation associated with the lens. Several example options are presented, as well as the option to create a custom lens. To create a custom lens, a user may use a naming convention such as from Mathematical. Data Sets (e.g., which may be represented by colored bars) ma
PRIORITY APPLICATION
This application claims the benefit of and priority to U.S. Provisional Patent Application No. 62/380,476, filed on Aug. 28, 2016, titled âHolomathâ, which is incorporated herein by reference.
BACKGROUND
Computing systems can help generate new environments including virtual reality environments and/or mixed reality environments. Virtual reality is an immersive experience, which simulates physical presence in a real or imagined environment. For example, a virtual reality environment can immerse a physical, real-world person with computer-generated graphics in a computer-generated, virtual scene via a virtual reality display device. Mixed reality, which can also be known as augmented reality, is a hybrid reality experience, which merges real worlds and virtual worlds. Mixed reality is a technology that produces mixed reality environments where a physical, real-world person and/or objects in physical, real-world scenes co-exist with virtual, computer-generated people and/or objects in real time. For example, a mixed reality environment can augment a physical, real-world scene and/or a physical, real-world person with computer-generated graphics in the physical, real-world scene viewed via a mixed reality display device.
Co-located and/or remotely located users can communicate via virtual reality or mixed reality technologies. Various additional and/or alternative technologies are available to enable remotely located users to communicate with one another. For instance, remotely located users can communicate via visual communication service providers that leverage online video chat, online voice calls, online video conferencing, remote desktop sharing, etc.
SUMMARY
Techniques described herein include graphically representing mathematical functions or operators in a mixed reality display device. The graphical representations may be moved around in a virtual reality space that includes displayed source material, such as data. If the graphical representations are located along a line of sight from a virtual location of a user of the mixed reality display device to the displayed source data, then the mathematical functions or operators represented by the graphical representations are applied to the displayed source data. Results may be displayed on the graphical representation virtually located closest to the virtual location of the user. Techniques described herein allow for simultaneous collaboration among mixed reality display devices of multiple users interacting with the graphical representations in the virtual reality space.
It should be appreciated that the above-described subject matter can be implemented as a computer-controlled apparatus, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. These and various other features will be apparent from a reading of the following Detailed Description and a review of the associated drawings.
This Summary is provided to introduce a selection of techniques in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended that this Summary be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
The Detailed Description is set forth with reference to the accompanying figures, in which the left-most digit of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in the same or different figures indicates similar or identical items or features.
FIG. 1 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 2 is a schematic diagram showing an example of a head mounted mixed reality display device.
FIG. 3 is a schematic diagram showing an example of a view of a mixed reality environment wherein two or more users can interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 4 is a schematic view depicting a spatial region in which a mixed reality environment is visible to a user via a display of a corresponding device, according to some examples.
FIG. 5 illustrates an example lens.
FIG. 6 is a schematic sequence of arrangements of example lenses along a sight line.
FIG. 7 is a schematic sequence of arrangements of example lenses along a sight line.
FIG. 8 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 9 is a flow diagram that illustrates an example process to cause virtual content to be presented in the mixed reality environment.
FIG. 10 is a flow diagram that illustrates an example process to cause virtual content to be presented in the mixed reality environment in different modes (e.g., presenter mode or sharing mode).
FIG. 11 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 12 is a flow diagram that illustrates an example process to cause virtual content to be presented in the mixed reality environment.
FIG. 13 is a schematic diagram showing an example environment for enabling two or more users in a mixed reality environment to interact with one another and/or with virtual content that is presented in the mixed reality environment.
FIG. 14 is a flow diagram that illustrates an example process to cause the visibility of virtual content to be modified in a mixed reality environment.
FIG. 15 is a flow diagram that illustrates an example process to cause an interaction associated with the virtual content to be performed via one or more devices in a mixed reality environment.
FIG. 16 is a flow diagram that illustrates an example process of performing mathematical operations in a virtual or mixed reality environment.
FIG. 17 is a flow diagram that illustrates another example process of performing mathematical operations in a virtual or mixed reality environment.
DETAILED DESCRIPTION
This disclosure describes techniques for enabling two or more users in a mixed reality environment to collaborate with one another and/or with virtual content that is presented in the mixed reality environment. The techniques described herein can enhance mixed reality collaborations between users in mixed reality environments. In at least one example, the techniques are directed to mixed reality social collaborations between two or more users who are physically located in a same real scene, as described below, and the real scene is unmarked (i.e., lacking predetermined and/or machine vision-specific markings for directing interactions between the two or more users). The techniques described herein can have various applications, including but not limited to, enabling users that are located in a same real scene to share virtual content and/or interact with the virtual content in a mixed reality environment via mixed reality display devices. The techniques described herein enable enhanced user interfaces to be presented on displays of mixed reality devices thereby enhancing mixed reality collaborations between users and the mixed reality experience.
For the purposes of this discussion, physical, real-world objects (âreal objectsâ) or physical, real-world people (âreal peopleâ and/or âreal personâ) describe objects or people, respectively, that physically exist in a physical, real-world scene (âreal sceneâ) associated with a mixed reality display. Real objects and/or real people can move in and out of a field of view based on movement patterns of the real objects and/or movement of a user and/or user device. Virtual, computer-generated content (âvirtual contentâ and/or âcontent itemsâ) can describe content that is generated by one or more computing devices to supplement the real scene in a user's field of view. In at least one example, virtual content can include one or more pixels each having a respective color or brightness that are collectively presented on a display such to represent a person, object, etc. that is not physically present in a real scene. That is, in at least one example, virtual content can include graphics that are representative of objects (âvirtual objectsâ), people (âvirtual peopleâ and/or âvirtual personâ), biometric data, effects, etc. Virtual content can include two dimensional graphics, three dimensional objects, content associated with applications, etc. Virtual content can be rendered into the mixed reality environment via techniques described herein. In additional and/or alternative examples, virtual content can include computer-generated content such as sound, video, global positioning system (UPS), etc.
Mixed reality experiences offer different opportunities to affect self-perception and new ways for communication to occur. The techniques described herein enable users to interact with one another and/or with virtual content in mixed reality environments using mixed reality devices. In at least one example, the techniques described herein can enable conversational partners to share virtual content and/or interact with virtual content in mixed reality environments. While the techniques described herein are directed to mixed reality environments, as described above, mixed reality may also be known as augmented reality. Accordingly, the techniques described herein should not be construed to exclude augmented reality environments.
In various examples described herein, a mixed or virtual reality system may incorporate any of a variety of math programs or applications that host math functions or operators, such as MAPLE®, WOLFRAM MATHEMATICA® (hereinafter âMathematicaâ) or any symbolic mathematical computation program, and EXCEL®, just to name a few examples. Such examples may be described by the term âHoloMath,â which may be considered a framework for mathematical collaboration in a mixed or virtual reality. Math functions or operators may be visually (e.g., virtually) represented in a display of a mixed or virtual reality system. Moreover, the functions or operators may be visually applied to source material, such as tabulated or graphical data, and results of such application may be displayed in the mixed or virtual reality system. Such a system may allow multiple collaborators, who may be located remotely from one another, to simultaneously look at the same results and explore structure of various similar functions or operators collaboratively.
In particular examples, HoloMath may be capable of allowing visualization of discrete two-dimensional (2D) data sets and functions, mathematical operations on the above, represented by lenses that users can look through to obtain results. Additionally, HoloMath may be capable of combining two or more functions to obtain a result depending on a chosen algebraic operation, for example, and if an operation is non-commutative, HoloMath may allow for looking from opposing sides to give opposing results, and creation of three-dimensional (3D) tensor networks and their contractions.
In a 3D world of a display of a mixed or virtual reality system, elements of a mathematical expression (e.g., one or more functions and/or operators (including tensors)) may be represented by 3D virtual objects. In some examples, the 3D virtual objects are virtual lenses, which may be positioned so that as a user of the mixed or virtual reality system looks along a sight line through one or more of the virtual lenses toward displayed source material, the user may observe the source material modified by the virtual lenses. In this case, each of the virtual lenses may represent a respective function or operator. Accordingly, these functions or operators are applied to the source material if the sight line intersects these virtual lenses.
As a user virtually moves around in the 3D world, the user's sight line to the source material changes so that various virtual lenses move in or out of the sight line. As this occurs, different functions or operators may be alternately applied to the source material, based on which virtual lenses are intersected by the sight line at a particular time. Multiple users, which may be located remotely from one another, may simultaneously interact with such virtual lenses in the 3D world. Thus, individual users may have respective lines of sight so that each user may see results of different (or the same) functions or operators applied to the source material. In this fashion, collaborating users may explore a space of different alternatives in a meta-structure of mathematical expressions, for example.
In particular examples, the user interface in HoloMath may be a âToolkit,â which a user can make appear or disappear by saying âToolkit,â for example. The Toolkit may appear virtually by default at some distance (e.g., about four feet) in front of the user, and the user can reposition the Toolkit by gazing at a center cube and air tapping, just to described a specific example. Depending on the settings, these actions may make the Toolkit follow the user's gaze or hand position. To stop such movement or placement, the user may air tap again.
The Toolkit may have two panes of buttonsâthe left pane may have mathematical options, while the right pane may hold various input tools, as well as Help and Settings menu, for example.
The actions of different buttons on the right pane include, but are not limited to, Settings, Move Tool, Selection Tool, Remove Tool, Evaluation Tool, etc. A Settings tool (e.g., a cog wheel) may be used to bring up the Settings Menu, in which the user can change the HoloMath Server IP address and connect to this address. The user can also choose whether the Move tool uses the gaze or hand position. Move Tool (e.g., arrows) may enable movement of objects on air tap. Move Tool may follow either the user's gaze or hand position, depending on the option chosen in Settings menu, for example. Selection Tool may allow the selection of multiple objects. Remove Tool (e.g., trash bin) may be used to delete objects on air tap. Evaluation Tool (e.g., calculator) may be used to evaluate tensor contraction.
The buttons on the left pane may create mathematical objects. Transformation button (e.g., magnifying lens) may be used to create a virtual lens with a mathematical operation. Looking through the lens at a data set (e.g., source material), the user can see the data set transformed by the operation associated with the lens. Several example options are presented, as well as the option to create a custom lens. To create a custom lens, a user may use a naming convention such as from Mathematical. Data Sets (e.g., which may be represented by colored bars) may be used to create a visualization of a discrete data set. Some examples have 2-D domains and are represented by both their matrix plots, which can be placed on walls, as well as 3D plots placed in front. To create a custom data set, a user may use, for example, the Mathematica language conventions and provide the name of the data set used in the command defining it. Function button (e.g., which may be represented by a colored curved surface) may allow for the definition of a mathematical function. A user can specify the name, but this is not necessary. Functions of two or more variables can be plotted. Tensor button (e.g., which may be represented by a cake with candles on both sides) may be used to create an abstract tensor. A user may choose from a set of predefined choices regarding such a tensor. Once one or more tensors are in a scene, a user can click on the nodes on their covariant or contravariant legs to perform contractions. To evaluate the resulting contraction, the user may either click the Evaluation Tool on the Toolkit, or say âevaluate contraction.â
Gestures and user input may be performed by any of a number of techniques. Selection of different Toolkit options as well as objects may be performed by airtapping the options or objects once, for example. In some cases, by default, the movement of 3D objects may be set to follow the user's gaze, but an option in the Settings menu (e.g., top-rightmost button in the Toolkit) may allow movement to follow hand position. In this latter case, to stop movement, the user may gaze at the object being moving and air tap to place it. To perform rotations around a vertical axis, the user may first air tap an object (with the movement tool selected), then pinch and move a hand left to right. In some examples, HoloMath allows the use of a physical BLUETOOTH® keyboard, in place of the virtual one.
In some examples, voice commands may be used as input. In the following, words in quotes represent spoken words. For example, âtoolkitâ or âtoolsâ toggles the appearance of the Toolkit in the scene. âNode placementâ toggles the mode of placing graph nodes at hand position on air tapping. Saying ânode placementâ again may stop this action before air tapping anything else to avoid placing unnecessary nodes. These nodes can be connected with each other to form graphs, by air tapping two different nodes in turn. âEvaluate contractionâ or âcomputeâ may be used to evaluate tensor contraction.
In various examples, Holomath may be implemented with or without a separate server. For example, Holomath may be run on either a single headset or be distributed on headsets of those sharing the experience.
Descriptions above (e.g., tools, implementations, etc.) are non-limiting examples and additional or alternative tools or implementations may be used.
Illustrative Environments
FIG. 1 is a schematic diagram showing an example environment 100 for enabling two or more users in a mixed reality environment to interact with one another and with virtual content that is presented in the mixed reality environment. More particularly, the example environment 100 can include a service provider 102 , one or more networks 104 , one or more users 106 (e.g., user 106 A, user 106 B, user 106 C, etc.) and one or more devices 108 (e.g., device 108 A, device 108 B, device 108 C, etc.) associated with the one or more users 106 (e.g., user 106 A, user 106 B, user 106 C, etc.).
The service provider 102 can be any entity, server(s), platform, console, computer, etc., that facilitates two or more users 106 interacting in a mixed reality environment to enable individual users (e.g., user 106 A, user 106 B, and/or user 106 C) of the two or more users 106 to interact with one another and/or with virtual content in the mixed reality environment. The service provider 102 can be implemented in a non-distributed computing environment or can be implemented in a distributed computing environment, possibly by running some modules on devices 108 or other remotely located devices. As shown, the service provider 102 can include one or more server(s) 110 , which can include one or more processing unit(s) (e.g., processor(s) 112 ) and computer- readable media 114 , such as memory. In various examples, the service provider 102 can access, receive, and/or determine authentication data from a device (e.g., device 108 A), access content data associated with virtual content items, send rendering data associated with individual virtual content items to the device (e.g., device 108 A), and cause the individual virtual content items to be presented on a display associated with the device (e.g., device 108 A). For the purpose of this discussion, rendering data may include instructions for rendering a graphical representation of a virtual content item via a display of a device (e.g., device 108 A). For instance, the rendering data may include instructions describing the geometry, viewpoint, texture, lighting, shading, etc. associated with a virtual content item. In some examples, the service provider 102 may send rendering data to devices 108 and the devices 108 can render the graphical representations via displays associated with the devices. In other examples, as described below, the service provider 102 may render frames and may send the frames to the devices 108 for presentation via the displays.
In some examples, the service provider 102 can receive frame requests from a device (e.g., device 108 A) and can send frame messages to the device (e.g., device 108 A) to mitigate latency caused by movement that occurs between sending the frame requests to the service provider 102 and receiving frame messages at and/or rendering corresponding frames via the device (e.g., device 108 A). In at least one example, the service provider 102 can receive requests from individual devices (e.g., device 108 A, device 108 B, device 108 C, etc.) of the one or more devices 108 associated with sharing virtual content items with other devices 108 (e.g., a request to view and/or access a virtual content items) and/or requests for performing interactions on the virtual content items, and the service provider 102 can synchronize communications and/or content rendering between the devices 108 to ensure that the virtual content items and interactions directed to the virtual content items are presented to corresponding users 106 at a substantially same time so that each of the users 106 has a same experience.
In some examples, the networks 104 can be any type of network known in the art, such as the Internet. Moreover, the devices 108 can communicatively couple to the networks 104 in any manner, such as by a global or local wired or wireless connection (e.g., local area network (LAN), intranet, Bluetooth, etc.). The networks 104 can facilitate communication between the server(s) 110 and the devices 108 associated with the one or more users 106 .
Examples support scenarios where device(s) that can be included in the one or more server(s) 110 can include one or more computing devices that operate in a cluster or other clustered configuration to share resources, balance load, increase performance, provide fail-over support or redundancy, or for other purposes. Device(s) included in the one or more server(s) 110 can represent, but are not limited to, desktop computers, server computers, web-server computers, personal computers, mobile computers, laptop computers, tablet computers, wearable computers, implanted computing devices, telecommunication devices, automotive computers, network enabled televisions, thin clients, terminals, game consoles, gaming devices, work stations, media players, digital video recorders (Milts), set-top boxes, cameras, integrated components for inclusion in a computing device, appliances, or any other sort of computing device.
Device(s) that can be included in the one or more server(s) 110 can include any type of computing device having one or more processing unit(s) (e.g., processor(s) 112 ) operably connected to computer- readable media 114 such as via a bus, which in some instances can include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any variety of local, peripheral, and/or independent buses. Executable instructions stored on computer- readable media 114 can include, for example, an input module 116 , a content database 118 , a content management module 120 , a frame rendering module 122 , a positioning module 124 , a math module 126 , a permissions module 128 , and one or more applications 130 , and other modules, programs, or applications that are loadable and executable by the processor(s) 112 .
Alternatively, or in addition, the functionality described herein can be performed, at least in part, by one or more hardware logic components such as accelerators. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc. Device(s) that can be included in the one or more server(s) 110 can further include one or more input/output (I/O) interface(s) coupled to the bus to allow device(s) to communicate with other devices such as input peripheral devices (e.g., a keyboard, a mouse, a pen, a game controller, a voice input device, a touch input device, gestural input device, a tracking device, a mapping device, an image camera, a time-of-flight (TOF) camera, a depth sensor, a physiological sensor, and the like) and/or output peripheral devices (e.g., a display, a printer, audio speakers, a haptic output, and the like). Such network interface(s) can include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications over a network. For simplicity, some components are omitted from the illustrated environment.
Processing unit(s) (e.g., processor(s) 112 ) can represent, for example, a CPU-type processing unit, a GPU-type processing unit, an HPU-type processing unit, a field-programmable gate array (FPGA), another class of digital signal processor (DSP), or other hardware logic components that can, in some instances, be driven by a CPU. For example, and without limitation, illustrative types of hardware logic components that can be used include Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASSPs). System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc. In various examples, the processing unit(s) (e.g., processor(s) 112 ) can execute one or more modules and/or processes to cause the server(s) 110 to perform a variety of functions, as set forth above and explained in further detail in the following disclosure. Additionally, each of the processing unit(s) (e.g., processor(s) 112 ) can possess its own local memory, which also can store program modules, program data, and/or one or more operating systems.
In at least one configuration, the computer- readable media 114 of the server(s) 110 can include components that facilitate interaction between the service provider 102 and the one or more devices 108 . The components can represent pieces of code executing on a computing device. For example, the computer- readable media 114 can include the input module 116 , the content database 118 , the content management module 120 , the frame rendering module 122 , the positioning module 124 , the math module 126 , the permissions module 128 , and the one or more applications 130 , etc. In at least some examples, the modules can be implemented as computer-readable instructions, various data structures, and so forth via at leak one processing unit(s) (e.g., processor(s) 112 ) to enable two or more users 106 in a mixed reality environment to interact with one another and with virtual content that is presented in the mixed reality environment. Functionality to perform these operations can be included in multiple devices or a single device.
Depending on the exact configuration and type of the server(s) 110 , the computer- readable media 114 can include computer storage media and/or communication media. Computer storage media can include volatile memory, nonvolatile memory, and/or other persistent and/or auxiliary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer memory is an example of computer storage media. Thus, computer storage media includes tangible and/or physical forms of media included in a device and/or hardware component that is part of a device or external to a device, including but not limited to random-access memory (RAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), phase change memory (PRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), optical cards or other optical storage media, miniature hard drives, memory cards, magnetic cassettes, magnetic tape, magnetic disk storage, magnetic cards or other magnetic storage devices or media, solid-state memory devices, storage arrays, network attached storage, storage area networks, hosted computer storage or any other storage memory, storage device, and/or storage medium that can be used to store and maintain information for access by a computing device.
In contrast, communication media can embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. The term âmodulated data signalâ means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. Such signals or carrier waves, etc. can be propagated on wired media such as a wired network or direct-wired connection, and/or wireless media such as acoustic, RF, infrared and other wireless media. As defined herein, computer storage media does not include communication media. That is, computer storage media does not include communications media consisting solely of a modulated data signal, a carrier wave, or a propagated signal, per se.
The input module 116 is configured to receive input from one or more devices 108 (e.g., device 108 A, device 108 B, device 108 C, etc.) each corresponding to a user (e.g., user 106 A, user 106 B, user 106 C, etc.). In at least one example, the input module 116 can access, receive, and/or determine authentication data from a device (e.g., device 108 A). The authentication data can correspond to a user identification and password associated with a user (e.g., user 106 A) associated with the device (e.g., device 108 A), biometric identification associated with a user (e.g., user 106 A) associated with the device (e.g., device 108 A), etc. In at least one example, the authentication data can be leveraged to determine presence of corresponding devices 108 in a mixed reality environment. For the purpose of this discussion, presence may indicate that a device (e.g., device 108 A) is located in and/or interacting with other devices (e.g., device 108 B, device 108 C, etc.) and/or virtual content in a mixed reality environment.
In additional and/or alternative examples, the authentication data can be utilized to determine virtual content items that are available to the user (e.g., user 106 A) and the user's (e.g., user 106 A) permissions corresponding to viewing and/or interacting with each of the virtual content items. In at least one example, the authentication data can be utilized for causing virtual content items to be presented in a same mixed reality environment where a user ( e.g. user 106 A) previously left the virtual content item and in a same position where the user (e.g., user 106 A) previously left the virtual content item (e.g., if a user (e.g., user 106 A) removes his or her head mounted display device (e.g., device 108 A), turns off his or her device (e.g., device 108 A), etc.).
The content database 118 is configured to store content data associated with virtual content. Content data associated with the individual virtual content items can be stored in the content database 118 . Each individual virtual content item can be associated with data indicating an owner identification, a content identification, and permissions (i.e., permissions data). Data associated with an owner of a virtual content item may identify a user (e.g., user 106 A, user 106 B, user 106 C, etc.) that generated and/or has control over the permissions associated with a virtual content item. That is, an owner of a virtual content item can correspond to a user (e.g., user 106 A, user 106 B, user 106 C, etc.) that generated and/or has control over the permissions associated with the virtual content item. Content identification can correspond to data indicating the content associated with the virtual content item. Permissions data can include information indicating which users 106 and/or corresponding devices 108 have permission to view and/or interact with the virtual content in the mixed reality environment (i.e., which users 106 the owner has shared the virtual content with). For instance, the permission data can reflect whether a virtual content item is public, private, visible by some devices (e.g., device 108 A, device 108 B, and/or device 108 C), etc. Additionally and/or alternatively, the permissions data can indicate which interactions particular users 106 can perform and/or which interactions particular users 106 are prohibited from performing. In some examples, the permissions data can be based on input from the owner of the corresponding virtual content item, as described below.
In at least one example, the user (e.g., user 106 A) associated with a device (e.g., device 108 A) that initially requests the virtual content item can be the owner of the virtual content item such that he or she can modify the permissions associated with the virtual content item. In at least one example, the owner of the virtual content item can determine which other users (e.g., user 106 B and/or user 106 C) can view the virtual content item (i.e., whether the virtual content item is visible to the other users 106 ). For instance, in an example, an owner of a virtual content item can utilize a menu (e.g., a dropdown menu, a radial menu, etc.) or other mechanisms to share the virtual content item with all users 106 in a same mixed reality environment (i.e., make the virtual content item public), share the virtual content item with some users (e.g., user 106 A, user 106 B, and/or user 106 C) in the same mixed reality environment, or not share the virtual content item with any other users 106 (i.e., make the virtual content item private). That is, in some examples, the owner of the virtual content item can determine whether a virtual content item is visible or not visible via other devices 108 . In other examples, the owner of the virtual content item can determine which other users (e.g., user 106 B and/or user 106 C) can interact with the virtual content item via corresponding devices (e.g., device 108 B and/or device 108 C) and/or which interactions are permitted.
The content management module 120 manages the ownership of virtual content items and can leverage the permissions data to determine which of the other users (e.g., user 106 B and/or user 106 C) and/or corresponding devices (e.g., device 106 B and/or user 106 C) have permission to view individual virtual content items and/or interact with individual virtual content items. That is, the content management module 120 may access the content data to determine devices 108 with which a content item has been shared and/or interactions available for each of the devices 108 . As described above, the content data may include permissions data which indicates whether a content item is public, private, or has been shared with one or more devices (e.g., device 108 B, device 108 C, etc.) and/or interactions available for each of the devices 108 .
In various examples, the frame rendering module 122 can receive frame request messages from a requesting device (e.g., device 108 A) of the one or more devices 108 . Frame request messages can include, but are not limited to, pose information associated with each eye of a user (e.g., user 106 A), a timestamp, a desired resolution, and a desired field of view. Pose information can include a position and a rotation relative to a common coordinate system (i.e., a coordinate system that is consistently defined for both the device (e.g., device 108 A) and the service provider 102 ), which for the purpose of this discussion, may be referred to as the worldspace coordinate system. A time stamp may represent a time in which the frame request message was generated and/or sent. A desired resolution may be a desired level of detail associated with rendered virtual content (i.e., a higher resolution amounts to more detail in the virtual content). In some examples, resolution can describe a pixel count in a digital image. A desired field of view may describe an extent to which the observable world is desired to be seen at any given time through a display of a mixed reality display device (e.g., device 108 A, device 108 B, device 108 C, etc.). In some examples, field of view may describe an angle of view.
The frame request message can be processed by the frame rendering module 122 to enable virtual c
CLAIMS
Claims ( 18 )
What is claimed:
1. A system comprising:
a mixed or virtual reality display device; and
a computing device communicatively coupled to the mixed or virtual reality display device, the computing device comprising:
one or more processors;
memory; and
one or more modules stored in the memory and executable by the one or more processors to perform operations comprising:
displaying source material via a display of the mixed or virtual reality display device;
displaying a graphical representation of an operator via the display of the mixed or virtual reality display device;
determining that a virtual line of sight from a virtual location of a user of the mixed or virtual reality display device to the source material intersects the graphical representation of the operator;
based, at least in part, on the virtual line of sight intersecting the graphical representation of the operator, displaying, via the display of the mixed or virtual reality display device, first results of at least one portion of the source material being modified by the operator;
displaying additional graphical representations of respective operators via the display of the mixed or virtual reality display device;
receiving real-time commands to direct at least one of positioning or ordering of the additional graphical representations of the respective operators to be located in new locations;
determining that the virtual line of sight to the source material intersects the additional graphical representations of the respective operators located in the new locations; and
based, at least in part, on the virtual line of sight intersecting one or more of the additional graphical representations of the respective operators, displaying, via the display of the mixed or virtual reality display device, second results of the at least one portion of the source material being modified by one or more of the respective operators.
2. The system as claim 1 recites, wherein the first results of the at least one portion of the source material being modified by the operator is displayed in the graphical representation of the operator.
3. The system as claim 1 recites, wherein the operator is a mathematical operator, function, scaler, or graphical operator.
4. The system as claim 1 recites, wherein the graphical representation is a virtual lens.
5. The system as claim 1 recites, wherein based at least in part on the virtual line of sight intersecting two or more of the additional graphical representations of the respective operators, then the operations further comprise:
displaying, via the display of the mixed or virtual reality display device, the second results of the at least one portion of the source material being modified by a compound operator comprising two or more of the respective operators.
6. The system as claim 5 recites, wherein the positioning or ordering of the additional graphical representations of the respective operators affects the second results of the compound operator.
7. The system as claim 1 recites, wherein the mixed or virtual reality display device is a first mixed or virtual reality display device, the operations further comprising:
determining that a second virtual line of sight from a virtual location of a second user of a second mixed or virtual reality display device to the source material intersects the graphical representation of the operator; and
based at least in part on the second virtual line of sight intersecting the graphical representation of the operator, displaying, via a second display of the second mixed or virtual reality display device, the first results of the at least one portion of the source material being modified by the operator.
8. The system as claim 7 recites, the operations further comprising:
determining that the second virtual line of sight from the second mixed or virtual reality display device to the presented source material intersects a second graphical representation of a second operator; and
based at least in part on the second virtual line of sight intersecting the second graphical representation of the second operator, displaying, via the second display of the second mixed or virtual reality display device, third results of the at least one portion of the source material being modified by the second operator.
9. The system as claim 1 recites, wherein the source material comprises tabulated data.
10. The system as claim 1 recites, wherein the source material comprises real-time data from a measuring device.
11. A method comprising:
displaying source material via a display of a mixed or virtual reality display device;
displaying a graphical representation of an operator via the display of the mixed or virtual reality display device;
determining that a virtual line of sight from a virtual location of a user of the mixed or virtual reality display device to the source material intersects the graphical representation of the operator;
based at least in part on the virtual line of sight intersecting the graphical representation of the operator, displaying, via the display of the mixed or virtual reality display device, first results of at least one portion of the source material being modified by the operator;
displaying additional graphical representations of respective operators via the display of the mixed or virtual reality display device;
receiving real-time commands to direct at least one of positioning or ordering of the additional graphical representations of the respective operators to be located in new locations;
determining that the virtual line of sight to the source material intersects the additional graphical representations of the respective operators located in the new locations; and
based, at least in part, on the virtual line of sight intersecting one or more of the additional graphical representations of the respective operators, displaying, via the display of the mixed or virtual reality display device, second results of the at least one portion of the source material being modified by one or more of the respective operators.
12. The method as claim 11 recites, wherein the first results of the at least one portion of the source material being modified by the operator is displayed in the graphical representation of the operator.
13. The method as claim 11 recites, wherein the operator is a mathematical operator, function, scaler, or graphical operator.
14. The method as claim 11 recites, further comprising, based at least in part on the virtual line of sight intersecting two or more of the additional graphical representations of the respective operators:
displaying, via the display of the mixed or virtual reality display device, the second results of the at least one portion of the source material being modified by a compound operator comprising two or more of the respective operators.
15. The method as claim 14 recites, wherein the positioning or ordering of the additional graphical representations of the respective operators affects the second results of the compound operator.
16. The method as claim 11 recites, wherein the mixed or virtual reality, display device is a first mixed or virtual reality display device, and further comprising:
determining that a second virtual line of sight from a virtual location of a second user of a second mixed or virtual reality display device to the source material intersects the graphical representation of the operator; and
based, at least in part, on the second virtual line of sight intersecting the graphical representation of the operator, displaying, via a second display of the second mixed or virtual reality display device, the first results of the at least one portion of the source material being modified by the operator.
17. The method as claim 16 recites, further comprising:
determining that the second virtual line of sight from the second mixed or virtual reality display device to the presented source material intersects a second graphical representation of a second operator; and
based, at least in part, on the second virtual line of sight intersecting the second graphical representation of the second operator, displaying, via the second display of the second mixed or virtual reality display device, third results of the at least one portion of the source material being modified by the second operator.
18. A method comprising:
displaying data via a display of a mixed or virtual reality display device;
displaying a virtual optical system including a graphical representation corresponding to one or more mathematical or logical operators via the display of the mixed or virtual reality display device;
in response to an intersection of the virtual optical system with a virtual line of sight from a virtual location of a user of the mixed or virtual reality display device to the data, displaying, via the display of the mixed or virtual reality display device, first results of at least one portion of the data being modified by the one or more mathematical or logical operators;
displaying additional graphical representations of respective operators via the display of the mixed or virtual reality display device;
receiving real-time commands to direct at least one of positioning or ordering of the additional graphical representations of the respective operators to be located in new locations;
determining that the virtual line of sight to the source material intersects the additional graphical representations of the respective operators located in the new locations; and
based, at least in part, on the virtual line of sight intersecting one or more of the additional graphical representations of the respective operators, displaying, via the display of the mixed or virtual reality display device, second results of the at least one portion of the data being modified fey one or more of the respective operators.
US15/394,517
2016-08-28
2016-12-29
Math operations in mixed or virtual reality
Active
2037-03-18
US10192363B2
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US15/394,517
US10192363B2
( en )
2016-08-28
2016-12-29
Math operations in mixed or virtual reality
Applications Claiming Priority (2)
Application Number
Priority Date
Filing Date
Title
US201662380476P
2016-08-28
2016-08-28
US15/394,517
US10192363B2
( en )
2016-08-28
2016-12-29
Math operations in mixed or virtual reality
Publications (2)
Publication Number
Publication Date
US20180061132A1
US20180061132A1 ( en )
2018-03-01
US10192363B2
true
US10192363B2 ( en )
2019-01-29
Family
ID=61240604
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US15/394,517
Active
2037-03-18
US10192363B2
( en )
2016-08-28
2016-12-29
Math operations in mixed or virtual reality
Country Status (1)
Country
Link
US
( 1 )
US10192363B2
( en )
Cited By (8)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20180331841A1
( en )
*
2017-05-12
2018-11-15
Tsunami VR, Inc.
Systems and methods for bandwidth optimization during multi-user meetings that use virtual environments
US11218522B1
( en )
2020-08-28
2022-01-04
Tmrw Foundation Ip S. Ã R.L.
Data processing system and method using hybrid system architecture for image processing tasks
US20220070239A1
( en )
*
2020-08-28
2022-03-03
Tmrw Foundation Ip S. Ã R.L.
System and method to provision cloud computing-based virtual computing resources within a virtual environment
US12034785B2
( en )
2020-08-28
2024-07-09
Tmrw Foundation Ip S.Ãr.L.
System and method enabling interactions in virtual environments with virtual presence
US12200032B2
( en )
2020-08-28
2025-01-14
Tmrw Foundation Ip S.Ãr.L.
System and method for the delivery of applications within a virtual environment
US12273402B2
( en )
2020-08-28
2025-04-08
Tmrw Foundation Ip S.Ãr.L.
Ad hoc virtual communication between approaching user graphical representations
US12273400B2
( en )
2020-08-28
2025-04-08
Tmrw Foundation Ip S.Ãr.L.
Graphical representation-based user authentication system and method
US12432265B2
( en )
2020-08-28
2025-09-30
Tmrw Group Ip
System and method for virtually broadcasting from within a virtual environment
Families Citing this family (15)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
DE112016007015T5
( en )
*
2016-07-29
2019-03-21
Mitsubishi Electric Corporation
DISPLAY DEVICE, DISPLAY CONTROL DEVICE AND DISPLAY CONTROL METHOD
US10867445B1
( en )
*
2016-11-16
2020-12-15
Amazon Technologies, Inc.
Content segmentation and navigation
CN108398787B
( en )
*
2018-03-20
2023-05-16
京ä¸æ¹ç§æéå¢è¡ä»½æéå ¬å¸
Augmented reality display device, method and augmented reality glasses
US11095947B2
( en )
*
2018-06-12
2021-08-17
Number 9, LLC
System for sharing user-generated content
US10650239B2
( en )
2018-07-25
2020-05-12
At&T Intellectual Property I, L.P.
Context-based object location via augmented reality device
CN112639685B
( en )
*
2018-09-04
2024-03-08
è¹æå ¬å¸
Display device sharing and interaction in simulated reality (SR)
US11538205B2
( en )
*
2018-09-19
2022-12-27
Chegg, Inc.
Augmented reality mathematics in learning platforms
WO2020139588A1
( en )
2018-12-24
2020-07-02
Dts, Inc.
Room acoustics simulation using deep learning image analysis
US11805176B1
( en )
2020-05-11
2023-10-31
Apple Inc.
Toolbox and context for user interactions
WO2021231080A1
( en )
2020-05-11
2021-11-18
Carnelian Laboratories Llc
Automatic determination of application state in a multi-user environment
US20220254114A1
( en )
*
2021-02-08
2022-08-11
CITA Equity Partners, LLC
Shared mixed reality and platform-agnostic format
WO2022233434A1
( en )
*
2021-05-07
2022-11-10
Telefonaktiebolaget Lm Ericsson (Publ)
Method and arrangements for graphically visualizing data transfer in a 3d virtual environment
US20230259199A1
( en )
*
2022-02-15
2023-08-17
Google Llc
Selection of real-world objects using a wearable device
US12217372B2
( en )
2022-10-17
2025-02-04
T-Mobile Usa, Inc.
Generating mixed reality content based on data from a wireless device
CN117806223B
( en )
*
2023-12-28
2024-12-03
å½æä¸ç ¤åç ¤äº¬å港å£æéå ¬å¸
An automatic ship loading control system
Citations (9)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20040104935A1
( en )
2001-01-26
2004-06-03
Todd Williamson
Virtual reality immersion system
US6842175B1
( en )
1999-04-22
2005-01-11
Fraunhofer Usa, Inc.
Tools for interacting with virtual environments
US20090187389A1
( en )
2008-01-18
2009-07-23
Lockheed Martin Corporation
Immersive Collaborative Environment Using Motion Capture, Head Mounted Display, and Cave
US20100238161A1
( en )
*
2009-03-19
2010-09-23
Kenneth Varga
Computer-aided system for 360º heads up display of safety/mission critical data
US20110063286A1
( en )
2009-09-15
2011-03-17
Palo Alto Research Center Incorporated
System for interacting with objects in a virtual environment
US20120105473A1
( en )
*
2010-10-27
2012-05-03
Avi Bar-Zeev
Low-latency fusing of virtual and real content
US8717360B2
( en )
2010-01-29
2014-05-06
Zspace, Inc.
Presenting a view within a three dimensional scene
US20140282220A1
( en )
*
2013-03-14
2014-09-18
Tim Wantland
Presenting object models in augmented reality images
US20150193979A1
( en )
2014-01-08
2015-07-09
Andrej Grek
Multi-user virtual reality interaction environment
2016
2016-12-29
US
US15/394,517
patent/US10192363B2/en
active
Active
Patent Citations (9)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US6842175B1
( en )
1999-04-22
2005-01-11
Fraunhofer Usa, Inc.
Tools for interacting with virtual environments
US20040104935A1
( en )
2001-01-26
2004-06-03
Todd Williamson
Virtual reality immersion system
US20090187389A1
( en )
2008-01-18
2009-07-23
Lockheed Martin Corporation
Immersive Collaborative Environment Using Motion Capture, Head Mounted Display, and Cave
US20100238161A1
( en )
*
2009-03-19
2010-09-23
Kenneth Varga
Computer-aided system for 360º heads up display of safety/mission critical data
US20110063286A1
( en )
2009-09-15
2011-03-17
Palo Alto Research Center Incorporated
System for interacting with objects in a virtual environment
US8717360B2
( en )
2010-01-29
2014-05-06
Zspace, Inc.
Presenting a view within a three dimensional scene
US20120105473A1
( en )
*
2010-10-27
2012-05-03
Avi Bar-Zeev
Low-latency fusing of virtual and real content
US20140282220A1
( en )
*
2013-03-14
2014-09-18
Tim Wantland
Presenting object models in augmented reality images
US20150193979A1
( en )
2014-01-08
2015-07-09
Andrej Grek
Multi-user virtual reality interaction environment
Non-Patent Citations (13)
* Cited by examiner, â Cited by third party
Title
Aguerreche, et al., " A Comparison of Three Interactive Techniques for Collaborative Manipulation of Objects in Virtual Reality ", In Proceedings of Computer Graphics International, Jun. 8, 2010, 4 pages.
Billinghurst et al., " Designing Augmented Reality Interfaces ", in the Journal of the ACM SIGGRAPH Computer Graphics-Learning through computer-generated visualization, vol. 39, Iss. 1, Feb. 2005, pp. 17-22.
Brudy, " Interactive Menus in Augmented Reality Environments ", in the Proceedings of the Media Informatics Advanced Seminar 'Beyond the Desktop', 2013, 8 pages.
Brudy, " Interactive Menus in Augmented Reality Environments ", in the Proceedings of the Media Informatics Advanced Seminar âBeyond the Desktopâ, 2013, 8 pages.
Jusufi et al., The Network Lens: Interactive Exploration of Multivariate Networks Using Visual Filtering, 2010 14th International Conference Information Visualisation, Jul. 2010, pp. 35-42.
*
Kato, et al., " Virtual Object Manipulation on a Table-Top AR Environment ", In Proceedings of IEEE and ACM International Symposium on Augmented Reality, Oct. 5, 2000, 9 pages.
Kaufmann, " Collaborative Augmented Reality in Education ", In Publication of Vienna University of Technology, Feb. 3, 2003, 4 pages.
Kaufmann, " Virtual Environments for Mathematics and Geometry Education ", In Publication of Themes in Science and Technology Education, vol. 2, Issue 1-2, Jan. 18, 2011, pp. 131-152.
Kaufmann, et al., " Learning Objects for Education With Augmented Reality ", In Proceedings of European Distance and E-Leaming Network, Jun. 14, 2006, 6 pages.
LaVoila, " From Research to Games: Interacting with 3D Space ", retrieved on Aug. 29, 2016, available at: <<http://www.gamasutra.com/view/feature/4331/from_research_to_games_.php?print=1>>, Gamasutra, Apr. 27, 2010, 9 pages.
Looser et al., An Evaluation of Virtual Lenses for Object Selection in Augmented Reality, Graphite '07 Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia, Dec. 2007, pp. 203-210.
*
Moeslund, et al., " The ARTHUR System: An Augmented Round Table ", in the Journal of Virtual Reality and Broadcasting, retrieved on Aug. 29, 2016, 6 pages.
Zhang, et al., " Social Interactions in Multiscale CVEs ", In Proceedings of 4th International Conference on Collaborative Virtual Environments, Sep. 30, 2002, 8 pages.
Cited By (10)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20180331841A1
( en )
*
2017-05-12
2018-11-15
Tsunami VR, Inc.
Systems and methods for bandwidth optimization during multi-user meetings that use virtual environments
US11218522B1
( en )
2020-08-28
2022-01-04
Tmrw Foundation Ip S. Ã R.L.
Data processing system and method using hybrid system architecture for image processing tasks
US20220070239A1
( en )
*
2020-08-28
2022-03-03
Tmrw Foundation Ip S. Ã R.L.
System and method to provision cloud computing-based virtual computing resources within a virtual environment
US12034785B2
( en )
2020-08-28
2024-07-09
Tmrw Foundation Ip S.Ãr.L.
System and method enabling interactions in virtual environments with virtual presence
US12107907B2
( en )
2020-08-28
2024-10-01
Tmrw Foundation Ip S.Ãr.L.
System and method enabling interactions in virtual environments with virtual presence
US12200032B2
( en )
2020-08-28
2025-01-14
Tmrw Foundation Ip S.Ãr.L.
System and method for the delivery of applications within a virtual environment
US12273401B2
( en )
*
2020-08-28
2025-04-08
Tmrw Foundation Ip S.Ãr.L.
System and method to provision cloud computing-based virtual computing resources within a virtual environment
US12273402B2
( en )
2020-08-28
2025-04-08
Tmrw Foundation Ip S.Ãr.L.
Ad hoc virtual communication between approaching user graphical representations
US12273400B2
( en )
2020-08-28
2025-04-08
Tmrw Foundation Ip S.Ãr.L.
Graphical representation-based user authentication system and method
US12432265B2
( en )
2020-08-28
2025-09-30
Tmrw Group Ip
System and method for virtually broadcasting from within a virtual environment
Also Published As
Publication number
Publication date
US20180061132A1
( en )
2018-03-01
Similar Documents
Publication
Publication Date
Title
US20180061132A1
( en )
2018-03-01
Math operations in mixed or virtual reality
US9818228B2
( en )
2017-11-14
Mixed reality social interaction
US12106416B2
( en )
2024-10-01
Artificial reality system architecture for concurrent application execution and collaborative 3D scene rendering
US12348730B2
( en )
2025-07-01
Reprojecting holographic video to enhance streaming bandwidth/quality
US10325407B2
( en )
2019-06-18
Attribute detection tools for mixed reality
US11430192B2
( en )
2022-08-30
Placement and manipulation of objects in augmented reality environment
CN121548800A
( en )
2026-02-17
Systems and methods for managing the display of participants in a real-time communication session
US10567449B2
( en )
2020-02-18
Apparatuses, methods and systems for sharing virtual elements
KR100963238B1
( en )
2010-06-10
Tabletop-Mobile Augmented Reality System for Personalization and Collaboration
JP7008730B2
( en )
2022-01-25
Shadow generation for image content inserted into an image
JP2009252240A
( en )
2009-10-29
System, method and program for incorporating reflection
CN109725956B
( en )
2022-02-01
Scene rendering method and related device
US20190259198A1
( en )
2019-08-22
Systems and methods for generating visual representations of a virtual object for display by user devices
CN110546951A
( en )
2019-12-06
Composite stereoscopic image content capture
KR20170091710A
( en )
2017-08-09
Digital video rendering
US11978152B2
( en )
2024-05-07
Computer-assisted graphical development tools
CN106204704A
( en )
2016-12-07
The rendering intent of three-dimensional scenic and device in virtual reality
WO2023177773A1
( en )
2023-09-21
Stereoscopic features in virtual reality
Eitsuka et al.
2013
Authoring animations of virtual objects in augmented reality-based 3d space
CN113476835B
( en )
2024-06-07
Picture display method and device
Li et al.
2013
Motion interactive system with omni-directional display
CN118689363A
( en )
2024-09-24
Method, device, electronic device and storage medium for displaying 3D images
WO2024083302A1
( en )
2024-04-25
Virtual portal between physical space and virtual space in extended reality environments
Shao
2018
Research on the Digital Promotion and Development of the Achang Forging Skills in Yunnan
Legal Events
Date
Code
Title
Description
2016-12-29
AS
Assignment
Owner name : MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LANIER, JARON;BANBURSKI, ANDRZEJ;SIGNING DATES FROM 20161218 TO 20161229;REEL/FRAME:041222/0550
2019-01-09
STCF
Information on status: patent grant
Free format text : PATENTED CASE
2022-07-13
MAFP
Maintenance fee payment
Free format text : PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY
Year of fee payment : 4