ABSTRACT
Abstract
Head mounted display systems configured to facilitate the exchange of biometric information between the head mounted display system and another computing device are disclosed. The head mounted display system can comprise a virtual or augmented reality device. After displaying a consent request regarding biometric information with the head mounted display system, a response to the consent request that includes a consent indication regarding an aspect of the biometric information can be determined. After obtaining biometric information from a wearer utilizing e.g., a camera of the head mounted display, and processing the biometric information, a biometric information processing result can be generated. The result can be communicated from the head mounted display system to another computing device.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a continuation application of U.S. patent application Ser. No. 15/462,426, filed Mar. 17, 2017, entitled âHead Mounted Display System Configured to Exchange Biometric Information,â which claims the benefit of priority to U.S. Patent Application No. 62/311,745, filed Mar. 22, 2016, entitled âBiometric Information Exchange System,â the content of which is hereby incorporated by reference herein in its entirety.
FIELD
The present disclosure relates to virtual reality and augmented reality imaging and visualization systems, and in particular to systems and methods for exchanging information between such imaging and visualization systems.
BACKGROUND
A wearable display system, such as a virtual or an augmented reality device, can capture biometric information of a user. Biometric information can include physiological characteristics and/or behavioral characteristics associated with the wearer. Physiological characteristics can include body pose (e.g., eye gaze), pupil dilation state, facial characteristics (e.g., grinning or frowning), voice characteristics, pulse rate or blood pressure, skin condition (e.g., perspiration), and so forth. Behavioral characteristics can include gait, vocal characteristics, tendencies of the wearer, and so forth.
SUMMARY
Disclosed herein are embodiments of wearable display systems such as, e.g., augmented or virtual reality display devices, configured to exchange biometric information of a wearer of the wearable display system. In some embodiments, a head mounted display system comprises: a first image capture device configured to capture a plurality of eye images of an eye of a user; a second image capture device configured to capture a plurality of outside-world images; a display configured to display virtual images to the eye of a wearer of the head mounted display; and a hardware processor in communication with the first image capture device, the second image capture device, and the display, the hardware processor programmed to: communicate a consent request to a wearer of the head mounted display using the display, the request comprising a plurality of consent categories for biometric information of the wearer; receive a response to the consent request; determine that the response includes a consent indication from the wearer of the head mounted display, the consent indication regarding a consent category that the wearer has indicated agreement; obtain the plurality of eye images of the wearer of the head mounted display captured by the first image capture device; and in response to a request for biometric information from a biometric information acquirer computing device, transmit the plurality of eye images, to the biometric information acquirer computing device, based at least partly on the consent category that the wearer has indicated agreement and the request for biometric information.
Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 depicts an illustration of an augmented reality scenario with certain virtual reality objects, and certain actual reality objects viewed by a person.
FIG. 2 schematically illustrates an example of a wearable display system.
FIG. 3 schematically illustrates aspects of an approach for simulating three-dimensional imagery using multiple depth planes.
FIG. 4 schematically illustrates an example of a waveguide stack for outputting image information to a user.
FIG. 5 shows example exit beams that may be outputted by a waveguide.
FIG. 6 is a schematic diagram showing a display system including a waveguide apparatus, an optical coupler subsystem to optically couple light to or from the waveguide apparatus, and a control subsystem, used in the generation of a multi-focal volumetric display, image, or light field.
FIG. 7 is a block diagram depicting an illustrative operating environment in which a biometric information exchange system exchanges information between wearable display systems and computing devices of potential acquirers of biometric information of a wearer.
FIG. 8A illustrates an example display page that generated by a biometric information exchange system for a wearer of a wearable display system to provide consent for viewing offers of biometric information.
FIG. 8B illustrates an example display page generated by the biometric information exchange system and a corresponding popup that can be displayed on that example display page when a wearer consents to view offers provided by the biometric information exchange system, as initiated in FIG. 8A .
FIG. 8C illustrates an example display page that generated by a biometric information exchange system for a wearer to provide consent for receiving offers regarding certain categories of biometric information of the wearer.
FIG. 8D illustrates an example display page generated by the biometric information exchange system and a corresponding popup that can be displayed on that example display page when a wearer consents to receive offers regarding certain categories of biometric information with the biometric information exchange system, as initiated in FIG. 8C .
FIG. 8E illustrates an example display page generated by a biometric information exchange system for a wearer to provide consent for sharing biometric information of the wearer.
FIG. 9 shows a flow diagram of an illustrative method for exchanging biometric information.
Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.
DETAILED DESCRIPTION
Example Augmented Reality Scenario
Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ or âaugmented realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality âVRâ scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality âARâ scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; or a mixed reality âMRâ scenario that typically involves merging real and virtual worlds to produce new environment where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR, and MR technology.
FIG. 1 depicts an illustration of an augmented reality scenario with certain virtual reality objects, and certain actual reality objects viewed by a person. FIG. 1 depicts an augmented reality scene 100 , wherein a user of an AR technology sees a real-world park- like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120 . In addition to these items, the user of the AR technology also perceives that he âseesâ a robot statue 130 standing upon the real- world platform 120 , and a cartoon-like avatar character 140 (e.g., a bumble bee) flying by which seems to be a personification of a bumble bee, even though these elements do not exist in the real world.
In order for a three-dimensional (3-D) display to produce a true sensation of depth, and more specifically, a simulated sensation of surface depth, it is desirable for each point in the display's visual field to generate the accommodative response corresponding to its virtual depth. If the accommodative response to a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues of convergence and stereopsis, the human eye may experience an accommodation conflict, resulting in unstable imaging, harmful eye strain, headaches, and, in the absence of accommodation information, almost a complete lack of surface depth.
VR, AR, and MR experiences can be provided by display systems having displays in which images corresponding to a plurality of depth planes are provided to a viewer. The images may be different for each depth plane (e.g., provide slightly different presentations of a scene or object) and may be separately focused by the viewer's eyes, thereby helping to provide the user with depth cues based on the accommodation of the eye required to bring into focus different image features for the scene located on different depth plane and/or based on observing different image features on different depth planes being out of focus. As discussed elsewhere herein, such depth cues provide credible perceptions of depth. To produce or enhance VR, AR, and MR experiences, display systems can use biometric information to enhance those experiences. For example, as described herein, a biometric information exchange system can facilitate exchange of biometric information between a wearer of the mixed reality display system and a biometric information acquirer computing device.
A mixed reality device (MRD), for example, a head mounted MR display system, may acquire substantial biometric information about the wearer of the MRD. The biometric information can include, for example, information about the wearer's head or eye movements (e.g., where the user is looking), pupil sizes (e.g., which may indicate wearer interest or excitement), timings (e.g., how long the user is looking at an item), imagery of the wearer's environment (e.g., what the wearer is looking at), the wearer's location, and so forth. The MRD may include sensors to measure the wearer's heart rate, pulse, blood pressure, skin response (e.g., perspiration), brain activity, etc. This biometric information may be stored in the device or in a networked storage system and may comprise a substantially large data set of detailed information about the wearer of the MRD. Although the foregoing has been described in the context of a mixed reality device, any wearable display system (e.g., the wearable display system 200 shown in FIG. 2 or the display system 400 in FIGS. 4 and 6 ) that captures or obtains biometric information may be utilized with the systems and methods described herein.
An entity may wish to acquire some portion of the wearer's biometric information. For example, a big box retailer may wish to determine when and for how long the wearer visits its retail stores, where the wearer travels within the store, what items and for how long the wearer looks at them, etc. As another example, a game developer may wish to determine what game characters the wearer looks at during play of the game, what selection choices the wearer makes when interacting with the characters, the wearer's level of interest, apprehension, or excitement during game play, and so forth.
Wearers of wearable display systems may have concerns regarding confidentiality of their respective biometric information. Similarly, potential acquirers of biometric information may desire to preserve such biometric information in the strictest confidence. Accordingly, both wearers and potential acquires of biometric information may desire to utilize a system or method for facilitating the exchange of biometric information that preserves the confidentiality of that information. The biometric information exchange system described herein preserves the confidentiality of biometric information. For example, described further below, biometric information exchange system may transmit a consent request regarding biometric information to a wearer of the wearable display system, before accessing any biometric information associated with that wearable display system. If the biometric information exchange system does not receive a consent indication from the wearer, no biometric information may be exchanged on the exchange system. On the other hand, if a wearer indicates agreement to an aspect of the consent request, the biometric information exchange system may facilitate access to the biometric information of that wearer, for example, via a secure connection.
Generally described, once a wearer has indicated agreement indicated to a consent request, the wearable display system may utilize a biometric information exchange system to provide biometric information to a computing device of a biometric information acquirer, for example, a biometric information acquirer computing device may obtain biometric information retrieved from a wearable display system over a secure connection during a biometric information exchange. Over that secure connection, the exchange system may grant access to biometric information of the wearer if that biometric information is preserved in the strictest confidence, whether by the secure connection in which the information is transmitted to a computing device of an acquirer of the biometric information or by an exchange system that facilitates the storage of such biometric information for future exchanges. Accordingly, the systems and methods disclosed herein facilitate the exchange of information between a wearable display system and a computing device of a potential acquirer of the biometric information.
Example Wearable Display System
FIG. 2 illustrates an example of a wearable display system 200 that can be used to present a VR, AR, or MR experience to a display system wearer or viewer 204 . The wearable display system 200 may be programmed to perform any of the applications or embodiments described herein (e.g., eye image processing). The <figure-callout id="200" label="display system" filename
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a continuation application of U.S. patent application Ser. No. 15/462,426, filed Mar. 17, 2017, entitled âHead Mounted Display System Configured to Exchange Biometric Information,â which claims the benefit of priority to U.S. Patent Application No. 62/311,745, filed Mar. 22, 2016, entitled âBiometric Information Exchange System,â the content of which is hereby incorporated by reference herein in its entirety.
FIELD
The present disclosure relates to virtual reality and augmented reality imaging and visualization systems, and in particular to systems and methods for exchanging information between such imaging and visualization systems.
BACKGROUND
A wearable display system, such as a virtual or an augmented reality device, can capture biometric information of a user. Biometric information can include physiological characteristics and/or behavioral characteristics associated with the wearer. Physiological characteristics can include body pose (e.g., eye gaze), pupil dilation state, facial characteristics (e.g., grinning or frowning), voice characteristics, pulse rate or blood pressure, skin condition (e.g., perspiration), and so forth. Behavioral characteristics can include gait, vocal characteristics, tendencies of the wearer, and so forth.
SUMMARY
Disclosed herein are embodiments of wearable display systems such as, e.g., augmented or virtual reality display devices, configured to exchange biometric information of a wearer of the wearable display system. In some embodiments, a head mounted display system comprises: a first image capture device configured to capture a plurality of eye images of an eye of a user; a second image capture device configured to capture a plurality of outside-world images; a display configured to display virtual images to the eye of a wearer of the head mounted display; and a hardware processor in communication with the first image capture device, the second image capture device, and the display, the hardware processor programmed to: communicate a consent request to a wearer of the head mounted display using the display, the request comprising a plurality of consent categories for biometric information of the wearer; receive a response to the consent request; determine that the response includes a consent indication from the wearer of the head mounted display, the consent indication regarding a consent category that the wearer has indicated agreement; obtain the plurality of eye images of the wearer of the head mounted display captured by the first image capture device; and in response to a request for biometric information from a biometric information acquirer computing device, transmit the plurality of eye images, to the biometric information acquirer computing device, based at least partly on the consent category that the wearer has indicated agreement and the request for biometric information.
Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 depicts an illustration of an augmented reality scenario with certain virtual reality objects, and certain actual reality objects viewed by a person.
FIG. 2 schematically illustrates an example of a wearable display system.
FIG. 3 schematically illustrates aspects of an approach for simulating three-dimensional imagery using multiple depth planes.
FIG. 4 schematically illustrates an example of a waveguide stack for outputting image information to a user.
FIG. 5 shows example exit beams that may be outputted by a waveguide.
FIG. 6 is a schematic diagram showing a display system including a waveguide apparatus, an optical coupler subsystem to optically couple light to or from the waveguide apparatus, and a control subsystem, used in the generation of a multi-focal volumetric display, image, or light field.
FIG. 7 is a block diagram depicting an illustrative operating environment in which a biometric information exchange system exchanges information between wearable display systems and computing devices of potential acquirers of biometric information of a wearer.
FIG. 8A illustrates an example display page that generated by a biometric information exchange system for a wearer of a wearable display system to provide consent for viewing offers of biometric information.
FIG. 8B illustrates an example display page generated by the biometric information exchange system and a corresponding popup that can be displayed on that example display page when a wearer consents to view offers provided by the biometric information exchange system, as initiated in FIG. 8A .
FIG. 8C illustrates an example display page that generated by a biometric information exchange system for a wearer to provide consent for receiving offers regarding certain categories of biometric information of the wearer.
FIG. 8D illustrates an example display page generated by the biometric information exchange system and a corresponding popup that can be displayed on that example display page when a wearer consents to receive offers regarding certain categories of biometric information with the biometric information exchange system, as initiated in FIG. 8C .
FIG. 8E illustrates an example display page generated by a biometric information exchange system for a wearer to provide consent for sharing biometric information of the wearer.
FIG. 9 shows a flow diagram of an illustrative method for exchanging biometric information.
Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.
DETAILED DESCRIPTION
Example Augmented Reality Scenario
Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ or âaugmented realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality âVRâ scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality âARâ scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; or a mixed reality âMRâ scenario that typically involves merging real and virtual worlds to produce new environment where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR, and MR technology.
FIG. 1 depicts an illustration of an augmented reality scenario with certain virtual reality objects, and certain actual reality objects viewed by a person. FIG. 1 depicts an augmented reality scene 100 , wherein a user of an AR technology sees a real-world park- like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120 . In addition to these items, the user of the AR technology also perceives that he âseesâ a robot statue 130 standing upon the real- world platform 120 , and a cartoon-like avatar character 140 (e.g., a bumble bee) flying by which seems to be a personification of a bumble bee, even though these elements do not exist in the real world.
In order for a three-dimensional (3-D) display to produce a true sensation of depth, and more specifically, a simulated sensation of surface depth, it is desirable for each point in the display's visual field to generate the accommodative response corresponding to its virtual depth. If the accommodative response to a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues of convergence and stereopsis, the human eye may experience an accommodation conflict, resulting in unstable imaging, harmful eye strain, headaches, and, in the absence of accommodation information, almost a complete lack of surface depth.
VR, AR, and MR experiences can be provided by display systems having displays in which images corresponding to a plurality of depth planes are provided to a viewer. The images may be different for each depth plane (e.g., provide slightly different presentations of a scene or object) and may be separately focused by the viewer's eyes, thereby helping to provide the user with depth cues based on the accommodation of the eye required to bring into focus different image features for the scene located on different depth plane and/or based on observing different image features on different depth planes being out of focus. As discussed elsewhere herein, such depth cues provide credible perceptions of depth. To produce or enhance VR, AR, and MR experiences, display systems can use biometric information to enhance those experiences. For example, as described herein, a biometric information exchange system can facilitate exchange of biometric information between a wearer of the mixed reality display system and a biometric information acquirer computing device.
A mixed reality device (MRD), for example, a head mounted MR display system, may acquire substantial biometric information about the wearer of the MRD. The biometric information can include, for example, information about the wearer's head or eye movements (e.g., where the user is looking), pupil sizes (e.g., which may indicate wearer interest or excitement), timings (e.g., how long the user is looking at an item), imagery of the wearer's environment (e.g., what the wearer is looking at), the wearer's location, and so forth. The MRD may include sensors to measure the wearer's heart rate, pulse, blood pressure, skin response (e.g., perspiration), brain activity, etc. This biometric information may be stored in the device or in a networked storage system and may comprise a substantially large data set of detailed information about the wearer of the MRD. Although the foregoing has been described in the context of a mixed reality device, any wearable display system (e.g., the wearable display system 200 shown in FIG. 2 or the display system 400 in FIGS. 4 and 6 ) that captures or obtains biometric information may be utilized with the systems and methods described herein.
An entity may wish to acquire some portion of the wearer's biometric information. For example, a big box retailer may wish to determine when and for how long the wearer visits its retail stores, where the wearer travels within the store, what items and for how long the wearer looks at them, etc. As another example, a game developer may wish to determine what game characters the wearer looks at during play of the game, what selection choices the wearer makes when interacting with the characters, the wearer's level of interest, apprehension, or excitement during game play, and so forth.
Wearers of wearable display systems may have concerns regarding confidentiality of their respective biometric information. Similarly, potential acquirers of biometric information may desire to preserve such biometric information in the strictest confidence. Accordingly, both wearers and potential acquires of biometric information may desire to utilize a system or method for facilitating the exchange of biometric information that preserves the confidentiality of that information. The biometric information exchange system described herein preserves the confidentiality of biometric information. For example, described further below, biometric information exchange system may transmit a consent request regarding biometric information to a wearer of the wearable display system, before accessing any biometric information associated with that wearable display system. If the biometric information exchange system does not receive a consent indication from the wearer, no biometric information may be exchanged on the exchange system. On the other hand, if a wearer indicates agreement to an aspect of the consent request, the biometric information exchange system may facilitate access to the biometric information of that wearer, for example, via a secure connection.
Generally described, once a wearer has indicated agreement indicated to a consent request, the wearable display system may utilize a biometric information exchange system to provide biometric information to a computing device of a biometric information acquirer, for example, a biometric information acquirer computing device may obtain biometric information retrieved from a wearable display system over a secure connection during a biometric information exchange. Over that secure connection, the exchange system may grant access to biometric information of the wearer if that biometric information is preserved in the strictest confidence, whether by the secure connection in which the information is transmitted to a computing device of an acquirer of the biometric information or by an exchange system that facilitates the storage of such biometric information for future exchanges. Accordingly, the systems and methods disclosed herein facilitate the exchange of information between a wearable display system and a computing device of a potential acquirer of the biometric information.
Example Wearable Display System
FIG. 2 illustrates an example of a wearable display system 200 that can be used to present a VR, AR, or MR experience to a display system wearer or viewer 204 . The wearable display system 200 may be programmed to perform any of the applications or embodiments described herein (e.g., eye image processing). The display system 200 includes a display 208 , and various mechanical and electronic modules and systems to support the functioning of the display 208 . The display 208 may be coupled to a frame 212 , which is wearable by a display system user, wearer, or viewer 204 and which is configured to position the display 208 in front of the eyes of the wearer 204 . The display 208 may be a light field display. In some embodiments, a speaker 216 is coupled to the frame 212 and positioned adjacent the ear canal of the user. In some embodiments, another speaker, not shown, is positioned adjacent the other ear canal of the user to provide for stereo/shapeable sound control. The display system 200 can include an outward-facing imaging system which observes the world in the environment around the wearer (see, e.g., the imaging system 454 shown in FIG. 4 ). The display system 200 can also include an inward-facing imaging system which can track the eye movements of the wearer (see, e.g., the inward-facing imaging system 452 shown in FIG. 4 ). The inward-facing imaging system may track either one eye's movements or both eyes' movements. The display 208 is operatively coupled 220 , such as by a wired lead or wireless connectivity, to a local data processing module 224 which may be mounted in a variety of configurations, such as fixedly attached to the frame 212 , fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user 204 (e.g., in a backpack-style configuration, in a belt-coupling style configuration).
The frame 212 can have one or more cameras attached or mounted to the frame 212 to obtain images of the wearer's eye(s). In one embodiment, the camera(s) may be mounted to the frame 212 in front of a wearer's eye so that the eye can be imaged directly. In other embodiments, the camera can be mounted along a stem of the frame 212 (e.g., near the wearer's ear). In such embodiments, the display 208 may be coated with a material that reflects light from the wearer's eye back toward the camera. The light may be infrared light, since iris features are prominent in infrared images.
The local processing and data module 224 may comprise a hardware processor, as well as non-transitory digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing, caching, and storage of data. The data may include data (a) captured from sensors (which may be, e.g., operatively coupled to the frame 212 or otherwise attached to the user 204 ), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, gyroscopes, global positioning satellite (GPS units, radio devices, voice detectors, galvanic sensors, pulse or blood pressure sensors, electrocardiographic sensors, and/or electroencephalographic sensors; and/or (b) acquired and/or processed using remote processing module 228 and/or remote data repository 232 , possibly for passage to the display 208 after such processing or retrieval. The local processing and data module 224 may be operatively coupled to the remote processing module 228 and remote data repository 232 by communication links 236 and/or 240 , such as via wired or wireless communication links, such that these remote modules 228 , 232 are available as resources to the local processing and data module 224 . The image capture device(s) can be used to capture the eye images used in the eye image processing procedures. In addition, the remote processing module 228 and remote data repository 232 may be operatively coupled to each other.
In some embodiments, the remote processing module 228 may comprise one or more processors configured to analyze and process data and/or image information such as video information captured by an image capture device. The video data may be stored locally in the local processing and data module 224 and/or in the remote data repository 232 . In some embodiments, the remote data repository 232 may comprise a digital data storage facility, which may be available through the internet or other networking configuration in a âcloudâ resource configuration. In some embodiments, all data is stored and all computations are performed in the local processing and data module 224 , allowing fully autonomous use from a remote module.
In some implementations, the local processing and data module 224 and/or the remote processing module 228 are programmed to perform embodiments of systems and methods as described herein. The image capture device can capture video for a particular application (e.g., video of the wearer's eye for an eye-tracking application or video of a wearer's hand or finger for a gesture identification application). The video can be analyzed by one or both of the processing modules 224 , 228 . In some cases, off-loading at least some of the iris code generation to a remote processing module (e.g., in the âcloudâ) may improve efficiency or speed of the computations. The parameters of the systems and methods disclosed herein can be stored in data modules 224 and/or 228 .
The results of the analysis can be used by one or both of the processing modules 224 , 228 for additional operations or processing. For example, in various applications, biometric identification, eye-tracking, recognition, or classification of gestures, objects, poses, etc. may be used by the wearable display system 200 . For example, the wearable display system 200 may analyze video captured of a hand of the wearer 204 and recognize a gesture by the wearer's hand (e.g., picking up a real or virtual object, signaling assent or dissent (e.g., âthumbs upâ, or âthumbs downâ), etc.), and the wearable display system.
In some implementations, the display system 200 can also include an electrodermal sensor (e.g., a galvanic sensor) operable to detect skin sensing activity such as, e.g., skin conductance or galvanic skin response, which may be indicative of activity in the wearer's autonomic nervous system (e.g., causing perspiration or indicating stress). The display system 200 can also include other types of electrical sensors, such as electrical sensors detecting brain activity (e.g., as performed in an electroencephalogram test) or electrical sensors detecting heart activity (e.g., as performed in an electrocardiogram test). The electrodermal sensor and/or the electrical sensor may be operatively coupled by communication links such as via a wired or wireless communication links, to the display system 200 . For example, a wired or wireless communication link may couple the electrodermal sensor and/or the electrical sensors to the local processing and data module 224 , which, in turn, is associated with any processing modules operable to process biometric information of the display system 200 . Accordingly, as an example, the electrodermal sensor can detect electrodermal activity or other measurements associated with the scan of the wearer of the display system 200 . Such biometric information can also be processed by the display system 200 .
The human visual system is complicated and providing a realistic perception of depth is challenging. Without being limited by theory, it is believed that viewers of an object may perceive the object as being three-dimensional due to a combination of vergence and accommodation. Vergence movements (e.g., rolling movements of the pupils toward or away from each other to converge the lines of sight of the eyes to fixate upon an object) of the two eyes relative to each other are closely associated with focusing (or âaccommodationâ) of the lenses of the eyes. Under normal conditions, changing the focus of the lenses of the eyes, or accommodating the eyes, to change focus from one object to another object at a different distance will automatically cause a matching change in vergence to the same distance, under a relationship known as the âaccommodation-vergence reflex.â Likewise, a change in vergence will trigger a matching change in accommodation, under normal conditions. Display systems that provide a better match between accommodation and vergence may form more realistic or comfortable simulations of three-dimensional imagery.
FIG. 3 illustrates aspects of an approach for simulating three-dimensional imagery using multiple depth planes. With reference to FIG. 3 , objects at various distances from eyes
302 and 304 on the z-axis are accommodated by the eyes
302 and 304 so that those objects are in focus. The eyes
302 and 304 assume particular accommodated states to bring into focus objects at different distances along the z-axis. Consequently, a particular accommodated state may be said to be associated with a particular one of depth planes 306 , with an associated focal distance, such that objects or parts of objects in a particular depth plane are in focus when the eye is in the accommodated state for that depth plane. In some embodiments, three-dimensional imagery may be simulated by providing different presentations of an image for each of the eyes
302 and 304 , and also by providing different presentations of the image corresponding to each of the depth planes. While shown as being separate for clarity of illustration, it will be appreciated that the fields of view of the eyes
302 and 304 may overlap, for example, as distance along the z-axis increases. In addition, while shown as flat for ease of illustration, it will be appreciated that the contours of a depth plane may be curved in physical space, such that all features in a depth plane are in focus with the eye in a particular accommodated state. Without being limited by theory, it is believed that the human eye typically can interpret a finite number of depth planes to provide depth perception. Consequently, a highly believable simulation of perceived depth may be achieved by providing, to the eye, different presentations of an image corresponding to each of these limited number of depth planes.
Example Waveguide Stack Assembly
FIG. 4 illustrates an example of a waveguide stack for outputting image information to a user. A display system 400 includes a stack of waveguides, or stacked waveguide assembly 405 that may be utilized to provide three-dimensional perception to the eye 410 or brain using a plurality of
waveguides
420 , 422 , 424 , 426 , 428 . In some embodiments, the display system 400 may correspond to system 200 of FIG. 2 , with FIG. 4 schematically showing some parts of that system 200 in greater detail. For example, in some embodiments, the waveguide assembly 405 may be integrated into the display 208 of FIG. 2 .
With continued reference to FIG. 4 , the waveguide assembly 405 may also include a plurality of
features
430 , 432 , 434 , 436 between the waveguides. In some embodiments, the
features
430 , 432 , 434 , 436 may be lenses. In some embodiments, the
features
430 , 432 , 434 , 436 may not be lenses. Rather, they may be spacers (e.g., cladding layers and/or structures for forming air gaps).
The
waveguides
420 , 422 , 424 , 426 , 428 and/or the plurality of
lenses
430 , 432 , 434 , 436 may be configured to send image information to the eye with various levels of wavefront curvature or light ray divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane.
Image injection devices
440 , 442 , 444 , 446 , 448 may be utilized to inject image information into the
waveguides
420 , 422 , 424 , 426 , 428 , each of which may be configured to distribute incoming light across each respective waveguide, for output toward the eye 410 . Light exits an output surface of the
image injection devices
440 , 442 , 444 , 446 , 448 and is injected into a corresponding input edge of the
waveguides
420 , 422 , 424 , 426 , 428 . In some embodiments, a single beam of light (e.g., a collimated beam) may be injected into each waveguide to output an entire field of cloned collimated beams that are directed toward the eye 410 at particular angles (and amounts of divergence) corresponding to the depth plane associated with a particular waveguide.
In some embodiments, the
image injection devices
440 , 442 , 444 , 446 , 442 are discrete displays that each produce image information for injection into a
corresponding waveguide
420 , 422 , 424 , 426 , 428 , respectively. In some other embodiments, the
image injection devices
440 , 442 , 446 , 446 , 448 are the output ends of a single multiplexed display which may, for example, pipe image information via one or more optical conduits (such as fiber optic cables) to each of the
image injection devices
440 , 442 , 444 , 446 , 448 .
A controller 450 controls the operation of the stacked waveguide assembly 405 and the
image injection devices
440 , 442 , 444 , 446 , 448 . In some embodiments, the controller 450 includes programming (e.g., instructions in a non-transitory computer-readable medium) that regulates the timing and provision of image information to the
waveguides
420 , 422 , 424 , 426 , 428 . In some embodiments, the controller 450 may be a single integral device, or a distributed system connected by wired or wireless communication channels. The controller 450 may be part of the processing modules 224 or 228 (illustrated in FIG. 2 ) in some embodiments. In some embodiments, the controller may be in communication with an inward-facing imaging system 452 (e.g., a digital camera), an outward-facing imaging system 454 (e.g., a digital camera), and/or a user input device 456 . The inward-facing imaging system 452 (e.g., a digital camera) can be used to capture images of the eye 410 to, for example, determine the size and/or orientation of the pupil of the eye 410 . The outward-facing imaging system 454 can be used to image a portion of the world 458 . The user can input commands to the controller 450 via the user input device 456 to interact with the display system 400 .
The
waveguides
420 , 422 , 424 , 426 , 428 may be configured to propagate light within each respective waveguide by total internal reflection (TIR). The
waveguides
420 , 422 , 424 , 426 , 428 may each be planar or have another shape (e.g., curved), with major top and bottom surfaces and edges extending between those major top and bottom surfaces. In the illustrated configuration, the
waveguides
420 , 422 , 424 , 426 , 428 may each include light extracting
optical elements
460 , 462 , 464 , 466 , 468 that are configured to extract light out of a waveguide by redirecting the light, propagating within each respective waveguide, out of the waveguide to output image information to the eye 410 . Extracted light may also be referred to as outcoupled light, and light extracting optical elements may also be referred to as outcoupling optical elements. An extracted beam of light is outputted by the waveguide at locations at which the light propagating in the waveguide strikes a light redirecting element. The light ex
CLAIMS
Claims ( 26 )
What is claimed is:
1. A computer-implemented method performed by a head mounted display system, the method comprising:
capturing eye image data using at least one image capture device of the head mounted display system, the eye image data including at least one image of at least one eye of a wearer of the head mounted display system;
determining biometric information of the wearer based at least in part on the captured eye image data, wherein the biometric information is associated with one or more physiological or behavioral characteristics of the wearer, and wherein determining the biometric information includes segmenting one or more regions of the at least one image that correspond to an iris of the at least one eye of the wearer;
electronically accessing biometric data sharing preferences associated with the wearer, wherein the biometric data sharing preferences includes a first sharing preference category and a second sharing preference category;
detecting initiation of a first action being performed by the wearer based at least in part on the determined biometric information, wherein the first action is defined by a biometric information acquirer, the initiation of the first action being detected based at least partly on a determination that the at least one eye is pointing in a particular direction that is determined based on analyzing the segmented one or more regions of the at least one image of the eye image data;
detecting termination of the first action based at least in part on the determined biometric information, wherein performance of the first action occurs over a length of time starting at a first time of the detected initiation of the first action and ending at a second time of the detected termination of the first action;
in response to a determination that the biometric data sharing preferences indicate that the wearer has not agreed to the first sharing preference category, suppressing transmission, to the biometric information acquirer, of biometric information that is associated with the first sharing preference category; and
in response to a determination that the biometric data sharing preferences indicate that the wearer has agreed to the second sharing preference category, transmitting, to the biometric information acquirer, biometric information that is (i) associated with the second consent category, and (ii) determined during performance of the first action.
2. The computer-implemented method of claim 1 , wherein the biometric data sharing preferences comprises data associated with at least one of: an offer category, a partial sharing category, or a complete sharing category.
3. The computer-implemented method of claim 1 , further comprising:
generating display instructions configured to facilitate display, on a display associated with the head mounted display system, of adjustable biometric data sharing preferences.
4. The computer-implemented method of claim 3 , further comprising:
in response to receiving an indication of an adjustment to the adjustable biometric data sharing preferences, generating display instructions configured to facilitate display of biometric exchange offers utilizing the display associated with the head mounted display system.
5. The computer-implemented method of claim 4 , wherein the biometric exchange offers comprises offers associated with biometric information associated with items from an electronic catalog or biometric information based on a location of the wearer.
6. The computer-implemented method of claim 5 , further comprising:
filtering or ranking the biometric exchange offers based on a threshold exchange rate.
7. The computer-implemented method of claim 5 , further comprising:
filtering or ranking the biometric exchange offers based on an exclusion of an aspect of biometric information.
8. The computer-implemented method of claim 1 , further comprising:
forming a secure connection between the head mounted display system and a computing device associated with the biometric information acquirer.
9. The computer-implemented method of claim 1 , wherein the performance of the first action is based at least in part on one or more of: a preconfigured time, a preconfigured time period, a location, or multiple locations.
10. The computer-implemented method of claim 1 , wherein the transmission of biometric information to the biometric information acquirer is performed while also suppressing transmission, to the biometric information acquirer, of biometric information that is (i) associated with the second consent category, and (ii) determined before initiation or after termination of the first action.
11. A computer system comprising:
a computer readable medium storing computer executable instructions; and
one or more processors in communication with the computer readable medium, and configured to execute the computer executable instructions to cause the computer system to:
receive eye image data captured using at least one image capture device of a head mounted display system, the eye image data including at least one image of at least one eye of a wearer of the head mounted display system
determine biometric information of the wearer based at least in part on the captured eye image data, wherein the biometric information is associated with one or more physiological or behavioral characteristics of the wearer, and wherein determining the biometric information includes segmenting one or more regions of the at least one image that correspond to an iris of the at least one eye of the wearer;
electronically access biometric data sharing preferences associated with the wearer, wherein the biometric data sharing preferences includes a first sharing preference category and a second sharing preference category;
detect initiation of a first action being performed by the wearer based at least in part on the determined biometric information, wherein the first action is defined by a biometric information acquirer, the initiation of the first action being detected based at least partly on a determination that the at least one eye is pointing in a particular direction that is determined based on the segmented one or more regions of the at least one image of the eye image data;
detect termination of the first action based at least in part on the determined biometric information, wherein performance of the first action occurs over a length of time starting at a first time of the detected initiation of the first action and ending at a second time of the detected termination of the first action;
in response to a determination that the biometric data sharing preferences indicate that the wearer has not agreed to the first sharing preference category, suppress transmission, to the biometric information acquirer, of biometric information that is associated with the first sharing preference category; and
in response to a determination that the biometric data sharing preferences indicate that the wearer has agreed to the second sharing preference category, transmit, to the biometric information acquirer, biometric information that is (i) associated with the second consent category, and (ii) determined during performance of the first action.
12. The computer system of claim 11 , wherein the biometric data sharing preferences comprises data associated with at least one of: an offer category, a partial sharing category, or a complete sharing category.
13. The computer system of claim 11 , wherein the one or more processors is further configured to execute the computer executable instructions to cause the computer system to:
generate display instructions configured to facilitate display, on a display associated with the head mounted display system, of adjustable biometric data sharing preferences.
14. The computer system of claim 13 , wherein the one or more processors is further configured to execute the computer executable instructions to cause the computer system to:
in response to receiving an indication of an adjustment to the adjustable biometric data sharing preferences, generate display instructions configured to facilitate display of biometric exchange offers utilizing the display associated with the head mounted display system.
15. The computer system of claim 14 , wherein the biometric exchange offers comprises offers associated with biometric information associated with items from an electronic catalog or biometric information based on a location of the wearer.
16. The computer system of claim 15 , wherein the one or more processors is further configured to execute the computer executable instructions to cause the computer system to:
filtering or ranking the biometric exchange offers based on a threshold exchange rate.
17. The computer system of claim 15 , wherein the one or more processors is further configured to execute the computer executable instructions to cause the computer system to:
filtering or ranking the biometric exchange offers based on an exclusion of an aspect of biometric information.
18. The computer system of claim 11 , wherein the one or more processors is further configured to execute the computer executable instructions to cause the computer system to:
forming a secure connection between the head mounted display system and a computing device associated with the biometric information acquirer.
19. The computer system of claim 11 , wherein the performance of the first action is based at least in part on one or more of: a preconfigured time, a preconfigured time period, a location, or multiple locations.
20. The computer system of claim 11 , wherein the transmission of biometric information to the biometric information acquirer is performed while also suppressing transmission, to the biometric information acquirer, of biometric information that is (i) associated with the second consent category, and (ii) determined before initiation or after termination of the first action.
21. The computer-implemented method of claim 1 , wherein determining the biometric information of the wearer is further based at least in part on captured surrounding environmental data.
22. The computer-implemented method of claim 1 , wherein determining the biometric information further includes generating an iris code based on the segmented one or more regions of the at least one image of the eye image data.
23. The computer-implemented method of claim 1 , wherein segmenting the one or more regions of the at least one image including determining an interior boundary of the iris and an exterior boundary of the iris as separate segments of the at least one image.
24. The computer system of claim 11 , wherein determining the biometric information of the wearer is further based at least in part on captured surrounding environmental data.
25. The computer system of claim 11 , wherein determining the biometric information further includes generating an iris code based on the segmented one or more regions of the at least one image of the eye image data.
26. The computer system of claim 11 , wherein segmenting the one or more regions of the at least one image including determining an interior boundary of the iris and an exterior boundary of the iris as separate segments of the at least one image.
US17/099,057
2016-03-22
2020-11-16
Head mounted display system configured to exchange biometric information
Active
2037-04-02
US11436625B2
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US17/099,057
US11436625B2
( en )
2016-03-22
2020-11-16
Head mounted display system configured to exchange biometric information
Applications Claiming Priority (3)
Application Number
Priority Date
Filing Date
Title
US201662311745P
2016-03-22
2016-03-22
US15/462,426
US10867314B2
( en )
2016-03-22
2017-03-17
Head mounted display system configured to exchange biometric information
US17/099,057
US11436625B2
( en )
2016-03-22
2020-11-16
Head mounted display system configured to exchange biometric information
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US15/462,426
Continuation
US10867314B2
( en )
2016-03-22
2017-03-17
Head mounted display system configured to exchange biometric information
Publications (2)
Publication Number
Publication Date
US20210073845A1
US20210073845A1 ( en )
2021-03-11
US11436625B2
true
US11436625B2 ( en )
2022-09-06
Family
ID=59898990
Family Applications (2)
Application Number
Title
Priority Date
Filing Date
US15/462,426
Active
2037-11-15
US10867314B2
( en )
2016-03-22
2017-03-17
Head mounted display system configured to exchange biometric information
US17/099,057
Active
2037-04-02
US11436625B2
( en )
2016-03-22
2020-11-16
Head mounted display system configured to exchange biometric information
Family Applications Before (1)
Application Number
Title
Priority Date
Filing Date
US15/462,426
Active
2037-11-15
US10867314B2
( en )
2016-03-22
2017-03-17
Head mounted display system configured to exchange biometric information
Country Status (10)
Country
Link
US
( 2 )
US10867314B2
( en )
EP
( 3 )
EP3979106A1
( en )
JP
( 2 )
JP7233927B2
( en )
KR
( 2 )
KR102338407B1
( en )
CN
( 2 )
CN109154983B
( en )
AU
( 2 )
AU2017236782B2
( en )
CA
( 1 )
CA3017795A1
( en )
IL
( 2 )
IL292705B2
( en )
NZ
( 1 )
NZ746117A
( en )
WO
( 1 )
WO2017165231A1
( en )
Families Citing this family (28)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
CA2962899C
( en )
2014-09-29
2022-10-04
Robert Dale Tekolste
Architectures and methods for outputting different wavelength light out of waveguides
CN104834852B
( en )
*
2015-05-04
2018-07-13
æ å·Tcl移å¨éä¿¡æéå ¬å¸
A kind of method and system that mobile terminal is unlocked based on high quality eyeprint image
AU2017236782B2
( en )
2016-03-22
2021-01-28
Magic Leap, Inc.
Head mounted display system configured to exchange biometric information
US9904058B2
( en )
2016-05-12
2018-02-27
Magic Leap, Inc.
Distributed light manipulation over imaging waveguide
CN110574099B
( en )
*
2017-05-01
2022-07-12
å®æ³¢ç¦ææ¯æéå ¬å¸
Head tracking based field sequential saccadic separation reduction
CA3006893C
( en )
*
2017-09-07
2023-01-10
The Toronto-Dominion Bank
Digital identity network interface system
US10836403B2
( en )
2017-12-04
2020-11-17
Lear Corporation
Distractedness sensing system
EP3528172B1
( en )
2018-02-19
2023-12-27
Braun GmbH
System for classifying the usage of a handheld consumer device
US10867218B2
( en )
*
2018-04-26
2020-12-15
Lear Corporation
Biometric sensor fusion to classify vehicle passenger state
US11119573B2
( en )
*
2018-09-28
2021-09-14
Apple Inc.
Pupil modulation as a cognitive control signal
CN111127537A
( en )
*
2018-10-29
2020-05-08
ææ¯è¡ä»½å ¬å¸
Method and apparatus for detecting shadows in a headset
JP6724109B2
( en )
*
2018-10-31
2020-07-15
æ ªå¼ä¼ç¤¾ãã¯ã³ã´
Information display terminal, information transmission method, computer program
WO2020150193A1
( en )
*
2019-01-17
2020-07-23
Apple Inc.
Head-mounted display with facial interface for sensing physiological conditions
CN113785258A
( en )
2019-03-22
2021-12-10
æ æ®åå±å ¬å¸ï¼æé责任åä¼ä¼ä¸
Detecting eye measurements
US12059980B2
( en )
2019-06-21
2024-08-13
Lear Corporation
Seat system and method of control
US11524691B2
( en )
2019-07-29
2022-12-13
Lear Corporation
System and method for controlling an interior environmental condition in a vehicle
US11687778B2
( en )
2020-01-06
2023-06-27
The Research Foundation For The State University Of New York
Fakecatcher: detection of synthetic portrait videos using biological signals
US11537701B2
( en )
*
2020-04-01
2022-12-27
Toyota Motor North America, Inc.
Transport related n-factor authentication
DE102021108544A1
( en )
*
2021-04-06
2022-10-06
Perfect-iD GmbH
System for user-authorized processing and release of user-attribute-related data to a requester
EP4388398B1
( en )
2021-09-24
2026-04-01
Apple Inc.
Interaction events based on physiological response to illumination
US12293837B2
( en )
2022-03-14
2025-05-06
O/D Vision Inc.
Systems and methods for artificial intelligence based warning of potential health concerns
US12381009B2
( en )
2022-03-14
2025-08-05
O/D Vision Inc.
Systems and methods for artificial intelligence based standard of care support
US12285243B2
( en )
2022-03-14
2025-04-29
O/D Vision Inc.
Multi sensor handheld medical diagnostic device
US12380735B2
( en )
*
2022-03-14
2025-08-05
O/D Vision Inc.
Systems and methods for biometric identification using patterns and blood flow characteristics of the outer eye
US12257025B2
( en )
2022-03-14
2025-03-25
O/D Vision Inc.
AI enabled multisensor connected telehealth system
CA3247296A1
( en )
*
2022-04-01
2023-10-05
Lucid Inc.
Method, system, and medium for measuring, calibrating and training psychological absorption
TWI914570B
( en )
2022-11-09
2026-02-11
å è±åéåæ¥éè¡è¡ä»½æéå ¬å¸
Speech interactive system and method thereof
US12602803B2
( en )
2024-03-28
2026-04-14
Htc Corporation
Head-mounted display and method for depth prediction
Citations (47)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20040078219A1
( en )
2001-12-04
2004-04-22
Kimberly-Clark Worldwide, Inc.
Healthcare networks with biosensors
JP2004171343A
( en )
2002-11-21
2004-06-17
Nippon Telegr & Teleph Corp <Ntt>
Distributed control method and system for personal information
US6850221B1
( en )
1995-09-05
2005-02-01
Interlink Electronics, Inc.
Trigger operated electronic device
US20060028436A1
( en )
1992-03-05
2006-02-09
Armstrong Brad A
Image controller
US20070081123A1
( en )
2005-10-07
2007-04-12
Lewis Scott W
Digital eyewear
JP2009015449A
( en )
2007-07-02
2009-01-22
Sony Corp
Biological information sharing system, biological information expression device, and biological information expression method
JP2010061402A
( en )
2008-09-03
2010-03-18
Olympus Corp
Information presentation system, program, and information storage medium
US20110098928A1
( en )
*
2009-09-04
2011-04-28
Nike, Inc.
Monitoring and Tracking Athletic Activity
US20120127062A1
( en )
2010-11-18
2012-05-24
Avi Bar-Zeev
Automatic focus improvement for augmented reality displays
US20120162549A1
( en )
2010-12-24
2012-06-28
Chunyu Gao
Ergonomic Head Mounted Display Device And Optical System
US20130082922A1
( en )
2011-09-29
2013-04-04
Samuel A. Miller
Tactile glove for human-computer interaction
US20130117377A1
( en )
2011-10-28
2013-05-09
Samuel A. Miller
System and Method for Augmented and Virtual Reality
US20130125027A1
( en )
2011-05-06
2013-05-16
Magic Leap, Inc.
Massive simultaneous remote digital presence world
US20130208234A1
( en )
2005-10-07
2013-08-15
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
US20130242262A1
( en )
2005-10-07
2013-09-19
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
US20130286048A1
( en )
2012-04-25
2013-10-31
Christian STERNITZKE
Method and system for managing data in terminal-server environments
US20130346168A1
( en )
2011-07-18
2013-12-26
Dylan T X Zhou
Wearable augmented reality eyeglass communication device including mobile phone and mobile computing via virtual touch screen gesture control and neuron command
US20140071539A1
( en )
2012-09-11
2014-03-13
Magic Leap, Inc.
Ergonomic head mounted display device and optical system
US20140118225A1
( en )
2012-10-31
2014-05-01
Robert Jerauld
Wearable emotion detection and feedback system
US20140139551A1
( en )
2012-11-21
2014-05-22
Daniel McCulloch
Augmented reality help
US20140177023A1
( en )
2012-04-05
2014-06-26
Augmented Vision Inc.
Apparatus for optical see-through head mounted display with mutual occlusion and opaqueness control capability
US20140267420A1
( en )
2013-03-15
2014-09-18
Magic Leap, Inc.
Display system and method
US20140306866A1
( en )
2013-03-11
2014-10-16
Magic Leap, Inc.
System and method for augmented and virtual reality
US20140324480A1
( en )
2013-12-19
2014-10-30
Jericho Systems Corporation
Interface and Repository for Facilitating Patient Consent
US20140337634A1
( en )
*
2013-05-08
2014-11-13
Google Inc.
Biometric Authentication Substitute For Passwords On A Wearable Computing Device
US20140341441A1
( en )
2013-05-20
2014-11-20
Motorola Mobility Llc
Wearable device user authentication
US20140375680A1
( en )
2013-06-24
2014-12-25
Nathan Ackerman
Tracking head movement when wearing mobile device
US20150016777A1
( en )
2012-06-11
2015-01-15
Magic Leap, Inc.
Planar waveguide apparatus with diffraction element(s) and system employing same
US8950867B2
( en )
2011-11-23
2015-02-10
Magic Leap, Inc.
Three dimensional virtual and augmented reality display system
US20150073907A1
( en )
*
2013-01-04
2015-03-12
Visa International Service Association
Wearable Intelligent Vision Device Apparatuses, Methods and Systems
US20150103306A1
( en )
2013-10-16
2015-04-16
Magic Leap, Inc.
Virtual or augmented reality headsets having adjustable interpupillary distance
US20150178939A1
( en )
2013-11-27
2015-06-25
Magic Leap, Inc.
Virtual and augmented reality systems and methods
US20150205126A1
( en )
2013-11-27
2015-07-23
Magic Leap, Inc.
Virtual and augmented reality systems and methods
US20150222883A1
( en )
2014-01-31
2015-08-06
Magic Leap, Inc.
Multi-focal display system and method
US20150222884A1
( en )
2014-01-31
2015-08-06
Magic Leap, Inc.
Multi-focal display system and method
KR20150093532A
( en )
2014-02-07
2015-08-18
주ìíì¬ ìì¤ì
Method and apparatus for managing information
US20150268415A1
( en )
2013-01-15
2015-09-24
Magic Leap, Inc.
Ultra-high resolution scanning fiber display
US20150302652A1
( en )
2014-04-18
2015-10-22
Magic Leap, Inc.
Systems and methods for augmented and virtual reality
US20150326570A1
( en )
2014-05-09
2015-11-12
Eyefluence, Inc.
Systems and methods for discerning eye signals and continuous biometric identification
US20150346490A1
( en )
2014-05-30
2015-12-03
Magic Leap, Inc.
Methods and systems for generating virtual content display with a virtual or augmented reality apparatus
US20150346495A1
( en )
2014-05-30
2015-12-03
Magic Leap, Inc.
Methods and system for creating focal planes in virtual and augmented reality
US20160011419A1
( en )
2010-12-24
2016-01-14
Magic Leap, Inc.
Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
US20160019360A1
( en )
*
2013-12-04
2016-01-21
Apple Inc.
Wellness aggregator
US20160026253A1
( en )
2014-03-11
2016-01-28
Magic Leap, Inc.
Methods and systems for creating virtual and augmented reality
US9310559B2
( en )
2012-06-11
2016-04-12
Magic Leap, Inc.
Multiple depth plane three-dimensional display using a wave guide reflector array projector
USD758367S1
( en )
2015-05-14
2016-06-07
Magic Leap, Inc.
Virtual reality headset
US20170278122A1
( en )
2016-03-22
2017-09-28
Magic Leap, Inc.
Head mounted display system configured to exchange biometric information
Family Cites Families (6)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20100201512A1
( en )
*
2006-01-09
2010-08-12
Harold Dan Stirling
Apparatus, systems, and methods for evaluating body movements
AU2011220382A1
( en )
*
2010-02-28
2012-10-18
Microsoft Corporation
Local advertising content on an interactive head-mounted eyepiece
US9063566B2
( en )
*
2011-11-30
2015-06-23
Microsoft Technology Licensing, Llc
Shared collaboration using display device
EP2801065A4
( en )
*
2012-01-05
2015-08-05
Visa Int Service Ass
APPARATUS, METHODS AND SYSTEMS FOR VISUAL TRANSACTION CAPTURE
IN2015KN00682A
( en )
*
2012-09-03
2015-07-17
Sensomotoric Instr Ges Für Innovative Sensorik Mbh
CN103455746B
( en )
*
2013-09-10
2016-10-05
ç¾åº¦å¨çº¿ç½ç»ææ¯ï¼å京ï¼æéå ¬å¸
head-mounted display apparatus
2017
2017-03-17
AU
AU2017236782A
patent/AU2017236782B2/en
active
Active
2017-03-17
CA
CA3017795A
patent/CA3017795A1/en
not_active
Abandoned
2017-03-17
EP
EP21211283.3A
patent/EP3979106A1/en
not_active
Withdrawn
2017-03-17
IL
IL292705A
patent/IL292705B2/en
unknown
2017-03-17
CN
CN201780031396.8A
patent/CN109154983B/en
active
Active
2017-03-17
US
US15/462,426
patent/US10867314B2/en
active
Active
2017-03-17
KR
KR1020187030516A
patent/KR102338407B1/en
active
Active
2017-03-17
EP
EP17770869.0A
patent/EP3433707B1/en
not_active
Revoked
2017-03-17
EP
EP20197270.0A
patent/EP3779740B1/en
not_active
Revoked
2017-03-17
NZ
NZ746117A
patent/NZ746117A/en
unknown
2017-03-17
WO
PCT/US2017/023037
patent/WO2017165231A1/en
not_active
Ceased
2017-03-17
CN
CN202210698201.8A
patent/CN115032795B/en
active
Active
2017-03-17
KR
KR1020217040166A
patent/KR102504245B1/en
active
Active
2017-03-17
JP
JP2018548074A
patent/JP7233927B2/en
active
Active
2018
2018-09-06
IL
IL261671A
patent/IL261671B/en
unknown
2020
2020-11-16
US
US17/099,057
patent/US11436625B2/en
active
Active
2021
2021-04-22
AU
AU2021202479A
patent/AU2021202479B2/en
active
Active
2022
2022-03-01
JP
JP2022030788A
patent/JP7416552B2/en
active
Active
Patent Citations (67)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US9081426B2
( en )
1992-03-05
2015-07-14
Anascape, Ltd.
Image controller
US20060028436A1
( en )
1992-03-05
2006-02-09
Armstrong Brad A
Image controller
US6850221B1
( en )
1995-09-05
2005-02-01
Interlink Electronics, Inc.
Trigger operated electronic device
US20040078219A1
( en )
2001-12-04
2004-04-22
Kimberly-Clark Worldwide, Inc.
Healthcare networks with biosensors
JP2004171343A
( en )
2002-11-21
2004-06-17
Nippon Telegr & Teleph Corp <Ntt>
Distributed control method and system for personal information
US20070081123A1
( en )
2005-10-07
2007-04-12
Lewis Scott W
Digital eyewear
US20130242262A1
( en )
2005-10-07
2013-09-19
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
US20130208234A1
( en )
2005-10-07
2013-08-15
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
JP2009015449A
( en )
2007-07-02
2009-01-22
Sony Corp
Biological information sharing system, biological information expression device, and biological information expression method
JP2010061402A
( en )
2008-09-03
2010-03-18
Olympus Corp
Information presentation system, program, and information storage medium
US20110098928A1
( en )
*
2009-09-04
<td item