ABSTRACT
Abstract
A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment may vary for different users and may change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes by determining whether features of the eye are at certain vertical positions relative to the HMD. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, provide feedback on the quality of the fit to the user, and take actions to reduce or minimize effects of any misalignment.
Description
PRIORITY CLAIM
This application is a continuation of U.S. patent application Ser. No. 17/255,315, entitled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND EYES OF A USERâ, filed Dec. 22, 2020, which is a 371 of international PCT App. PCT/US2019/043096, entitled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND EYES OF A USERâ, filed Jul. 23, 2019, which claims priority to: U.S. Patent Prov. App. 62/702,866, entitled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND EYES OF A USERâ and filed on Jul. 24, 2018, which is incorporated herein by reference in its entirety.
INCORPORATION BY REFERENCE
This application incorporates by reference the entirety of each of the following patent applications and publications: U.S. application Ser. No. 14/555,585 filed on Nov. 27, 2014, published on Jul. 23, 2015 as U.S. Publication No. 2015/0205126; U.S. application Ser. No. 14/690,401 filed on Apr. 18, 2015, published on Oct. 22, 2015 as U.S. Publication No. 2015/0302652; U.S. application Ser. No. 14/212,961 filed on Mar. 14, 2014, now U.S. Pat. No. 9,417,452 issued on Aug. 16, 2016; U.S. application Ser. No. 14/331,218 filed on Jul. 14, 2014, published on Oct. 29, 2015 as U.S. Publication No. 2015/0309263; U.S. Patent Publication No. 2016/0270656; U.S. Patent Publication No. 2015/0178939, published Jun. 25, 2015; U.S. Patent Publication No. 2015/0016777; U.S. patent application Ser. No. 15/274,823; U.S. patent application Ser. No. 15/296,869; U.S. patent application Ser. No. 15/717,747, filed Sep. 27, 2017; U.S. patent application Ser. No. 15/497,726, filed Apr. 26, 2017; U.S. Patent Publication No. 2017/0053165, published Feb. 23, 2017; U.S. Patent Publication No. 2017/0053166, published Feb. 23, 2017; U.S. application Ser. No. 15/341,760, filed on Nov. 2, 2016, published on May 4, 2017 as U.S. Publication No. 2017/0122725; U.S. application Ser. No. 15/341,822, filed on Nov. 2, 2016, published on May 4, 2017 as U.S. Publication No. 2017/0124928; U.S. Provisional Patent Application No. 62/618,559, filed Jan. 17, 2018; U.S. Provisional Patent Application No. 62/642,761, filed Mar. 14, 2018; and U.S. Provisional Patent Application No. 62/644,321, filed Mar. 16, 2018.
FIELD
The present disclosure relates to display systems, including virtual reality and augmented reality display systems, and, more particularly, to systems and methods for evaluating fit of a display on a user.
BACKGROUND
Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ, âaugmented realityâ, or âmixed realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or âVRâ, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or âARâ, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; a mixed reality, or âMRâ, related to merging real and virtual worlds to produce new environments where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR and MR technology.
SUMMARY
In some embodiments, a display system is configured to project light to an eye of a user to display virtual image content. The display system comprises: a frame configured to be supported on a head of the user, a head-mounted display disposed on the frame, one or more eye-tracking cameras configured to image an eye of the user, and processing electronics in communication with the head-mounted display and the one or more eye-tracking cameras. The display is configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time. The processing electronics is configured to: determine whether the head-mounted display is properly registered to the eye of the user by determining whether imaged features of the eye are within a predetermined range of vertical positions relative to the head-mounted display; and provide feedback to the user if the head-mounted display is not properly adjusted to fit the user.
In some other embodiments, a method is provided for evaluating registration of virtual image content from a head-mounted display system by a user's eye. The method comprises imaging the eye, determining whether imaged features of the eye are within a predetermined range of vertical positions relative to a head-mounted display, and providing a notification based on a determined position of the imaged features. The notification indicates at least that the head-mounted display and the eye are not properly registered.
Additional examples of embodiments are enumerated below.
Example 1. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising:
a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image an eye of the user; and processing electronics in communication with the head-mounted display and the one or more eye-tracking cameras, the processing electronics configured to:
determine whether the head-mounted display is properly registered to the eye of the user by determining whether imaged features of the eye are within a predetermined range of vertical positions relative to the head-mounted display; and provide feedback to the user if the head-mounted display is not properly adjusted to fit the user.
Example 2. The display system of Example 1, wherein the one or more eye-tracking cameras are configured to image a left eye of the user and a right eye of the user,
wherein the processing electronics are further configured to determine a left eye tracking confidence score that indicates a confidence level in a position of the left eye of the user and to determine a right eye tracking confidence score that indicates a confidence level in a position of the right eye of the user, and wherein, when one of the confidence scores is greater than the other, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on the left eye or right eye of the user associated with the greater confidence score.
Example 3. The display system of Example 1, wherein the one or more eye-tracking cameras are configured to image a left eye of the user and a right eye of the user and
wherein, when the left eye and right eye of the user are vertically offset from each other but by less than a first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on a position of the left eye and right eye of the user that is furthest from a desired vertical position.
Example 4. The display system of Example 3, wherein, when the left eye and right eye of the user are vertically offset from each other by less than a second predetermined threshold that is less than the first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on an average position of the left eye and right eye of the user.
Example 5. The display system of Example 4, wherein, when the left eye and right eye of the user are vertically offset from each other by more than the first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on the average position of the left eye and right eye of the user.
Example 6. The display system of Example 1, further comprising at least one interchangeable fit piece removably mounted to the frame and configured to adjust a fit of the frame.
Example 7. The display system of Example 6, wherein the interchangeable fit piece comprises an interchangeable nose bridge configured to adjust the fit of the frame between the frame and a nose bridge of the user.
Example 8. The display system of Example 6, wherein the interchangeable fit piece comprises an interchangeable forehead pad configured to adjust the fit of the frame between the frame and a forehead of the user.
Example 9. The display system of any of Example 6, wherein the interchangeable fit piece comprises an interchangeable back pad configured to adjust the fit of the frame between the frame and a back of the head of the user.
Example 10. The display system of any of Example 1, wherein the providing feedback to the user if the head-mounted display is not properly adjusted to fit the user comprises providing a suggestion to the user to swap out a currently-installed interchangeable fit piece for another interchangeable fit piece.
Example 11. A method for evaluating registration of virtual image content from a head-mounted display system by a user's eye, the method comprising:
imaging the eye; determining whether imaged features of the eye are within a predetermined range of vertical positions relative to a head-mounted display; and providing a notification based on a determined position of the imaged features, where the notification indicates at least that the head-mounted display and the eye are not properly registered.
Example 12. The method of Example 11, wherein determining whether the imaged features of the eye are within a predetermined range of vertical positions comprises determining positions of glints of the eye.
Example 13. The method of Example 12, further comprising determining a position of a pupil of the eye based upon the glints of the eye.
Example 14. The method of Example 11, wherein the head-mounted display system is configured to project light into the eye to display virtual image content in the field of view of the user, and wherein providing the notification comprises displaying the notification as virtual image content.
Example 15. The method of Example 11, further comprising automatically tracking a pupil of the eye over time and notifying the user when a center of rotation of the eye moves outside of the predetermined range of vertical positions.
Example 16. The method of Example 11, further comprising changing from a first field of view of the head-mounted display system to a second field of view of the head-mounted display system when the position of the eye is outside a display registration volume,
wherein the head-mounted display system comprises at least one display having the first field of view when the position of the eye is inside the display registration volume, wherein the at least one display has the second field of view when the position of the eye is outside the display registration volume, and wherein the second field of view is smaller than the first field of view.
Example 17. The method of Example 11, wherein the head-mounted display system comprises at least one interchangeable fit piece, wherein providing the notification comprises indicating that the wearable system is not properly fitted to the user and suggesting or instructing the user to replace a currently-installed interchangeable fit piece with an alternative interchangeable fit piece.
Example 18. The method of Example 17, wherein the at least one interchangeable fit piece comprises at least one fit piece selected from the group consisting of: a nose bridge pad, a forehead pad, and a back pad that goes between the wearable system and a back of a user's head.
Example 19. The method of Example 18, wherein the at least one interchangeable fit piece comprises at least one interchangeable nose bridge pad, and further comprising determining that the head-mounted display is too low with respect to the eye, and wherein providing the notification to the user further comprises prompting the user to install a larger nose bridge pad.
Example 20. The method of Example 11, further comprising:
identifying a plurality of pixels of a display of the head-mounted display system that the user is expected to perceive as dimmed as a result of the first position of the eye being outside a display registration volume; and boosting brightness of the plurality of pixels of the display relative to other pixels in the display to mitigate the expected dimming.
Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person.
FIG. 2 schematically illustrates an example of a wearable system.
FIG. 3 schematically illustrates example components of a wearable system.
FIG. 4 schematically illustrates an example of a waveguide stack of a wearable device for outputting image information to a user.
FIG. 5 schematically illustrates an example of an eye and an example coordinate system for determining an eye pose of an eye.
FIG. 6 is a schematic diagram of a wearable system that includes an eye tracking system.
FIG. 7 A is a block diagram of a wearable system that may include an eye tracking system.
FIG. 7 B is a block diagram of a render controller in a wearable system.
FIG.
PRIORITY CLAIM
This application is a continuation of U.S. patent application Ser. No. 17/255,315, entitled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND EYES OF A USERâ, filed Dec. 22, 2020, which is a 371 of international PCT App. PCT/US2019/043096, entitled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND EYES OF A USERâ, filed Jul. 23, 2019, which claims priority to: U.S. Patent Prov. App. 62/702,866, entitled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND EYES OF A USERâ and filed on Jul. 24, 2018, which is incorporated herein by reference in its entirety.
INCORPORATION BY REFERENCE
This application incorporates by reference the entirety of each of the following patent applications and publications: U.S. application Ser. No. 14/555,585 filed on Nov. 27, 2014, published on Jul. 23, 2015 as U.S. Publication No. 2015/0205126; U.S. application Ser. No. 14/690,401 filed on Apr. 18, 2015, published on Oct. 22, 2015 as U.S. Publication No. 2015/0302652; U.S. application Ser. No. 14/212,961 filed on Mar. 14, 2014, now U.S. Pat. No. 9,417,452 issued on Aug. 16, 2016; U.S. application Ser. No. 14/331,218 filed on Jul. 14, 2014, published on Oct. 29, 2015 as U.S. Publication No. 2015/0309263; U.S. Patent Publication No. 2016/0270656; U.S. Patent Publication No. 2015/0178939, published Jun. 25, 2015; U.S. Patent Publication No. 2015/0016777; U.S. patent application Ser. No. 15/274,823; U.S. patent application Ser. No. 15/296,869; U.S. patent application Ser. No. 15/717,747, filed Sep. 27, 2017; U.S. patent application Ser. No. 15/497,726, filed Apr. 26, 2017; U.S. Patent Publication No. 2017/0053165, published Feb. 23, 2017; U.S. Patent Publication No. 2017/0053166, published Feb. 23, 2017; U.S. application Ser. No. 15/341,760, filed on Nov. 2, 2016, published on May 4, 2017 as U.S. Publication No. 2017/0122725; U.S. application Ser. No. 15/341,822, filed on Nov. 2, 2016, published on May 4, 2017 as U.S. Publication No. 2017/0124928; U.S. Provisional Patent Application No. 62/618,559, filed Jan. 17, 2018; U.S. Provisional Patent Application No. 62/642,761, filed Mar. 14, 2018; and U.S. Provisional Patent Application No. 62/644,321, filed Mar. 16, 2018.
FIELD
The present disclosure relates to display systems, including virtual reality and augmented reality display systems, and, more particularly, to systems and methods for evaluating fit of a display on a user.
BACKGROUND
Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ, âaugmented realityâ, or âmixed realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or âVRâ, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or âARâ, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; a mixed reality, or âMRâ, related to merging real and virtual worlds to produce new environments where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR and MR technology.
SUMMARY
In some embodiments, a display system is configured to project light to an eye of a user to display virtual image content. The display system comprises: a frame configured to be supported on a head of the user, a head-mounted display disposed on the frame, one or more eye-tracking cameras configured to image an eye of the user, and processing electronics in communication with the head-mounted display and the one or more eye-tracking cameras. The display is configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time. The processing electronics is configured to: determine whether the head-mounted display is properly registered to the eye of the user by determining whether imaged features of the eye are within a predetermined range of vertical positions relative to the head-mounted display; and provide feedback to the user if the head-mounted display is not properly adjusted to fit the user.
In some other embodiments, a method is provided for evaluating registration of virtual image content from a head-mounted display system by a user's eye. The method comprises imaging the eye, determining whether imaged features of the eye are within a predetermined range of vertical positions relative to a head-mounted display, and providing a notification based on a determined position of the imaged features. The notification indicates at least that the head-mounted display and the eye are not properly registered.
Additional examples of embodiments are enumerated below.
Example 1. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising:
a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image an eye of the user; and processing electronics in communication with the head-mounted display and the one or more eye-tracking cameras, the processing electronics configured to:
determine whether the head-mounted display is properly registered to the eye of the user by determining whether imaged features of the eye are within a predetermined range of vertical positions relative to the head-mounted display; and provide feedback to the user if the head-mounted display is not properly adjusted to fit the user.
Example 2. The display system of Example 1, wherein the one or more eye-tracking cameras are configured to image a left eye of the user and a right eye of the user,
wherein the processing electronics are further configured to determine a left eye tracking confidence score that indicates a confidence level in a position of the left eye of the user and to determine a right eye tracking confidence score that indicates a confidence level in a position of the right eye of the user, and wherein, when one of the confidence scores is greater than the other, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on the left eye or right eye of the user associated with the greater confidence score.
Example 3. The display system of Example 1, wherein the one or more eye-tracking cameras are configured to image a left eye of the user and a right eye of the user and
wherein, when the left eye and right eye of the user are vertically offset from each other but by less than a first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on a position of the left eye and right eye of the user that is furthest from a desired vertical position.
Example 4. The display system of Example 3, wherein, when the left eye and right eye of the user are vertically offset from each other by less than a second predetermined threshold that is less than the first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on an average position of the left eye and right eye of the user.
Example 5. The display system of Example 4, wherein, when the left eye and right eye of the user are vertically offset from each other by more than the first predetermined threshold, the processing electronics are further configured to determine whether the head-mounted display is properly registered based on the average position of the left eye and right eye of the user.
Example 6. The display system of Example 1, further comprising at least one interchangeable fit piece removably mounted to the frame and configured to adjust a fit of the frame.
Example 7. The display system of Example 6, wherein the interchangeable fit piece comprises an interchangeable nose bridge configured to adjust the fit of the frame between the frame and a nose bridge of the user.
Example 8. The display system of Example 6, wherein the interchangeable fit piece comprises an interchangeable forehead pad configured to adjust the fit of the frame between the frame and a forehead of the user.
Example 9. The display system of any of Example 6, wherein the interchangeable fit piece comprises an interchangeable back pad configured to adjust the fit of the frame between the frame and a back of the head of the user.
Example 10. The display system of any of Example 1, wherein the providing feedback to the user if the head-mounted display is not properly adjusted to fit the user comprises providing a suggestion to the user to swap out a currently-installed interchangeable fit piece for another interchangeable fit piece.
Example 11. A method for evaluating registration of virtual image content from a head-mounted display system by a user's eye, the method comprising:
imaging the eye; determining whether imaged features of the eye are within a predetermined range of vertical positions relative to a head-mounted display; and providing a notification based on a determined position of the imaged features, where the notification indicates at least that the head-mounted display and the eye are not properly registered.
Example 12. The method of Example 11, wherein determining whether the imaged features of the eye are within a predetermined range of vertical positions comprises determining positions of glints of the eye.
Example 13. The method of Example 12, further comprising determining a position of a pupil of the eye based upon the glints of the eye.
Example 14. The method of Example 11, wherein the head-mounted display system is configured to project light into the eye to display virtual image content in the field of view of the user, and wherein providing the notification comprises displaying the notification as virtual image content.
Example 15. The method of Example 11, further comprising automatically tracking a pupil of the eye over time and notifying the user when a center of rotation of the eye moves outside of the predetermined range of vertical positions.
Example 16. The method of Example 11, further comprising changing from a first field of view of the head-mounted display system to a second field of view of the head-mounted display system when the position of the eye is outside a display registration volume,
wherein the head-mounted display system comprises at least one display having the first field of view when the position of the eye is inside the display registration volume, wherein the at least one display has the second field of view when the position of the eye is outside the display registration volume, and wherein the second field of view is smaller than the first field of view.
Example 17. The method of Example 11, wherein the head-mounted display system comprises at least one interchangeable fit piece, wherein providing the notification comprises indicating that the wearable system is not properly fitted to the user and suggesting or instructing the user to replace a currently-installed interchangeable fit piece with an alternative interchangeable fit piece.
Example 18. The method of Example 17, wherein the at least one interchangeable fit piece comprises at least one fit piece selected from the group consisting of: a nose bridge pad, a forehead pad, and a back pad that goes between the wearable system and a back of a user's head.
Example 19. The method of Example 18, wherein the at least one interchangeable fit piece comprises at least one interchangeable nose bridge pad, and further comprising determining that the head-mounted display is too low with respect to the eye, and wherein providing the notification to the user further comprises prompting the user to install a larger nose bridge pad.
Example 20. The method of Example 11, further comprising:
identifying a plurality of pixels of a display of the head-mounted display system that the user is expected to perceive as dimmed as a result of the first position of the eye being outside a display registration volume; and boosting brightness of the plurality of pixels of the display relative to other pixels in the display to mitigate the expected dimming.
Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person.
FIG. 2 schematically illustrates an example of a wearable system.
FIG. 3 schematically illustrates example components of a wearable system.
FIG. 4 schematically illustrates an example of a waveguide stack of a wearable device for outputting image information to a user.
FIG. 5 schematically illustrates an example of an eye and an example coordinate system for determining an eye pose of an eye.
FIG. 6 is a schematic diagram of a wearable system that includes an eye tracking system.
FIG. 7 A is a block diagram of a wearable system that may include an eye tracking system.
FIG. 7 B is a block diagram of a render controller in a wearable system.
FIG. 7 C is a block diagram of a registration observer in a head-mounted display system.
FIG. 8 A is a schematic diagram of an eye showing the eye's corneal sphere.
FIG. 8 B illustrates an example corneal glint detected by an eye-tracking camera.
FIGS. 8 C- 8 E illustrate example stages of locating a user's corneal center with an eye tracking module in a wearable system.
FIGS. 9 A- 9 C illustrate an example normalization of the coordinate system of eye tracking images.
FIGS. 9 D- 9 G illustrate example stages of locating a user's pupil center with an eye tracking module in a wearable system.
FIG. 10 illustrates an example of an eye including the eye's optical and visual axes and the eye's center of rotation.
FIG. 11 is a process flow diagram of an example of a method for using eye tracking in rendering content and providing feedback on registration in a wearable device.
FIGS. 12 A and 12 B illustrate a nominal position of a display element relative to a user's eye and illustrate a coordinate system for describing the positions of the display element and the user's eye relative to one another.
FIGS. 13 A and 13 B illustrate nominal positioning and positioning tolerances of a display element relative to a user's eye in a head-mounted display system.
FIGS. 13 C and 13 D illustrate a display registration volume and a user's eye viewing content from a display.
FIG. 14 illustrates an example of the perceived dimming of a display for various positions of a user's eye relative to the display.
FIGS. 15 A and 15 B are exploded perspective views of a head-mounted display system having interchangeable pieces such as back pads, forehead pads, and nose bridge pads to adjust fit of a head-mounted display of the display system for different users.
FIG. 16 is a process flow diagram of an example of a method for observing registration and providing feedback on registration with a head-mounted display system.
FIGS. 17 A- 17 H illustrate views of light fields projected by a display and how the intersections of the light fields may partly define a display registration volume.
FIG. 18 illustrates a top-down view of light fields projected by a display and how the intersections of the light fields may partly define a display registration volume.
FIG. 19 illustrates an example of an eye including glints and delineated regions for measuring the position of a pupil of the eye relative to a head-mounted display system.
FIG. 20 is an example of a process flow diagram for observing registration and providing feedback on registration with a head-mounted display system.
FIG. 21 is an illustrative screenshot showing fit indicators that may be provided to a wearer of a head-mounted display system.
FIGS. 22 A, 22 B, and 22 C illustrate process flow diagrams of an example of further details of a method for observing registration and providing feedback on registration with a head-mounted display system.
Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.
DETAILED DESCRIPTION
The display portion of a display system may include a head-mounted display (HMD) which may display a three-dimensional (3D) virtual object such that the object appears to be located within the user's ambient environment. As a result, the 3D virtual object may be perceived by the user in a similar manner as real world objects.
The HMD may display images by outputting spatially modulated light to the user, with the light corresponding to the virtual object. The spatially modulated light containing image information may be referred to as image light. To be perceived by the user, the image light travels from the HMD to an eye of the user, propagates through the pupil, and impinges on the eye's retina. It will be appreciated that if all or a portion of the image light for an image does not enter the pupil of the eye and/or does not impinge on the eye's retina, then the viewer would not see the image or the quality of the image may be degraded. As used herein, registration relates to the relative positioning of the display and the user's eyes. For example, a display may be said to be properly registered when the user's eyes and the display are positioned relative to one another for a desired amount of image light to enter the eye. A registration observer (e.g., a computer program) in the display device may be programmed to monitor whether the display is properly registered or positioned for the eye to receive the image light from the display.
In order to properly display content to users, e.g., by having the user's eyes positioned to receive image light, the user's eyes may need to be situated within a particular region or volume of space relative to the HMD. This volume may be referred to as the display registration volume. If the user's eyes are outside the display registration volume, display quality may be degraded (e.g., there may be dimming and/or displayed content that does not reach the users eyes). Various factors may combine to determining the positions of the user's eyes relative to the HMD and thus whether the user's eyes are situated within the desired display registration volume. As an example, anatomical variations between users may mean that the head-mounted display fits some users in a manner that places their eyes outside the display registration volume. As another example, the HMD may not be rigidly affixed to a user's head and may shift on the user's head over time, particularly when the user is moving around. As particular examples, the HMD may slip down the user's nose or tilt relative to a line (the interocular axis) between the user's eyes and, as a result, the HMD may not be able to provide desired virtual content (e.g., without some undesirable degradation) due to the shift of the display relative to the user's eyes.
Various systems and techniques described herein are at least in part directed to solving problems related to proper registration of a display to allow the viewer to view image content as desired. In some embodiments, a head-mounted display system may be configured to determine the position of an eye of the user. The display system may then determine whether the position of that eye is within a display registration volume of the head-mounted display system. Determining the position of the eye may include determining the position of a representative pointer volume associated with the eye e.g., the center of rotation of the eye. Determining whether the position of the eye is within the display registration volume may include determining whether the center of rotation of the eye is within the display registration volume. As discussed herein, the center of rotation of the eye may be determined using an inward-facing imaging system configured to image the eye. In addition, in some embodiments, the display registration volume is an imaginary volume associated with proper fit of the head-mounted display system relative to the user's eye. For example, the display registration volume may be a volume defined by a projection from the surface of the head-mounted display system outputting image light. More specifically, the display registration volume may be a three-dimensional geometric shape that tapers from a base to an apex. The shape of the display registration volume's base may be defined at least in part by the geometry of the display, and the depth of the display registration volume (i.e., the distance from base to apex on the z-axis) may be at least in part defined by the field of view (FOV) of the display. For example, a round or circular display may yield a conical display registration volume, and a polygonal display may yield a pyramidal display registration volume. As an additional example, a display with a larger FOV may yield a display registration volume having a smaller depth than a display with a smaller FOV. In some embodiments, the display registration volume may have the general shape of a truncated cone or pyramid. For example, the display registration volume may have the general shape of a frustum, e.g., a frustum of a pyramid such as a rectangular pyramid.
In some embodiments, an inward-facing imaging system of the head-mounted display system may acquire images of the user's face, including their eyes. The inward-facing imaging system may be an eye-tracking system, which may be mounted on a frame of the head-mounted display. The head-mounted display system may analyze the images to determine the relative position of the user's eyes and the HMD, and whether the position of each of the user's eyes falls within the display registration volume for that eye. Based on this information, the head-mounted display system may notify the user to adjust the fit of the HMD. For example, the notification may inform the user that the device has slipped and needs adjustment or a suggestion to make an adjustment of the HMD. In some embodiments, the head-mounted display system may take steps to mitigate any display degradation caused by misalignment of the HMD to the user, such as by boosting brightness in areas that would otherwise be dimmed by misalignment or by moving virtual content. Accordingly, such embodiments of the HMD may assist users with properly fitting the HMD and mitigating issues caused by improper fit of the HMD, such as when the HMD slips, moves, or tilts relative to the user's head.
Advantageously, the analysis of registration may be performed automatically utilizing images acquired from the inward-facing imaging system and information regarding the display registration volume stored or accessible by the display system. As a result, the fit of the HMD may be corrected upon first using the HMD, and optionally also during the course of continued usage of the HMD to ensure a high level of image quality in the use of the head-mounted display system.
Accordingly, a variety of implementations of systems and methods for observing registration of a head-mounted display system and taking action in response to the observed registration are provided herein. For example, the display system may be configured to observe registration by determining a center of rotation of a user's eyes, determining boundaries or location of a display system's registration volume, and determining whether the center of rotation is within that registration volume. It will be appreciated that the registration volume may be calculated by the display system and/or may be provided as predetermined information accessible by the display system. In some embodiments, in response to the observed registration, the display system may provide feedback to the user regarding whether and/or how registration may be improved.
As another example, the display system may be configured to observe registration and provide user feedback by estimating whether the center of rotation is within the registration volume by imaging the eye, but without specifically calculating the position of the center of rotation relative to the registration volume. Rather, the display system may be configured to image the eye and, based on those images, determine deviations of various features of the eyes from the desired orientations of those features. The display system may then make a determination of what particular adjustments may be made to the fit of the display system. The adjustments may be correlated with associated deviations of the various features of the eyes from their desired orientations. The desired orientations may be the orientations of those features when the centers of rotation of the eyes are within the registration volume. The adjustments may be adjustments correlated with addressing particular deviations and may include changing physical parts of the display system so that the display system sits on the user's head such that the centers of rotation of the eyes may be assumed to be in the desired registration volume in some embodiments.
Reference will now be made to the drawings, in which like reference numerals refer to like parts throughout. Unless indicated otherwise, the drawings are schematic and not necessarily drawn to scale.
Examples of 3D Displays of Wearable Systems
A wearable system (also referred to herein as a head-mounted display system or as an augmented reality (AR) system) may be configured to present 2D or 3D virtual images to a user. The images may be still images, frames of a video, or a video, in combination or the like. At least a portion of the wearable system may be implemented on a wearable device that may present a VR, AR, or MR environment, alone or in combination, for user interaction. The wearable device may be used interchangeably as an AR device (ARD). Further, for the purpose of the present disclosure, the term âARâ is used interchangeably with the term âMRâ.
FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person. In FIG. 1 , an MR scene 100 is depicted wherein a user of an MR technology sees a real-world park- like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120 . In addition to these items, the user of the MR technology also perceives that he âseesâ a robot statue 130 standing upon the real- world platform 120 , and a cartoon- like avatar character 140 flying by which seems to be a personification of a bumble bee, even though these elements do not exist in the real world.
In order for the 3D display to produce a true sensation of depth, and more specifically, a simulated sensation of surface depth, it may be desirable for each point in the display's visual field to generate an accommodative response corresponding to its virtual depth. If the accommodative response to a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues of convergence and stereopsis, the human eye may experience an accommodation conflict, resulting in unstable imaging, harmful eye strain, headaches, and, in the absence of accommodation information, almost a complete lack of surface depth.
VR, AR, and MR experiences may be provided by display systems having displays in which images corresponding to a plurality of depth planes are provided to a viewer. The images may be different for each depth plane (e.g., provide slightly different presentations of a scene or object) and may be separately focused by the viewer's eyes, thereby helping to provide the user with depth cues based on the accommodation of the eye required to bring into focus different image features for the scene located on different depth plane or based on observing different image features on different depth planes being out of focus. As discussed elsewhere herein, such depth cues provide credible perceptions of depth.
FIG. 2 illustrates an example of wearable system 200 which may be configured to provide an AR/VR/MR scene. The wearable system 200 may also be referred to as the AR system 200 . The wearable system 200 includes a display 220 , and various mechanical and electronic modules and systems to support the functioning of display 220 . The display 220 may be coupled to a frame 230 , which is wearable by a user, wearer, or viewer 210 . The display 220 may be positioned in front of the eyes of the user 210 . The display 220 may present AR/VR/MR content to a user. Because the display 220 may be worn on the head of the user 210 , it may also be referred to as a head-mounted display (HMD) and the wearable system 200 , comprising the display 220 , may also be referred to as a head-mounted display system.
In some embodiments, a speaker 240 is coupled to the frame 230 and positioned adjacent the ear canal of the user (in some embodiments, another speaker, not shown, is positioned adjacent the other ear canal of the user to provide for stereo/shapeable sound control). The display 220 may include an audio sensor (e.g., a microphone) 232 for detecting an audio stream from the environment and capture ambient sound. In some embodiments, one or more other audio sensors, not shown, are positioned to provide stereo sound reception. Stereo sound reception may be used to determine the location of a sound source. The wearable system 200 may perform voice or speech recognition on the audio stream.
The wearable system 200 may include an outward-facing imaging system 464 (shown in FIG. 4 ) which observes the world in the environment around the user. The wearable system 200 may also include an inward-facing imaging system 462 (shown in FIG. 4 ) which may track the eye movements of the user. The inward-facing imaging system may track either one eye's movements or both eyes' movements. The inward-facing imaging system 462 may be attached to the frame 230 and may be in electrical communication with the processing modules
260 or 270 , which may process image information acquired by the inward-facing imaging system to determine, e.g., the pupil diameters or orientations of the eyes, eye movements or eye pose of the user 210 . The inward-facing imaging system 462 may include one or more cameras. For example, at least one camera may be used to image each eye. The images acquired by the cameras may be used to determine pupil size or eye pose for each eye separately, thereby allowing presentation of image information to each eye to be dynamically tailored to that eye.
As an example, the wearable system 200 may use the outward-facing imaging system 464 or the inward-facing imaging system 462 to acquire images of a pose of the user. The images may be still images, frames of a video, or a video.
The display 220 may be operatively coupled 250 , such as by a wired lead or wireless connectivity, to a local data processing module 260 which may be mounted in a variety of configurations, such as fixedly attached to the frame 230 , fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user 210 (e.g., in a backpack-style configuration, in a belt-coupling style configuration).
The local processing and data module 260 may comprise a hardware processor, as well as digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing, caching, and storage of data. The data may include data a) captured from sensors (which may be, e.g., operatively coupled to the frame 230 or otherwise attached to the user 210 ), such as image capture devices (e.g., cameras in the inward-facing imaging system or the outward-facing imaging system), audio sensors (e.g., microphones), inertial measurement units (IMUs), accelerometers, compasses, global positioning system (GPS) units, radio devices, or gyroscopes; or b) acquired or processed using remote processing module 270 or remote data repository 280 , possibly for passage to the display 220 after such processing or retrieval. The local processing and data module 260 may be operatively coupled by communication links
262 or 264 , such as via wired or wireless communication links, to the remote processing module 270 or remote data repository 280 such that these remote modules are available as resources to the local processing and data module 260 . In addition, remote processing module 280 and remote data repository 280 may be operatively coupled to each other.
In some embodiments, the remote processing module 270 may comprise one or more processors configured to analyze and process data or image information. In some embodiments, the remote data repository 280 may comprise a digital data storage facility, which may be available through the internet or other networking configuration in a âcloudâ resource configuration. In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing fully autonomous use from a remote module.
Example Components of A Wearable System
FIG. 3 schematically illustrates example components of a wearable system. FIG. 3 shows a wearable system 200 which may include a display 220 and a frame 230 . A blown-up view 202 schematically illustrates various components of the wearable system 200 . In certain implements, one or more of the components illustrated in FIG. 3 may be part of the display 220 . The various components alone or in combination may collect a variety of data (such as e.g., audio or visual data) associated with the user of the wearable system 200 or the user's environment. It should be appreciated that other embodiments may have additional or fewer components depending on the application for which the wearable system is used. Nevertheless, FIG. 3 provides a basic idea of some of the various components and types of data that may be collected, analyzed, and stored through the wearable system.
FIG. 3 shows an example wearable system 200 which may include the display 220 . The display 220 may comprise a display lens 226 that may be mounted to a user's head or a housing or frame 230 , which corresponds to the frame 230 . The display lens 226 may comprise one or more transparent mirrors positioned by the housing 230 in front of the user's eyes
302 , 304 and may be configured to bounce projected light 338 into the eyes
302 , 304 and facilitate beam shaping, while also allowing for transmission of at least some light from the local environment. The wavefront of the projected light beam 338 may be bent or focused to coincide with a desired focal distance of the projected light. As illustrated, two wide-field-of-view machine vision cameras 316 (also referred to as world cameras) may be coupled to the housing 230 to image the environment around the user. These cameras 316 may be dual capture visible light/non-visible (e.g., infrared) light cameras. The cameras 316 may be part of the outward-facing imaging system 464 shown in FIG. 4 . Image acquired by the world cameras 316 may be processed by the pose processor 336 . For example, the pose processor 336 may implement one or more object recognizers 708 (e.g., shown in FIG. 7 ) to identify a pose of a user or another person in the user's environment or to identify a physical object in the user's environment.
With continued reference to FIG. 3 , a pair of scanned-laser shaped-wavefront (e.g., for depth) light projector modules with display mirrors and optics configured to project light 338 into the eyes
302 , 304 are shown. The depicted view also shows two miniature infrared cameras 324 paired with infrared light sources 326 (such as light emitting diodes âLEDâs), which are configured to be able to track the eyes
302 , 304 of the user to support rendering and user input. The cameras 324 may be part of the inward-facing imaging system 462 shown in FIG. 4 . The wearable system 200 may further feature a sensor assembly 339 , which may comprise X, Y, and Z axis accelerometer capability as well as a magnetic compass and X, Y, and Z axis gyro capability, preferably providing data at a relatively high frequency,
CLAIMS
Claims ( 19 )
What is claimed is:
1. A head-mounted display system comprising:
a head-mounted display configured to project light into the eyes of the user to display virtual image content;
an eye-tracking camera system configured to image a left eye of the user and a right eye of the user;
processing electronics comprising one or more processors; and
non-transitory computer-readable media storing instructions which, when executed by the processing electronics, cause the processing electronics to perform operations comprising:
calculating a left eye tracking confidence score indicative of a confidence level in a position of the left eye and a right eye tracking confidence score indicative of a confidence level in a position of the right eye;
determining whether the head-mounted display is properly registered to the eyes of the user; and
providing feedback to the user if the head-mounted display is not properly adjusted to fit the user,
wherein the left eye tracking confidence score is based on one or more of detecting a number of glints in the left eye, detecting that the user is blinking, detecting that the left eye is moving, and detecting that the pupil of the left eye is on a boundary of an eye tracking image, and
wherein the right eye tracking confidence score is based on one or more of detecting a number of glints in the right eye, detecting that the user is blinking, detecting that the right eye is moving, and detecting that the pupil of the right eye is on a boundary of an eye tracking image.
2. The head-mounted display system of claim 1 , wherein the head-mounted display comprises a stack of waveguides, wherein the waveguides are configured to output light to display the virtual image content, and wherein each waveguide comprises an incoupling optical element and an outcoupling optical element.
3. The head-mounted display system of claim 2 , wherein one of the waveguides outputs light with a different level of wavefront divergence than an other of the waveguides, and wherein different levels of wavefront divergence correspond to different depth planes.
4. The head-mounted display system of claim 1 , wherein the head-mounted display has a first field of view and a second field of view, wherein the second field of view is smaller than the first field of view, and wherein the operations further comprise:
changing from the first field of view to the second field of view when the position of at least one of the left eye or the right eye is outside the display registration volume.
5. The head-mounted display system of claim 1 , wherein the operations further comprise:
increasing brightness of a plurality of pixels of the head-mounted display in response to a determination that at least one of the left eye or the right eye is outside a display registration volume.
6. The head-mounted display system of claim 1 , wherein the eye-tracking camera system comprises a camera to image the left eye and an other camera to image the right eye.
7. The head-mounted display system of claim 1 , wherein, when the left eye and the right eye are vertically offset from each other by less than a first predetermined threshold, determining whether the head-mounted display is properly registered to the eyes of the user is evaluated based on a position of the left eye and a position of the right eye that is furthest from a desired vertical position.
8. The head-mounted display system of claim 7 , wherein, when the left eye and right eye are vertically offset from each other by less than a second predetermined threshold that is less than the first predetermined threshold, determining whether the head-mounted display is properly registered to the eyes of the user is evaluated based on an average position of the left eye and an average position of the right eye.
9. The head-mounted display system of claim 8 , wherein, when the left eye and right eye are vertically offset from each other by more than the first predetermined threshold, determining whether the head-mounted display is properly registered to the eyes of the user is evaluated based on the average position of the left eye and right eye.
10. The head-mounted display system of claim 1 , wherein, when one of the confidence scores is greater than the other, determining whether the head-mounted display is properly registered to the eyes of the user is evaluated based on an eye of the user associated with the greater confidence score while an eye of the user associated with the lower confidence score is not used in determining whether the head-mounted display is properly registered to the eyes of the user.
11. The head-mounted display system of claim 1 , further comprising:
a frame to which the head-mounted display is attached; and
an interchangeable fit piece for adjusting a fit of the frame, wherein the interchangeable fit piece is removably mounted to the frame.
12. The head-mounted display system of claim 11 , wherein the interchangeable fit piece comprises a forehead pad configured for adjusting the fit of the frame between the frame and a forehead of the user.
13. The head-mounted display system of claim 11 , wherein the interchangeable fit piece comprises an interchangeable back pad for adjusting the fit of the frame between the frame and a back of the head of the user.
14. The head-mounted display system of claim 1 , wherein providing feedback to the user if the head-mounted display is not properly adjusted to fit the user comprises: providing a suggestion to the user to change a currently-installed interchangeable fit piece.
15. A method for evaluating registration of virtual image content from a head-mounted display system by a left eye and a right eye of a user, the method comprising:
imaging the left eye and the right eye;
determining a left eye tracking confidence score indicative of a confidence level in a position of the left eye and a right eye tracking confidence score indicative of a confidence level in a position of the right eye;
determining whether imaged features of the left eye and the right eye are within a predetermined range of vertical positions relative to a head-mounted display of the head-mounted display system; and
providing a notification based on a determined position of the imaged features, wherein the notification indicates that the head-mounted display and at least one of the left eye or the right eye are not properly registered,
wherein the left eye tracking confidence score is based on one or more of detecting a number of glints in the left eye, detecting that the user is blinking, detecting that the left eye is moving, and detecting that the pupil of the left eye is on a boundary of an eye tracking image, and
wherein the right eye tracking confidence score is based on one or more of detecting a number of glints in the right eye, detecting that the user is blinking, detecting that the right eye is moving, and detecting that the pupil of the right eye is on a boundary of an eye tracking image.
16. The method of claim 15 , further comprising:
identifying a plurality of pixels of the head-mounted display that the user is expected to perceive as dimmed when a first position of at least one of the left eye or the right eye is outside a display registration volume; and
boosting brightness of the plurality of pixels of the display relative to other pixels in the display to mitigate the expected dimming.
17. The method of claim 15 , further comprising changing from a first field of view of the head-mounted display system to a second field of view of the head-mounted display system when the position of at least one of the left eye or the right eye is outside a display registration volume,
wherein the head-mounted display provides the first field of view when the position of the at least one of the left eye or the right eye is inside the display registration volume, wherein the head-mounted display provides the second field of view when the position of the at least one of the left eye or the right eye is outside the display registration volume, and wherein the second field of view is smaller than the first field of view.
18. The method of claim 15 , wherein the head-mounted display system comprises an interchangeable fit piece, and wherein providing the notification comprises indicating that the head-mounted display system is not properly fitted to the user and suggesting or instructing the user to replace a currently-installed interchangeable fit piece.
19. The method of claim 18 , wherein the at least one interchangeable fit piece comprises at least one fit piece selected from the group consisting of: a forehead pad, and a back pad that goes between the head-mounted display system and a back of a user's head.
US18/155,515
2018-07-24
2023-01-17
Display systems and methods for determining registration between display and eyes of user
Active
US11880043B2
( en )
Priority Applications (2)
Application Number
Priority Date
Filing Date
Title
US18/155,515
US11880043B2
( en )
2018-07-24
2023-01-17
Display systems and methods for determining registration between display and eyes of user
US18/525,487
US12578582B2
( en )
2018-07-24
2023-11-30
Display systems and methods for determining registration between display and eyes of user
Applications Claiming Priority (4)
Application Number
Priority Date
Filing Date
Title
US201862702866P
2018-07-24
2018-07-24
PCT/US2019/043096
WO2020023542A1
( en )
2018-07-24
2019-07-23
Display systems and methods for determining registration between a display and eyes of a user
US202017255315A
2020-12-22
2020-12-22
US18/155,515
US11880043B2
( en )
2018-07-24
2023-01-17
Display systems and methods for determining registration between display and eyes of user
Related Parent Applications (2)
Application Number
Title
Priority Date
Filing Date
PCT/US2019/043096
Continuation
WO2020023542A1
( en )
2018-07-24
2019-07-23
Display systems and methods for determining registration between a display and eyes of a user
US17/255,315
Continuation
US11567336B2
( en )
2018-07-24
2019-07-23
Display systems and methods for determining registration between display and eyes of user
Related Child Applications (1)
Application Number
Title
Priority Date
Filing Date
US18/525,487
Continuation
US12578582B2
( en )
2018-07-24
2023-11-30
Display systems and methods for determining registration between display and eyes of user
Publications (2)
Publication Number
Publication Date
US20230152597A1
US20230152597A1 ( en )
2023-05-18
US11880043B2
true
US11880043B2 ( en )
2024-01-23
Family
ID=69181904
Family Applications (3)
Application Number
Title
Priority Date
Filing Date
US17/255,315
Active
US11567336B2
( en )
2018-07-24
2019-07-23
Display systems and methods for determining registration between display and eyes of user
US18/155,515
Active
US11880043B2
( en )
2018-07-24
2023-01-17
Display systems and methods for determining registration between display and eyes of user
US18/525,487
Active
2039-11-01
US12578582B2
( en )
2018-07-24
2023-11-30
Display systems and methods for determining registration between display and eyes of user
Family Applications Before (1)
Application Number
Title
Priority Date
Filing Date
US17/255,315
Active
US11567336B2
( en )
2018-07-24
2019-07-23
Display systems and methods for determining registration between display and eyes of user
Family Applications After (1)
Application Number
Title
Priority Date
Filing Date
US18/525,487
Active
2039-11-01
US12578582B2
( en )
2018-07-24
2023-11-30
Display systems and methods for determining registration between display and eyes of user
Country Status (5)
Country
Link
US
( 3 )
US11567336B2
( en )
EP
( 2 )
EP3827426B1
( en )
JP
( 3 )
JP7382387B2
( en )
CN
( 1 )
CN112689869A
( en )
WO
( 1 )
WO2020023542A1
( en )
Families Citing this family (36)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
IL308285B2
( en )
2013-03-11
2024-11-01
Magic Leap Inc
System and method for augmented and virtual reality
KR102271198B1
( en )
2013-03-15
2021-06-29
ë§¤ì§ ë¦½, ì¸ì½í¬ë ì´í°ë
Display system and method
CA2901477C
( en )
2015-08-25
2023-07-18
Evolution Optiks Limited
Vision correction system, method and graphical user interface for implementation on electronic devices having a graphical display
KR102269065B1
( en )
2016-10-05
2021-06-24
ë§¤ì§ ë¦½, ì¸ì½í¬ë ì´í°ë
Periocular Testing for Mixed Reality Correction
KR102792495B1
( en )
2018-01-17
2025-04-04
ë§¤ì§ ë¦½, ì¸ì½í¬ë ì´í°ë
Determination of eye rotation center, depth plane selection, and render camera positioning in display systems
CA3088116A1
( en )
2018-01-17
2019-07-25
Magic Leap, Inc.
Display systems and methods for determining registration between a display and a user's eyes
US11093208B2
( en )
*
2018-02-16
2021-08-17
Valve Corporation
Using detected pupil location to align optical components of a head-mounted display
US11693239B2
( en )
*
2018-03-09
2023-07-04
Evolution Optiks Limited
Vision correction system and method, light field display and light field shaping layer and alignment therefor
US11353699B2
( en )
*
2018-03-09
2022-06-07
Evolution Optiks Limited
Vision correction system and method, light field display and light field shaping layer and alignment therefor
US11567336B2
( en )
2018-07-24
2023-01-31
Magic Leap, Inc.
Display systems and methods for determining registration between display and eyes of user
EP3830674A4
( en )
2018-08-03
2022-04-20
Magic Leap, Inc.
SELECTION OF DEPTH LEVELS FOR MULTI-LEVEL SCREEN DISPLAY SYSTEMS BY A USER CLASSIFICATION
US11966507B2
( en )
2018-10-22
2024-04-23
Evolution Optiks Limited
Light field vision testing device, adjusted pixel rendering method therefor, and vision testing system and method using same
US11327563B2
( en )
2018-10-22
2022-05-10
Evolution Optiks Limited
Light field vision-based testing device, adjusted pixel rendering method therefor, and online vision-based testing management system and method using same
US11500461B2
( en )
2019-11-01
2022-11-15
Evolution Optiks Limited
Light field vision-based testing device, system and method
US11902498B2
( en )
2019-08-26
2024-02-13
Evolution Optiks Limited
Binocular light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US11487361B1
( en )
2019-11-01
2022-11-01
Evolution Optiks Limited
Light field device and vision testing system using same
US12360592B2
( en )
2019-11-01
2025-07-15
Evolution Optiks Limited
Light field device and vision testing system using same
US11823598B2
( en )
2019-11-01
2023-11-21
Evolution Optiks Limited
Light field device, variable perception pixel rendering method therefor, and variable perception system and method using same
US12112665B2
( en )
2019-11-01
2024-10-08
Evolution Optiks Limited
Light field device, variable perception pixel rendering method therefor, and variable perception system and method using same
KR102862424B1
( en )
*
2019-11-22
2025-09-23
ìì§ì ì 주ìíì¬
Controlling of device based on user recognition
CN115053270B
( en )
2019-12-09
2026-01-09
å¥è·å ¬å¸
Systems and methods for operating a head-mounted display system based on user identity.
CN116033864A
( en )
*
2020-07-15
2023-04-28
å¥è·å ¬å¸
Eye tracking using a non-spherical cornea model
WO2022160014A1
( en )
*
2021-01-29
2022-08-04
ResMed Pty Ltd
A head mounted display system with positioning, stabilising and interfacing structures
US11982816B1
( en )
*
2021-03-23
2024-05-14
Apple Inc.
Wearable devices with adjustable fit
ES2988871T3
( en )
*
2021-06-30
2024-11-22
Tobii Ab
Eye tracking system
US20250147581A1
( en )
*
2021-09-22
2025-05-08
Apple Inc.
Fit guidance
CN115965672B
( en )
*
2021-10-09
2025-12-30
å京åèè·³å¨ç½ç»ææ¯æéå ¬å¸
Methods, devices, equipment and media for displaying three-dimensional objects
WO2023102500A1
( en )
*
2021-12-01
2023-06-08
Magic Leap, Inc.
Methods for controlling performance of extended reality display systems
CN114373216B
( en )
*
2021-12-07
2024-07-02
å¾æ¹(å京)å»çç§ææéå ¬å¸
Eye tracking method, device, equipment and storage medium for anterior segment OCTA
CN114356081B
( en )
*
2021-12-20
2025-03-21
æå°ç§ææéå ¬å¸
Image correction method, device, electronic device and head mounted display device
US20260079357A1
( en )
*
2022-03-24
2026-03-19
Apple Inc.
Adjustable head-mounted device
US12061735B2
( en )
*
2022-06-30
2024-08-13
Apple Inc.
Fit guidance for head-mountable devices
US20240090777A1
( en )
*
2022-09-20
2024-03-21
Apple Inc.
Facial interface having integrated health sensors
US20240121377A1
( en )
*
2022-10-06
2024-04-11
Blink Technologies Inc.
Multi-user gaze-tracking for personalized rendering from a 3d display
EP4685543A1
( en )
*
2024-07-23
2026-01-28
Essilor International
Eyewear equipment with assessment of an orientation of a wearer's eye
US12489881B1
( en )
*
2024-12-11
2025-12-02
Sony Group Corporation
Multi-user perspective projection in location-based entertainment
Citations (172)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US5280312A
( en )
1991-01-22
1994-01-18
Canon Kabushiki Kaisha
Visual axis detection apparatus
JPH09193803A
( en )
1996-01-19
1997-07-29
Furukawa Electric Co Ltd:The
Safety monitoring method near the platform
US5844656A
( en )
1996-11-07
1998-12-01
Xybernaut Corporation
Head mounted display with adjustment components
US6433760B1
( en )
1999-01-14
2002-08-13
University Of Central Florida
Head mounted display with eyetracking capability
US20020181115A1
( en )
2001-04-20
2002-12-05
John Hopkins University
Head mounted display with full field of view and high resolution
US20030030597A1
( en )
2001-08-13
2003-02-13
Geist Richard Edwin
Virtual display apparatus for mobile activities
US6611283B1
( en )
1997-11-21
2003-08-26
Canon Kabushiki Kaisha
Method and apparatus for inputting three-dimensional shape information
US20030184868A1
( en )
2001-05-07
2003-10-02
Geist Richard Edwin
Head-mounted virtual display apparatus with a near-eye light deflecting element in the peripheral field of view
JP2003307774A
( en )
2002-04-17
2003-10-31
Canon Inc
Optical equipment with gaze detection function
US6771424B1
( en )
1999-08-31
2004-08-03
Shimadzu Corporation
Head-mounted display device
US20040174496A1
( en )
2003-03-06
2004-09-09
Qiang Ji
Calibration-free gaze tracking under natural head movement
US6850221B1
( en )
1995-09-05
2005-02-01
Interlink Electronics, Inc.
Trigger operated electronic device
US20060028436A1
( en )
1992-03-05
2006-02-09
Armstrong Brad A
Image controller
US20060250322A1
( en )
2005-05-09
2006-11-09
Optics 1, Inc.
Dynamic vergence and focus control for head-mounted displays
US20070081123A1
( en )
2005-10-07
2007-04-12
Lewis Scott W
Digital eyewear
US20070120986A1
( en )
2005-11-08
2007-05-31
Takashi Nunomaki
Imaging device, information processing method, and computer program
US20070273983A1
( en )
2006-05-26
2007-11-29
Hebert Raymond T
Devices, methods, and systems for image viewing
US7312766B1
( en )
2000-09-22
2007-12-25
Canadian Space Agency
Method and system for time/motion compensation for head mounted displays
US20080002262A1
( en )
2006-06-29
2008-01-03
Anthony Chirieleison
Eye tracking head mounted display
US20080106489A1
( en )
2006-11-02
2008-05-08
Brown Lawrence G
Systems and methods for a head-mounted display
US20080117289A1
( en )
2004-08-06
2008-05-22
Schowengerdt Brian T
Variable Fixation Viewing Distance Scanned Light Displays
US7481531B2
( en )
2003-10-09
2009-01-27
Ipventure, Inc.
Eyeglasses with user monitoring
US20090160872A1
( en )
2007-12-20
2009-06-25
Raytheon Company
Imaging System
US20090167673A1
( en )
2007-12-26
2009-07-02
Kerofsky Louis J
Methods and Systems for Display Source Light Management with Variable Delay
US20100045932A1
( en )
2005-05-19
2010-02-25
The Johns Hopkins University
Wireless scleral search coil including systems for measuring eye movement and methods related thereto
US20100149073A1
( en )
2008-11-02
2010-06-17
David Chaum
Near to Eye Display System and Appliance
US20100283969A1
( en )
2007-10-23
2010-11-11
Mcgill University
Binocular vision assessment and/or therapy
US20110075257A1
( en )
2009-09-14
2011-03-31
The Arizona Board Of Regents On Behalf Of The University Of Arizona
3-Dimensional electro-optical see-through displays
US20120022220A1
( en )
2005-07-19
2012-01-26
Nektar Therapeutics
Method for Preparing Polymer Maleimides
US20120033179A1
( en )
2009-02-26
2012-02-09
Timo Kratzer
Method and apparatus for determining the location of the ocular pivot point
US20120068913A1
( en )
2010-09-21
2012-03-22
Avi Bar-Zeev
Opacity filter for see-through head mounted display
US20120127062A1
( en )
2010-11-18
2012-05-24
Avi Bar-Zeev
Automatic focus improvement for augmented reality displays
US20120139817A1
( en )
2009-08-13
2012-06-07
Bae Systems Plc
Head up display system
US20120162549A1
( en )
2010-12-24
2012-06-28
Chunyu Gao
Ergonomic Head Mounted Display Device And Optical System
US20120212484A1
( en )
2010-02-28
2012-08-23
Osterhout Group, Inc.
System and method for display content placement using distance and location information
US20120293407A1
( en )
2011-05-19
2012-11-22
Samsung Electronics Co. Ltd.
Head mounted display device and image display control method therefor
US20130022220A1
( en )
2011-07-20
2013-01-24
Google Inc.
Wearable Computing Device with Indirect Bone-Conduction Speaker
US20130038510A1
( en )
2011-08-09
2013-02-14
Google Inc.
Laser alignment of binocular head mounted display
US20130050833A1
( en )
2011-08-30
2013-02-28
John R. Lewis
Adjustment of a mixed reality display for inter-pupillary distance alignment
US20130050070A1
( en )
2011-08-29
2013-02-28
John R. Lewis
Gaze detection in a see-through, near-eye, mixed reality display
US20130083003A1
( en )
2011-09-30
2013-04-04
Kathryn Stone Perez
Personal audio/visual system
US20130083976A1
( en )
2011-10-03
2013-04-04
Qualcomm Incorporated
Image-based head position tracking method and system
US20130082922A1
( en )
2011-09-29
2013-04-04
Samuel A. Miller
Tactile glove for human-computer interaction
US20130093791A1
( en )
2011-10-13
2013-04-18
Microsoft Corporation
Touchscreen selection visual feedback
US20130117377A1
( en )
2011-10-28
2013-05-09
Samuel A. Miller
System and Method for Augmented and Virtual Reality
US20130114850A1
( en )
2011-11-07
2013-05-09
Eye-Com Corporation
Systems and methods for high-resolution gaze tracking
US20130114043A1
( en )
2011-11-04
2013-05-09
Alexandru O. Balan
See-through display brightness control
US20130125027A1
( en )
2011-05-06
2013-05-16
Magic Leap, Inc.
Massive simultaneous remote digital presence world
US20130120224A1
( en )
2011-11-11
2013-05-16
Elmer S. Cajigas
Recalibration of a flexible mixed reality device
US20130128364A1
( en )
2011-11-22
2013-05-23
Google Inc.
Method of Using Eye-Tracking to Center Image Content in a Display
WO2013077076A1
( en )
2011-11-24
2013-05-30
æ ªå¼ä¼ç¤¾ã¨ãã»ãã£ã»ãã£ã»ãã³ã¢
Expression output device and expression output method
US20130154906A1
( en )
2011-07-20
2013-06-20
Max Braun
Determining whether a wearable device is in use
US20130169683A1
( en )
2011-08-30
2013-07-04
Kathryn Stone Perez
Head mounted display with iris scan profiling
US20130208234A1
( en )
2005-10-07
2013-08-15
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
US20130235169A1
( en )
2011-06-16
2013-09-12
Panasonic Corporation
Head-mounted display and position gap adjustment method
US20130242262A1
( en )
2005-10-07
2013-09-19
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
JP2013195924A
( en )
2012-03-22
2013-09-30
Sony Corp
Head-mounted display
US20130285885A1
( en )
2012-04-25
2013-10-31
Andreas G. Nowatzyk
Head-mounted light-field display
US20130286053A1
( en )
2012-04-25
2013-10-31
Rod G. Fleck
Direct view augmented reality eyeglass-type display
US20130321925A1
( en )
2012-05-30
2013-12-05
Joel Jacobs
Customized head-mounted display device
US20130339433A1
( en )
2012-06-15
2013-12-19
Duke University
Method and apparatus for content rating using reaction sensing
US20140003762A1
( en )
2012-06-11
2014-01-02
Magic Leap, Inc.
Multiple depth plane three-dimensional display using a wave guide reflector array projector
US20140002442A1
( en )
2012-06-29
2014-01-02
Mathew J. Lamb
Mechanism to give holographic objects saliency in multiple spaces
US20140071539A1
( en )
2012-09-11
2014-03-13
Magic Leap, Inc.
Ergonomic head mounted display device and optical system
JP2014064248A
( en )
2012-09-24
2014-04-10
Ntt Docomo Inc
Image projection device and image projection method
US20140104143A1
( en )
2012-10-11
2014-04-17
Sony Computer Entertainment Europe Limited
Head mountable display
US20140146394A1
( en )
2012-11-28
2014-05-29
Nigel David Tout
Peripheral display for a near-eye display device
US20140177023A1
( en )
2012-04-05
2014-06-26
Augmented Vision Inc.
Apparatus for optical see-through head mounted display with mutual occlusion and opaqueness control capability
US20140232988A1
( en )
2011-10-22
2014-08-21
Alcon Pharmaceuticals Ltd.
Apparatus for Monitoring One or More Parameters of the Eye
US20140267420A1
( en )
2013-03-15
2014-09-18
Magic Leap, Inc.
Display system and method
US20140306866A1
( en )
2013-03-11
2014-10-16
Magic Leap, Inc.
System and method for augmented and virtual reality
US20140354514A1
( en )
2012-02-06
2014-12-04
Sony Corporation
Gaze tracking with projector
US20140361976A1
( en )
2013-06-07
2014-12-11
Sony Computer Entertainment Inc.
Switching mode of operation in a head mounted display
US20140375680A1
( en )
2013-06-24
2014-12-25
Nathan Ackerman