Conceptio › Archive › Google Patents

Google Patentsopen access

Display systems and methods for determining registration between a display and … — Magic Leap, Inc. (US12474575B2)

Magic Leap, Inc. · Google Patents

Google Patents · Patents · License: Open Access

Open Source ↗

lionelernestedwin magicleap

patent, google patents, intellectual property, US12474575B2, Magic Leap, Inc., Lionel Ernest Edwin, en, 2025

ABSTRACT

Abstract

A display system may include a head-mounted display (HMD) for rendering a three-dimensional virtual object which appears to be located in an ambient environment of a user of the display. One or more eyes of the user may not be in desired positions, relative to the HMD, to receive, or register, image information outputted by the HMD and/or to view an external environment. For example, the HMD-to-eye alignment may vary for different users and/or may change over time (e.g., as the HMD is displaced). The display system may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and/or may take actions to reduce or minimize effects of any misalignment.

Description

PRIORITY CLAIM

This application is a continuation of U.S. application Ser. No. 18/521,613, titled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND A USER'S EYESâ filed Nov. 28, 2023, which is a continuation of U.S. application Ser. No. 17/706,429, titled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND A USER'S EYESâ filed on Mar. 28, 2022 (now U.S. Pat. No. 11,880,033), which is a continuation of U.S. application Ser. No. 17/125,932, titled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND A USER'S EYESâ filed on Dec. 17, 2020 (now U.S. Pat. No. 11,290,706), which is a continuation of U.S. application Ser. No. 16/251,017, titled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND A USER'S EYESâ filed on Jan. 17, 2019 (now U.S. Pat. No. 10,917,634), which claims priority to: U.S. Provisional Application No. 62/644,321, titled âDISPLAY SYSTEMS AND METHODS FOR DETERMINING REGISTRATION BETWEEN A DISPLAY AND A USER'S EYESâ filed on Mar. 16, 2018; U.S. Provisional Application No. 62/618,559, titled âEYE CENTER OF ROTATION DETERMINATION, DEPTH PLANE SELECTION, AND RENDER CAMERA POSITIONING IN DISPLAY SYSTEMSâ filed on Jan. 17, 2018; and U.S. Provisional Application No. 62/702,849, titled âEYE CENTER OF ROTATION DETERMINATION, DEPTH PLANE SELECTION, AND RENDER CAMERA POSITIONING IN DISPLAY SYSTEMSâ filed on Jul. 24, 2018. The entire contents of each of the above-recited applications are hereby incorporated by reference into this application.

INCORPORATION BY REFERENCE

This application incorporates by reference the entirety of each of the following patent applications and publications: U.S. application Ser. No. 14/555,585 filed on Nov. 27, 2014, published on Jul. 23, 2015 as U.S. Publication No. 2015/0205126; U.S. application Ser. No. 14/690,401 filed on Apr. 18, 2015, published on Oct. 22, 2015 as U.S. Publication No. 2015/0302652; U.S. application Ser. No. 14/212,961 filed on Mar. 14, 2014, now U.S. Pat. No. 9,417,452 issued on Aug. 16, 2016; U.S. application Ser. No. 14/331,218 filed on Jul. 14, 2014, published on Oct. 29, 2015 as U.S. Publication No. 2015/0309263; U.S. Patent Publication No. 2016/0270656; U.S. Patent Publication No. 2015/0178939, published Jun. 25, 2015; U.S. Patent Publication No. 2015/0016777; U.S. patent application Ser. No. 15/274,823; U.S. patent application Ser. No. 15/296,869; U.S. patent application Ser. No. 15/717,747, filed Sep. 27, 2017; U.S. patent application Ser. No. 15/497,726, filed Apr. 26, 2017; U.S. Patent Publication No. 2017/0053165, published Feb. 23, 2017; U.S. Patent Publication No. 2017/0053166, published Feb. 23, 2017; U.S. application Ser. No. 15/341,760, filed on Nov. 2, 2016, published on May 4, 2017 as U.S. Publication No. 2017/0122725; U.S. application Ser. No. 15/341,822, filed on Nov. 2, 2016, published on May 4, 2017 as U.S. Publication No. 2017/0124928; U.S. Provisional Patent Application No. 62/618,559, filed Jan. 17, 2018; and U.S. Provisional Patent Application No. 62/642,761, filed Mar. 14, 2018.

FIELD

The present disclosure relates to display systems, including virtual reality and augmented reality display systems, and, more particularly, to systems and methods for evaluating fit of a display on a user.

BACKGROUND

Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ, âaugmented realityâ, or âmixed realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or âVRâ, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or âARâ, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; a mixed reality, or âMRâ, related to merging real and virtual worlds to produce new environments where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR and MR technology.

SUMMARY

Various examples of registration observation and response in a mixed reality system are disclosed.

In some embodiments, a display system is provided for projecting light to an eye of a user to display virtual image content. The display system comprises a frame configured to be supported on a head of the user, a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time, one or more eye-tracking cameras configured to image the user's eye, and processing electronics in communication with the display and the one or more eye-tracking cameras. The processing electronics are configured to determine a position of the eye based on images of the eye obtained with the one or more eye-tracking cameras, determine whether the position of the eye is within a display registration volume of the head-mounted display system, and provide a notification based on determining whether the position of the eye is within the display registration volume, where the notification indicates at least that the display and the eye are not properly registered.

In some other embodiments, a display system is configured to project light to an eye of a user to display virtual image content. The display system comprises a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras. The processing electronics are configured to: determine a position of the eye based on images of the eye obtained with the one or more eye-tracking cameras; determine whether the position of the eye is more than a first threshold distance outside of a viewing volume of the head-mounted display system; and in response to a determination that the position of the eye is more than the first threshold distance outside of the viewing volume of the head-mounted display system, provide feedback to the user indicating that the display and the eye are not properly registered for output.

In some other embodiments, a display system is provided for projecting light to an eye of a user to display virtual image content. The display system comprises a frame configured to be supported on a head of the user, a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time, one or more eye-tracking cameras configured to image the user's eye, and processing electronics in communication with the display and the one or more eye-tracking cameras. The processing electronics are configured to determine whether the light projected by head-mounted display is properly registered by the eye of the user and provide feedback to the user if the head-mounted display is not properly adjusted to fit the user to register the light projected by the display system.

In yet other embodiments, a method is provided for evaluating registration of virtual image content from a head-mounted display system by a user's eye. The method comprises determining a first position of the eye, determining whether the first position of the eye is within a display registration volume of the head-mounted display system, where the display registration volume is an imaginary volume associated with proper fit of the head-mounted display system relative to the user's eye, and providing a notification based on determining whether the position of the eye is within the display registration volume, where the notification indicates at least that the display and the eye are not properly registered.

Additional examples of embodiments are enumerated below.

PRIORITY CLAIM

INCORPORATION BY REFERENCE

FIELD

BACKGROUND

SUMMARY

Various examples of registration observation and response in a mixed reality system are disclosed.

Additional examples of embodiments are enumerated below.

Example 1. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to: determine a position of the eye based on images of the eye obtained with the one or more eye-tracking cameras; determine whether the position of the eye is within a display registration volume of the head-mounted display system; and provide a notification based on determining whether the position of the eye is within the display registration volume, where the notification indicates at least that the display and the eye are not properly registered. Example 2. The display system of Example 1, wherein the processing electronics are further configured to, upon determining that the position of the eye is outside of the display registration volume, provide feedback to the user that the head-mounted display is not properly adjusted to fit the user, wherein the feedback is the notification provided based on determining whether the position of the eyes within the display registration volume. Example 3. The display system of Example 1, further comprising at least one interchangeable fit piece removably mounted to the frame and configured to adjust a fit of the frame. Example 4. The display system of Example 3, wherein the interchangeable fit piece comprises an interchangeable nose bridge configured to adjust the fit of the frame between the frame and a nose bridge of the user. Example 5. The display system of Example 3 or 4, wherein the interchangeable fit piece comprises an interchangeable forehead pad configured to adjust the fit of the frame between the frame and a forehead of the user. Example 6. The display system of any of Examples 3 to 5, wherein the interchangeable fit piece comprises an interchangeable back pad configured to adjust the fit of the frame between the frame and a back of the head of the user. Example 7. The display system of any of Examples 2 to 6, wherein the processing electronics is further configured such that providing the notification comprises providing feedback to the user that the head-mounted display is not properly adjusted to fit the user comprises providing a suggestion to the user to swap out a currently-installed interchangeable fit piece for another interchangeable fit piece. Example 8. The display system of any of Examples 1 to 7, further comprising one or more light sources disposed on the frame with respect to the user's eye to illuminate the user's eye, the one or more eye-tracking cameras forming images of the eye using the light from the one or more light sources. Example 9. The display system of Example 8, wherein the one or more light sources comprises at least two light sources disposed on the frame with respect to the user's eye to illuminate the user's eye. Example 10. The display system of any of Examples 8 to 9, wherein the one or more light sources comprises infrared light emitters. Example 11. The display system of any of Examples 8 to 10, wherein one or more light sources form one or more glints on the eye and the processing electronics is configured to determine a location of the cornea based on the one or more glints. Example 12. The display system of any of Examples 1 to 11, wherein the position of the eye it is a location of a center of rotation of the eye. Example 13. The display system of any of Examples 1 to 11, wherein the cornea has associated therewith a cornea sphere having a center of curvature and the processing electronics is configured to determine a location of the center of curvature of the cornea sphere. Example 14. The display system of Example 1, wherein the processing electronics is configured to provide the notification by providing instructions causing the display to boost a brightness of a plurality of pixels of the display relative to other pixels of the display, wherein the plurality of pixels with a boosted brightness comprise pixels expected to undergo perceived dimming under improper registration. Example 15. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to: determine a position of the eye based on images of the eye obtained with the one or more eye-tracking cameras; determine whether the position of the eye is more than a first threshold distance outside of a viewing volume of the head-mounted display system; and in response to a determination that the position of the eye is more than the first threshold distance outside of the viewing volume of the head-mounted display system, provide feedback to the user indicating that the display and the eye are not properly registered for output. Example 16. The display system of Example 15, wherein the processing electronics are configured to determine whether the position of the eye is more than the first threshold distance outside of a viewing volume by at least: determining whether the position of the eye is less than a second threshold distance from the eyepiece; and in response to a determination that the position of the eye is less than the second threshold distance from the head-mounted display system, providing feedback to the user indicating that the display and the eye are not properly registered for output. Example 17. The display system of Example 15, wherein the processing electronics are configured to determine whether the position of the eye is more than the first threshold distance outside of a viewing volume by at least: determining whether the position of the eye is more than a second threshold distance from the eyepiece; and in response to a determination that the position of the eye is more than the second threshold distance from the head-mounted display system, providing feedback to the user indicating that the display and the eye are not properly registered for output. Example 18. The display system of Example 15, wherein the processing electronics are configured to determine whether the position of the eye is more than the first threshold distance outside of a viewing volume by at least: determining whether the position of the eye is more than a second threshold distance outside of a subspace of a field of view of the eye tracking camera; and in response to a determination that the position of the eye is more than the second threshold distance outside of the subspace of the viewing volume of the eye tracking camera, providing feedback to the user indicating that the display and the eye are not properly registered for output. Example 19. The display system of Example 15 wherein the viewing volume of the head-mounted display is a volume through which light representing every pixel of virtual image content presented by the head-mounted display is expected to pass. Example 20. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to: determine whether the light projected by head-mounted display is properly registered by the eye of the user; and provide feedback to the user if the head-mounted display is not properly adjusted to fit the user to register the light projected by the display system. Example 21. The display system of Example 20, further comprising at least one interchangeable fit piece removably mounted to the frame and configured to adjust a fit of the frame. Example 22. The display system of Example 21, wherein the interchangeable fit piece comprises an interchangeable nose bridge configured to adjust the fit of the frame between the frame and a nose bridge of the user. Example 23. The display system of Example 20 or 22, wherein the interchangeable fit piece comprises an interchangeable forehead pad configured to adjust the fit of the frame between the frame and a forehead of the user. Example 24. The display system of any of Examples 20 to 23, wherein the interchangeable fit piece comprises an interchangeable back pad configured to adjust the fit of the frame between the frame and a back of the head of the user. Example 25. The display system of any of Examples 20 to 24, wherein the processing electronics is further configured such that providing feedback to the user that the head-mounted display is not properly adjusted to fit the user comprises providing a suggestion to the user to swap out a currently-installed interchangeable fit piece for another interchangeable fit piece. Example 26. A method for evaluating registration of virtual image content from a head-mounted display system by a user's eye, the method comprising: determining a first position of the eye; determining whether the first position of the eye is within a display registration volume of the head-mounted display system, wherein the display registration volume is an imaginary volume associated with proper fit of the head-mounted display system relative to the user's eye; and providing a notification based on determining whether the position of the eye is within the display registration volume, where the notification indicates at least that the display and the eye are not properly registered. Example 27. The method of Example 26, wherein the head-mounted display system comprises an eye-tracking camera, wherein determining the first position of the eye comprises utilizing the eye-tracking camera to image the eye of the user. Example 28. The method of Example 27, wherein the first position of the eye is a position of the center of rotation of the eye, and further comprising calculating a center of rotation of the eye based upon imaging of the eye by the eye-tracking camera. Example 29. The method of Example 26, wherein the head-mounted display system is configured to project light into the eye to display virtual image content in the field of view of the user, and further comprising displaying an indication that the wearable system is properly fitted. Example 30. The method of any of Examples 26 to 29, further comprising automatically tracking the center of rotation of the eye over time with the head-mounted display system and notify the user when the center of rotation of the eye moves outside of the registration display volume. Example 31. The method of Examples 26 or 29, further comprising: determining a second position of the eye; determining that the second position of the eye is within the display registration volume; and in response to determining that the second position of the eye is within the display registration volume, providing additional feedback to the user indicating that the wearable system is properly fitted to the user. Example 32. The method of any of Examples 26 to 31, wherein, when the eye of the user is not within the display registration volume, at least some pixels of the head-mounted display system are dimmed or invisible to the user. Example 33. The method of any of Examples 26 to 32, further comprising changing a field of view of the head-mounted display system when the position of the eye is outside the display registration volume, wherein the head-mounted display system comprises at least one display having a first field of field when the position of the eye is inside the display registration volume, wherein the display has a second field of view when the position of the eye is outside the display registration volume, and wherein the second field of view is smaller than the first field of view. Example 34. The method of Example 33, wherein providing the notification comprises providing feedback to the user within the second field of view. Example 35. The method of any of Examples 26 to 34, wherein the wearable system comprises at least one interchangeable fit piece, and further comprising: providing a notification to the user indicating that the wearable system is not properly fitted to the user, wherein the notification comprises a suggestion or an instruction to the user to replace a currently-installed interchangeable fit piece with an alternative interchangeable fit piece. Example 36. The method of Example 35, wherein the interchangeable fit piece comprises at least one fit piece selected from the group consisting of: a nose bridge pad, a forehead pad, and a back pad that goes between the wearable system and a back of a user's head. Example 37. The method of Example 36, wherein the wearable system comprises at least one interchangeable nose bridge pad, further comprising determining that a display of the head-mounted system is too low with respect to the eye, and wherein providing the notification to the user comprises prompting the user to install a larger nose bridge pad. Example 38. The method of any of Examples 26 to 37, further comprising: identifying a plurality of pixels of a display of the head-mounted display system that the user is expected to perceive as dimmed as a result of the first position of the eye being outside the display registration volume; and boosting brightness of the plurality of pixels of the display relative to other pixels in the display to mitigate the expected dimming. Example 39. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content as rendered by a virtual render camera with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to: determine a distance away from the display at which the eye is located based on images of the eye obtained with the one or more eye-tracking cameras, and adjust a focal length of the virtual render camera based on the determined distance. Example 40. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to: determine a position of the eye relative to the display based on images of the eye obtained with the one or more eye-tracking cameras; determine an amount of pixels of virtual image content that the user is expected to perceive as dimmed based on the position of the eye relative to the display; and control operation of the display based on the determined amount of pixels. Example 41. The display system of Example 40, wherein the processing electronics are configured to determine an amount of pixels of virtual image content that the user is not expected to perceive as dimmed based on the position of the eye relative to the display. Example 42. The display system of Examples 40 or 41, wherein the processing electronics are configured to control operation of the display by: boosting a brightness of the pixels of virtual image content that the user is expected to perceive as dimmed based on the position of the eye relative to the display. Example 43. The display system of any of the Examples 40-42, wherein the amount of pixels of virtual image content comprises a percentage of pixels. Example 44. The display system of any of the Examples 40-43, wherein the processing electronics are configured to compare the amount of pixels of virtual image content with one or more thresholds and, in response to a determination that the amount of pixels of virtual image content exceeds one or more thresholds, provide feedback to the user indicating that the display and the eye are not properly registered for output. Example 45. A display system configured to project light to an eye of a user to display virtual image content, the display system comprising: a frame configured to be supported on a head of the user; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content with different amounts of wavefront divergence to present virtual image content appearing to be located at different depths at different periods of time; one or more eye-tracking cameras configured to image the user's eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to: define a registration volume relative to the display based on one or more parameters; determine a position of the eye based on images of the eye obtained with the one or more eye-tracking cameras; determine whether the position of the eye is within the registration volume of the head-mounted display system; and control operation of the display based on determining whether the position of the eye is within the display registration volume. Example 46. The display system of Example 45, wherein the one or more parameters comprise a type of application that is running on the display system. Example 47. The display system of any of Examples 45-46, wherein the one or more parameters comprise one or more physical parameters of the head-mounted display. Example 48. The display system of any of Examples 45-47, wherein the one or more physical parameters of the head-mounted display comprise one or more of a display field of view, a display surface size, a shape of the display, an outer housing of the display, an amount of optical power imparted by the display to light representing virtual image content. Example 49. The display system of any of Examples 45-48, wherein the processing electronics are configured to control operation of the display by presenting virtual image content to the user indicating at least that the display and the eye are not properly registered. Example 50. The display system of Example 15, wherein the processing electronics are configured to determine whether the position of the eye is more than the first threshold distance outside of the viewing volume by at least: determining whether the position of the eye is more than a second threshold distance outside of a subspace of the viewing volume of an outer housing of the head-mounted display; and in response to a determination that the position of the eye is more than the second threshold distance outside of the subspace of the viewing volume of the outer housing of the head-mounted display, providing feedback to the user indicating that the display and the eye are not properly registered for output. Example 52. The display system of Example 15, wherein the processing electronics are further configured to: identify an application running on the display system; and determine the first threshold distance based on the identified application.

Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person.

FIG. 2 schematically illustrates an example of a wearable system.

FIG. 3 schematically illustrates example components of a wearable system.

FIG. 4 schematically illustrates an example of a waveguide stack of a wearable device for outputting image information to a user.

FIG. 5 schematically illustrates an example of an eye.

FIG. 6 is a schematic diagram of a wearable system that includes an eye tracking system.

FIG. 7 A is a block diagram of a wearable system that may include an eye tracking system.

FIG. 7 B is a block diagram of a render controller in a wearable system.

FIG. 7 C is a block diagram of a registration observer in a head-mounted display system.

FIG. 8 A is a schematic diagram of an eye showing the eye's corneal sphere.

FIG. 8 B illustrates an example corneal glint detected by an eye-tracking camera.

FIGS. 8 C- 8 E illustrate example stages of locating a user's corneal center with an eye tracking module in a wearable system.

FIGS. 9 A- 9 C illustrate an example normalization of the coordinate system of eye tracking images.

FIGS. 9 D- 9 G illustrate example stages of locating a user's pupil center with an eye tracking module in a wearable system.

FIG. 10 illustrates an example of an eye including the eye's optical and visual axes and the eye's center of rotation.

FIG. 11 is a process flow diagram of an example of a method for using eye tracking in rendering content and providing feedback on registration in a wearable device.

FIGS. 12 A and 12 B illustrate a nominal position of a display element relative to a user's eye and illustrate a coordinate system for describing the positions of the display element and the user's eye relative to one another.

FIGS. 13 A and 13 B illustrate nominal positioning and positioning tolerances of a display element relative to a user's eye in a head-mounted display system.

FIGS. 13 C and 13 D illustrate a display registration volume and a user's eye viewing content from a display.

FIG. 14 illustrates an example of the perceived dimming of a display for various positions of a user's eye relative to the display.

FIGS. 15 A and 15 B are exploded perspective views of a head-mounted display system having interchangeable pieces such as back pads, forehead pads, and nose bridge pads to adjust fit of a head-mounted display of the display system for different users.

FIG. 16 is a process flow diagram of an example of a method for observing registration and providing feedback on registration with a head-mounted display system.

FIGS. 17 A- 17 H illustrate views of light fields projected by a display and how the intersections of the light fields may partly define a display registration volume.

FIG. 18 illustrates a top-down view of light fields projected by a display and how the intersections of the light fields may partly define a display registration volume.

FIG. 19 A illustrates a registration volume derived from a display housing of a head-mounted display system.

FIG. 19 B illustrates superimposed registration volumes of a display housing and a display of a head-mounted display system.

FIG. 19 C illustrates an aggregate registration volume derived from the superimposed registration volumes of FIG. 19 B .

Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.

DETAILED DESCRIPTION

The display portion of a display system may include a head-mounted display (HMD) which may display a three-dimensional (3D) virtual object such that the object appears to be located within the user's ambient environment. As a result, the 3D virtual object may be perceived by the user in a similar manner as real world objects.

The HMD may display images by outputting spatially modulated light to the user, with the light corresponding to the virtual object. The spatially modulated light may contain image information may be referred to as image light. To be perceived by the user, the image light travels from the HMD to an eye of the user, propagates through the pupil, and impinges on the eye's retina. It will be appreciated that if all or a portion of the image light for an image does not enter the pupil of the eye and/or does not impinge on the eye's retina, then the viewer would not see the image or the quality of the image may be degraded. As used herein, registration relates to the relative positioning of the display and the user's eyes. For example, a display may be said to be properly registered when the user's eyes and the display are positioned relative to one another for a desired amount of image light to enter the eye. A registration observer (e.g., a computer program) in the display device may be programmed to monitor whether the display is properly registered or positioned for the eye to receive the image light from the display.

In order to properly display content to users, e.g., by having the user's eyes positioned to receive image light, the user's eyes may need to be situated within a particular region or volume of space relative to the HMD. This volume may be referred to as the display registration volume. If the user's eyes are outside the display registration volume, display quality may be degraded (e.g., there may be dimming and/or displayed content that does not reach the users eyes). Various factors may combine to determine the positions of the user's eyes relative to the HMD and thus whether the user's eyes are situated within the desired display registration volume. As an example, anatomical variations between users may mean that the head-mounted display fits some users in a manner that places their eyes outside the display registration volume. As another example, the HMD may not be rigidly affixed to a user's head and may shift on the user's head over time, particularly when the user is moving around. As particular examples, the HMD may slip down the user's nose or tilt relative to a line (the interocular axis) between the user's eyes and, as a result, the HMD may not be able to provide desired virtual content (e.g., without some undesirable degradation) due to the shift of the display relative to the user's eyes.

Various systems and techniques described herein are at least in part directed to solving problems related to proper registration of a display to allow the viewer to view image content as desired. In some embodiments, a head-mounted display system may be configured to determine the position of an eye of the user. The display system may then determine whether the position of that eye is within a display registration volume of the head-mounted display system. Determining the position of the eye may include determining the position of a representative pointer volume associated with the eye e.g., the center of rotation of the eye. Determining whether the position of the eye is within the display registration volume may include determining whether the center of rotation of the eye is within the display registration volume. As discussed herein, the center of rotation of the eye may be determined using an inward-facing imaging system configured to image the eye. In addition, in some embodiments, the display registration volume is an imaginary volume associated with proper fit of the head-mounted display system relative to the user's eye. For example, the display registration volume may be a volume defined by a projection from the surface of the head-mounted display system outputting image light. More specifically, the display registration volume may be a three-dimensional geometric shape that tapers from a base towards an apex. The shape of the display registration volume's base may be defined at least in part by the geometry of the display, and the depth of the display registration volume (i.e., the distance from base to apex on the z-axis) may be at least in part defined by the field of view (FOV) of the display. For example, a round or circular display (e.g., the shape of the area on a surface from which image light is outputted to the viewer) may yield a conical display registration volume, and a polygonal display may yield a pyramidal display registration volume. As an additional example, a display with a larger FOV may yield a display registration volume having a smaller depth than a display with a smaller FOV. In some embodiments, the display registration volume may have the general shape of a truncated cone or pyramid. For example, the display registration volume may have the general shape of a frustum, e.g., a frustum of a pyramid such as a rectangular pyramid.

In some embodiments, an inward-facing imaging system of the head-mounted display system may acquire images of the user's face, including their eyes. The inward-facing imaging system may be an eye-tracking system, which may be mounted on a frame of the head-mounted display. The head-mounted display system may analyze the images to determine the relative position of the user's eyes and the HMD, and whether the position of each of the user's eyes falls within the display registration volume for that eye. Based on this information, the head-mounted display system may notify the user to adjust the fit of the HMD. For example, the notification may inform the user that the device has slipped and needs adjustment or a suggestion to make an adjustment of the HMD. In at least some embodiments, the head-mounted display system may take steps to mitigate any display degradation caused by misalignment of the HMD to the user, such as by boosting brightness or light output to the user in areas that would otherwise be dimmed by misalignment or by moving virtual content. Accordingly, such embodiments of the HMD may assist users with properly fitting the HMD and mitigating issues caused by improper fit of the HMD, such as when the HMD slips, moves, or tilts relative to the user's head. In some embodiments, it will be appreciated that the display system may be configured to notify the user of misalignment and to also take steps to mitigate display degradation caused by the misalignment. In some other embodiments, the display system may not provide a notification to the user; rather, the notification may simply be instructions or flags within the display system which trigger the display system to conduct actions to mitigate image degradation caused by misalignment.

Advantageously, the analysis of registration may be performed automatically utilizing images acquired from the inward-facing imaging system and information regarding the display registration volume stored or accessible by the display system. As a result, the fit of the HMD may be corrected upon first using the HMD, and optionally also during the course of continued usage of the HMD to ensure a high level of image quality in the use of the head-mounted display system.

Accordingly, a variety of implementations of systems and methods for observing registration of a head-mounted display system and taking action in response to the observed registration are provided herein.

Examples of 3D Display of a Wearable System

Reference will now be made to the drawings, in which like reference numerals refer to like parts throughout. Unless indicated otherwise, the drawings are schematic and not necessarily drawn to scale.

A wearable system (also referred to herein as a head-mounted display system or as an augmented reality (AR) system) may be configured to present 2D or 3D virtual images to a user. The images may be still images, frames of a video, or a video, in combination or the like. At least a portion of the wearable system may be implemented on a wearable device that may present a VR, AR, or MR environment, alone or in combination, for user interaction. The wearable device may be used interchangeably as an AR device (ARD). Further, for the purpose of the present disclosure, the term âARâ is used interchangeably with the term âMRâ.

FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person. In FIG. 1 , an MR scene 100 is depicted wherein a user of an MR technology sees a real-world park-like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120 . In addition to these items, the user of the MR technology also perceives that he âseesâ a robot statue 130 standing upon the real-world platform 120 , and a cartoon-like avatar character 140 flying by which seems to be a personification of a bumble bee, even though these elements do not exist in the real world.

In order for the 3D display to produce a true sensation of depth, and more specifically, a simulated sensation of surface depth, it may be desirable for each point in the display's visual field to generate an accommodative response corresponding to its virtual depth. If the accommodative response to a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues of convergence and stereopsis, the human eye may experience an accommodation conflict, resulting in unstable imaging, harmful eye strain, headaches, and, in the absence of accommodation information, almost a complete lack of surface depth.

VR, AR, and MR experiences may be provided by display systems having displays in which images corresponding to a plurality of depth planes are provided to a viewer. The images may be different for each depth plane (e.g., provide slightly different presentations of a scene or object) and may be separately focused by the viewer's eyes, thereby helping to provide the user with depth cues based on the accommodation of the eye required to bring into focus different image features for the scene located on different depth plane or based on observing different image features on different depth planes being out of focus. As discussed elsewhere herein, such depth cues provide credible perceptions of depth.

FIG. 2 illustrates an example of wearable system 200 which may be configured to provide an AR/VR/MR scene. The wearable system 200 may also be referred to as the AR system 200 . The wearable system 200 includes a display 220 , and various mechanical and electronic modules and systems to support the functioning of display 220 . The display 220 may be coupled to a frame 230 , which is wearable by a user, wearer, or viewer 210 . The display 220 may be positioned in front of the eyes of the user 210 . The display 220 may present AR/VR/MR content to a user. Because the display 220 may be configured to be worn on the head of the user 210 , it may also be referred to as a head-mounted display (HMD) and the wearable system 200 , comprising the display 220 , may also be referred to as a head-mounted display system.

In some embodiments, a speaker 240 is coupled to the frame 230 and positioned adjacent the ear canal of the user (in some embodiments, another speaker, not shown, is positioned adjacent the other ear canal of the user to provide for stereo/shapeable sound control). The display 220 may include an audio sensor (e.g., a microphone) 232 for detecting an audio stream from the environment and capture ambient sound. In some embodiments, one or more other audio sensors, not shown, are positioned to provide stereo sound reception. Stereo sound reception may be used to determine the location of a sound source. The wearable system 200 may perform voice or speech recognition on the audio stream.

The wearable system 200 may include an outward-facing imaging system 464 (shown in FIG. 4 ) which observes the world in the environment around the user. The wearable system 200 may also include an inward-facing imaging system 462 (shown in FIG. 4 ) which may track the eye movements of the user. The inward-facing imaging system may track either one eye's movements or both eyes' movements. The inward-facing imaging system 462 may be attached to the frame 230 and may be in electrical communication with the processing modules 260 or 270 , which may process image information acquired by the inward-facing imaging system to determine, e.g., the pupil diameters or orientations of the eyes, eye movements or eye pose of the user 210 . The inward-facing imaging system 462 may include one or more cameras. For example, at least one camera may be used to image each eye. The images acquired by the cameras may be used to determine pupil size or eye pose for each eye separately, thereby allowing presentation of image information to each eye to be dynamically tailored to that eye.

As an example, the wearable system 200 may use the outward-facing imaging system 464 or the inward-facing imaging system 462 to acquire images of a pose of the user. The images may be still images, frames of a video, or a video.

The display 220 may be operatively coupled 250 , such as by a wired lead or wireless connectivity, to a local data processing module 260 which may be mounted in a variety of configurations, such as fixedly attached to the frame 230 , fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise removably attached to the user 210 (e.g., in a backpack-style configuration, in a belt-coupling style configuration).

The local processing and data module 260 may comprise a hardware processor, as well as digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to assist in the processing, caching, and storage of data. The data may include data a) captured from sensors (which may be, e.g., operatively coupled to the frame 230 or otherwise attached to the user 210 ), such as image capture devices (e.g., cameras in the inward-facing imaging system or the outward-facing imaging system), audio sensors (e.g., microphones), inertial measurement units (IMUs), accelerometers, compasses, global positioning system (GPS) units, radio devices, or gyroscopes; or b) acquired or processed using remote processing module 270 or remote data repository 280 , possibly for passage to the display 220 after such processing or retrieval. The local processing and data module 260 may be operatively coupled by communication links 262 or 264 , such as via wired or wireless communication links, to the remote processing module 270 or remote data repository 280 such that these remote modules are available as resources to the local processing and data module 260 . In addition, remote processing module 280 and remote data repository 280 may be operatively coupled to each other.

In some embodiments, the remote processing module 270 may comprise one or more processors configured to analyze and process data or image information. In some embodiments, the remote data repository 280 may comprise a digital data storage facility, which may be available through the internet or other networking configuration in a âcloudâ resource configuration. In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing fully autonomous use from a remote module.

Example Components of a Wearable System

FIG. 3 schematically illustrates example components of a wearable system. FIG. 3 shows a wearable system 200 which may include a display 220 and a frame 230 . A blown-up view 202 schematically illustrates various components of the wearable system 200 . In certain implements, one or more of the components illustrated in FIG. 3 may be part of the display 220 . The various components alone or in combination may collect a variety of data (such as e.g., audio or visual data) associated with the user of the wearable system 200 or the user's environment. It should be appreciated that other embodiments may have additional or fewer components depending on the application for which the wearable system is used. Nevertheless, FIG. 3 provides a basic idea of some of the various components and types of data that may be collected, analyzed, and stored through the wearable system.

FIG. 3 shows an example wearable system 200 which may include the display 220 . The display 220 may comprise a display lens 226 that may be mounted to a user's head or a housing or frame 230 , which corresponds to the frame 230 . The display lens 226 may comprise one or more transparent mirrors positioned by the housing 230 in front of the user's eyes 302 , 304 and may be configured to bounce projected light 338 into the eyes 302 , 304 and facilitate beam shaping, while also allowing for transmission of at least some light from t

CLAIMS

Claims ( 17 )

What is claimed is:

1 . A display system configured to project light to an eye of a user to display virtual image content, the display system comprising:

a frame configured to be supported on a head of the user; a display disposed on the frame, the display configured to project light into the eye to display the virtual image content with different amounts of wavefront divergence to present the virtual image content appearing to be located at different depths while an external environment is viewable by the eye; one or more eye-tracking cameras configured to image the eye; and processing electronics in communication with the display and the one or more eye-tracking cameras, the processing electronics configured to:

determine a display registration volume relative to the display such that the virtual image content is received by the eye while the eye is situated within the display registration volume;

determine a housing registration volume relative to the frame such that an external field of view of the external environment proximal to the display is viewable by the eye while the eye is situated within the housing registration volume;

determine a combined registration volume that includes a plurality of points that are included in both the display registration volume and the housing registration volume;

determine a three-dimensional (3D) position of the eye based on one or more images of the eye obtained with the one or more eye-tracking cameras;

determine whether the position of the eye is within the combined registration volume; and

control operation of the display based on determining whether the position of the eye is within the combined registration volume, wherein controlling the operation of the display includes providing, to the user, a notification of misregistration based at least partly on determining that the eye is not within the combined registration volume.

2 . The display system of claim 1 , wherein the combined registration volume includes all of the points that are included in both the display registration volume and the housing registration volume.

3 . The display system of claim 1 , wherein the housing registration volume is defined relative to one or more housing openings in the frame.

4 . The display system of claim 3 , wherein the housing registration volume has a cone-like shape with a base defined by the shape and size of the one or more housing openings.

5 . The display system of claim 1 , wherein the position of the eye is a center of rotation of the eye.

6 . The display system of claim 1 , wherein the display registration volume has a frustum shape.

7 . The display system of claim 1 , wherein the notification of misregistration is provided to the user based on determining that the position of the eye is more than a threshold distance outside of the combined registration volume.

8 . The display system of claim 1 , wherein the display registration volume is defined based at least partly on a type of application that is running on the display system.

9 . Non-transitory computer-readable storage media storing instructions which, when executed by at least one processor, instruct the at least one processor to perform operations for controlling a wearable display configured to project light to an eye of a user to display virtual image content, the operations comprising:

determining a display registration volume relative to the display such that the virtual image content is received by the eye while the eye is situated within the display registration volume; determining a housing registration volume relative to the frame such that an external field of view of an external environment proximal to the display is viewable by the eye while the eye is situated within the housing registration volume; determining a combined registration volume that includes a plurality of points that are included in both the display registration volume and the housing registration volume; determining a three-dimensional (3D) position of the eye based on one or more images of the eye obtained with one or more eye-tracking cameras configured to image the eye while the display is being worn by the user; determining whether the position of the eye is within the combined registration volume; and controlling operation of the display based on determining whether the position of the eye is within the combined registration volume, wherein controlling the operation of the display includes providing, to the user, a notification of misregistration based at least partly on determining that the eye is not within the combined registration volume.

10 . A computer-implemented method for controlling a wearable display configured to project light to an eye of a user to display virtual image content, the method comprising:

11 . The computer-implemented method of claim 10 , wherein the combined registration volume includes all of the points that are included in both the display registration volume and the housing registration volume.

12 . The computer-implemented method of claim 10 , wherein the housing registration volume is defined relative to one or more housing openings in the frame.

13 . The computer-implemented method of claim 12 , wherein the housing registration volume has a cone-like shape with a base defined by the shape and size of the one or more housing openings.

14 . The computer-implemented method of claim 10 , wherein the position of the eye is a center of rotation of the eye.

15 . The computer-implemented method of claim 10 , wherein the display registration volume has a frustum shape.

16 . The computer-implemented method of claim 10 , wherein the notification of misregistration is provided to the user based on determining that the position of the eye is more than a threshold distance outside of the combined registration volume.

17 . The computer-implemented method of claim 10 , wherein the display registration volume is defined based at least partly on a type of application associated with virtual content being presented through the display.

US18/962,659

2018-01-17

2024-11-27

Display systems and methods for determining registration between a display and a user's eyes

Active

US12474575B2

( en )

Priority Applications (2)

Application Number

Priority Date

Filing Date

Title

US18/962,659

US12474575B2

( en )

2018-01-17

2024-11-27

Display systems and methods for determining registration between a display and a user's eyes

US19/362,422

US20260044001A1

( en )

2018-01-17

2025-10-19

Display systems and methods for determining registration between a display and a user's eyes

Applications Claiming Priority (8)

Application Number

Priority Date

Filing Date

Title

US201862618559P

2018-01-17

US201862644321P

2018-03-16

US201862702849P

2018-07-24

US16/251,017

US10917634B2

( en )

2018-01-17

2019-01-17

Display systems and methods for determining registration between a display and a user's eyes

US17/125,932

US11290706B2

( en )

2018-01-17

2020-12-17

Display systems and methods for determining registration between a display and a user's eyes

US17/706,429

US11880033B2

( en )

2018-01-17

2022-03-28

Display systems and methods for determining registration between a display and a user's eyes

US18/521,613

US12196952B2

( en )

2018-01-17

2023-11-28

Display systems and methods for determining registration between a display and a user's eyes

US18/962,659

US12474575B2

( en )

2018-01-17

2024-11-27

Display systems and methods for determining registration between a display and a user's eyes

Related Parent Applications (1)

Application Number

Title

Priority Date

Filing Date

US18/521,613

Continuation

US12196952B2

( en )

2018-01-17

2023-11-28

Display systems and methods for determining registration between a display and a user's eyes

Related Child Applications (1)

Application Number

Title

Priority Date

Filing Date

US19/362,422

Continuation

US20260044001A1

( en )

2018-01-17

2025-10-19

Display systems and methods for determining registration between a display and a user's eyes

Publications (2)

Publication Number

Publication Date

US20250085534A1

US20250085534A1 ( en )

2025-03-13

US12474575B2

true

US12474575B2 ( en )

2025-11-18

Family

ID=67213314

Family Applications (5)

Application Number

Title

Priority Date

Filing Date

US16/251,017

Active

US10917634B2

( en )

2018-01-17

2019-01-17

Display systems and methods for determining registration between a display and a user's eyes

US17/706,429

Active

US11880033B2

( en )

2018-01-17

2022-03-28

Display systems and methods for determining registration between a display and a user's eyes

US18/521,613

Active

US12196952B2

( en )

2018-01-17

2023-11-28

Display systems and methods for determining registration between a display and a user's eyes

US18/962,659

Active

US12474575B2

( en )

2018-01-17

2024-11-27

Display systems and methods for determining registration between a display and a user's eyes

US19/362,422

Pending

US20260044001A1

( en )

2018-01-17

2025-10-19

Display systems and methods for determining registration between a display and a user's eyes

Family Applications Before (3)

Application Number

Title

Priority Date

Filing Date

US16/251,017

Active

US10917634B2

( en )

2018-01-17

2019-01-17

Display systems and methods for determining registration between a display and a user's eyes

US17/706,429

Active

US11880033B2

( en )

2018-01-17

2022-03-28

Display systems and methods for determining registration between a display and a user's eyes

US18/521,613

Active

US12196952B2

( en )

2018-01-17

2023-11-28

Display systems and methods for determining registration between a display and a user's eyes

Family Applications After (1)

Application Number

Title

Priority Date

Filing Date

US19/362,422

Pending

US20260044001A1

( en )

2018-01-17

2025-10-19

Display systems and methods for determining registration between a display and a user's eyes

Country Status (9)

Country

Link

( 5 )

US10917634B2

( en )

( 3 )

EP4668065A1

( en )

( 3 )

JP7291708B2

( en )

( 2 )

KR102891539B1

( en )

( 2 )

CN111771179B

( en )

( 1 )

AU2019209950A1

( en )

( 1 )

CA3088116A1

( en )

( 2 )

IL275824B2

( en )

( 1 )

WO2019143864A1

( en )

Families Citing this family (70)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

KR102269065B1

( en )

2016-10-05

2021-06-24

ë§¤ì§ ë¦½, ì¸ì½í¬ë ì´í°ë

Periocular Testing for Mixed Reality Correction

JP2018061621A

( en )

2016-10-11

2018-04-19

ãªããã¹ ãã¼ã¨ã«ã·ã¼

Fundus image capturing apparatus, fundus image capturing method, and fundus image capturing program

US10984237B2

( en )

2016-11-10

2021-04-20

Neurotrack Technologies, Inc.

Method and system for correlating an image capturing device to a human user for analyzing gaze information associated with cognitive performance

TWI829656B

( en )

2017-10-22

2024-01-21

ä»¥è²ååé¯å§æ¯æéå¬å¸

Head-mounted augmented reality device employing an optical bench

CA3088116A1

( en )

2018-01-17

2019-07-25

Magic Leap, Inc.

Display systems and methods for determining registration between a display and a user's eyes

KR102792495B1

( en )

2018-01-17

2025-04-04

ë§¤ì§ ë¦½, ì¸ì½í¬ë ì´í°ë

Determination of eye rotation center, depth plane selection, and render camera positioning in display systems

CN111971609B

( en )

2018-04-06

2023-03-14

ä¾è§è·¯å½éå¬å¸

Method for customizing a head mounted device adapted for generating a virtual image

US11762207B2

( en )

2018-06-29

2023-09-19

Magic Leap, Inc.

Head mounted audio-visual display system

CN117369133A

( en )

2018-07-13

2024-01-09

å¥è·å¬å¸

Systems and methods for display binocular deformation compensation

US11567336B2

( en )

2018-07-24

2023-01-31

Magic Leap, Inc.

Display systems and methods for determining registration between display and eyes of user

EP3830674A4

( en )

2018-08-03

2022-04-20

Magic Leap, Inc.

SELECTION OF DEPTH LEVELS FOR MULTI-LEVEL SCREEN DISPLAY SYSTEMS BY A USER CLASSIFICATION

JP7338146B2

( en )

2018-11-27

2023-09-05

ã½ãã¼ã°ã«ã¼ãæ ªå¼ä¼ç¤¾

Display control device, display control method and display control program

TWI683136B

( en )

2019-01-03

2020-01-21

å®ç¢è¡ä»½æéå¬å¸

Video see-through head mounted display and control method thereof

WO2020154524A1

( en )

2019-01-25

2020-07-30

Magic Leap, Inc.

Eye-tracking using images having different exposure times

FR3093215B1

( en )

2019-02-22

2021-08-27

Fogale Nanotech

Method and device for monitoring the environment of a robot

EP3901919B1

( en )

2019-04-17

2025-04-16

Rakuten Group, Inc.

Display control device, display control method, program, and non-transitory computer-readable information recording medium

US11181973B2

( en )

2019-05-09

2021-11-23

Apple Inc.

Techniques related to configuring a display device

US11315288B2

( en )

2019-05-20

2022-04-26

Magic Leap, Inc.

Systems and techniques for estimating eye pose

US11353698B1

( en )

2019-05-29

2022-06-07

Facebook Technologies, Llc

Dual Purkinje imaging with ellipsoidal lensing structure

US11153512B1

( en )

2019-05-29

2021-10-19

Facebook Technologies, Llc

Imaging and display with ellipsoidal lensing structure

KR20220024410A

( en )

2019-06-27

2022-03-03

ë£¨ë¨¸ì¤ ë¦¬ë¯¸í°ë

Gaze tracking device and method based on eye imaging through a light guide optical element

US11217022B1

( en )

2019-06-28

2022-01-04

Snap Inc.

Light invariant pattern mapping

US11156829B2

( en )

2019-07-29

2021-10-26

Facebook Technologies, Llc

Pupil expander cailibration

US11064188B2

( en )

2019-07-29

2021-07-13

Htc Corporation

Driving method for calculating interpupillary distance and related head-mounted device

CN112346558B

( en )

2019-08-06

2024-08-02

è¹æå¬å¸

Eye tracking system

WO2021024525A1

( en )

2019-08-06

2021-02-11

ããã½ããã¯ï¼©ï½ããã¸ã¡ã³ãæ ªå¼ä¼ç¤¾

Display device

CN112419986A

( en )

2019-08-20

2021-02-26

Oppoå¹¿ä¸ç§»å¨éä¿¡æéå¬å¸

Image display method, device and system of head-mounted display device

US20210068652A1

( en )

2019-09-09

2021-03-11

Apple Inc.

Glint-Based Gaze Tracking Using Directional Light Sources

IL269809B

( en )

2019-10-03

2021-12-01

Eyejets Ltd

Compact retinal scanning device for tracking movement of the eyeâs pupil and applications thereof

US12231613B2

( en )

2019-11-06

2025-02-18

Hes Ip Holdings, Llc

System and method for displaying an object with depths

US11228745B2

( en )

2019-11-26

2022-01-18

Varjo Technologies Oy

Display apparatus and method of correcting image distortion therefor

CN115053270B

( en )

2019-12-09

2026-01-09

å¥è·å¬å¸

Systems and methods for operating a head-mounted display system based on user identity.

US11375180B2

( en )

2019-12-10

2022-06-28

Yousseff El Moumni

Video engine for generating an enhanced impression of depth based upon two-dimensioned video content

CN110927973A

( en )

2019-12-16

2020-03-27

Oppoå¹¿ä¸ç§»å¨éä¿¡æéå¬å¸

display device

CN110933390B

( en )

2019-12-16

2022-01-07

Oppoå¹¿ä¸ç§»å¨éä¿¡æéå¬å¸

Display method and device based on image projection

CN113010125B

( en )

2019-12-20

2024-03-19

ææ¯è¡ä»½å¬å¸

Method, computer program product, and binocular headset controller

CN114787687B

( en )

2019-12-25

2024-07-30

é²å§æ¯æéå¬å¸

Systems and methods for eye tracking based on redirection of light from the eye using an optical arrangement associated with a light guide optical element

US11687778B2

( en )

2020-01-06

2023-06-27

The Research Foundation For The State University Of New York

Fakecatcher: detection of synthetic portrait videos using biological signals

WO2021163373A1

( en )

2020-02-14

2021-08-19

Magic Leap, Inc.

3d object annotation

WO2021179876A1

( en )

2020-03-09

2021-09-16

Guangdong Oppo Mobile Telecommunications Corp., Ltd.

Method and system for implementing adaptive feature detection for vslam systems

US11768379B2

( en )

2020-03-17

2023-09-26

Apple Inc.

Electronic device with facial sensors

EP4121813A4

( en )

2020-03-20

2024-01-17

Magic Leap, Inc.

SYSTEMS AND METHODS FOR RETINA IMAGING AND TRACKING

WO2021258078A1

( en )

2020-06-19

2021-12-23

Hes Ip Holdings, Llc

Image display systems for eyebox expansion and methods of making the same

CN116033864A

( en )

2020-07-15

2023-04-28

å¥è·å¬å¸

Eye tracking using a non-spherical cornea model

JP7565428B2

( en )

2020-07-23

2024-10-10

ãã¸ãã¯ ãªã¼ãï¼ ã¤ã³ã³ã¼ãã¬ã¤ããã

Eye Tracking Using Alternating Sampling

TWI851916B

( en )

2020-08-14

2024-08-11

ç¾åæµ·ææºè²¡æ§è¡æéå¬å¸

Systems and methods for superimposing virtual image on real-time image

US11410580B2

( en )

2020-08-20

2022-08-09

Facebook Technologies, Llc.

Display non-uniformity correction

US11774759B2

( en )

2020-09-03

2023-10-03

Hes Ip Holdings, Llc

Systems and methods for improving binocular vision

US11953689B2

( en )

2020-09-30

2024-04-09

Hes Ip Holdings, Llc

Virtual image display system for virtual reality and augmented reality devices

EP3985483A1

( en )

2020-10-19

2022-04-20

Carl Zeiss Vision International GmbH

Computer-implemented method for determining a position of a center of rotation of an eye using a mobile device, mobile device and computer program

KR102803339B1

( en )

2020-11-10

2025-05-07

ì¼ì±ì ìì£¼ìíì¬

Wearable electronic device comprising display

US12529891B2

( en )

2020-12-17

2026-01-20

Lumus Ltd.

Optical systems and methods for eye tracking based on eye imaging via collimating element and light-guide optical element

CN112634127B

( en )

2020-12-22

2022-07-29

å¤©æ´¥å¤§å¦

An Unsupervised Stereo Image Redirection Method

US11733773B1

( en )

2020-12-29

2023-08-22

Meta Platforms Technologies, Llc

Dynamic uniformity correction for boundary regions

JP7601521B2

( en )

2021-02-08

2024-12-17

ãã¼ãº ã¢ã¤ãã¼ ãã¼ã«ãã£ã³ã°ã¹ ã¨ã«ã¨ã«ã·ã¼

Systems and methods for enhancing vision

US11681363B2

( en )

2021-03-29

2023-06-20

Meta Platforms Technologies, Llc

Waveguide correction map compression

KR20240097972A

( en )

2021-05-19

2024-06-27

ë£¨ë¨¸ì¤ ë¦¬ë¯¸í°ë

Active optical engine

CN114973238B

( en )

2021-12-08

2025-03-25

çèåä¸å¤§å¦

Fruit positioning method and device based on binocular vision system

US11710212B1

( en )

2022-01-21

2023-07-25

Meta Platforms Technologies, Llc

Display non-uniformity correction

US11754846B2

( en )

2022-01-21

2023-09-12

Meta Platforms Technologies, Llc

Display non-uniformity correction

CN114820811B

( en )

2022-04-18

2025-04-18

ä¸åå¤§å¦

A fundamental matrix solution and calibration method for synchronous cameras based on a moving sphere

JPWO2023203923A1

( en )

2022-04-22

2023-10-26

US12596431B2

( en )

2022-06-06

2026-04-07

Sony Interactive Entertainment Inc.

Virtual reality content display system and virtual reality content display method

CN120917405A

( en )

2023-03-20

2025-11-07

è¹æå¬å¸

System and method for specifying configuration of electronic device

US12579295B2

( en )

2023-03-30

2026-03-17

Apple Inc.

Systems and methods for electronic device access

US12561463B2

( en )

2023-03-30

2026-02-24

Apple Inc.

Systems and methods for controlling access to features of an electronic device

CN119583786A

( en )

2023-09-06

2025-03-07

åä¸ºææ¯æéå¬å¸

Display method and electronic device

US20250251897A1

( en )

2024-02-01

2025-08-07

Apple Inc.

Remote viewing and control of an electronic device

CN119033326B

( en )

2024-08-22

2026-02-24

ä¸æµ·èå½±æºåå»çç§ææéå¬å¸

Field of view detection methods, devices and equipment

CN119148392A

( en )

2024-11-11

2024-12-17

æµæ±å®ç¦§ç§ææéå¬å¸

Display device and near-to-eye display apparatus

Citations (160)

* Cited by examiner, â Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US5280312A

( en )

1991-01-22

1994-01-18

Canon Kabushiki Kaisha

Visual axis detection apparatus

JPH09193803A

( en )

1996-01-19

1997-07-29

Furukawa Electric Co Ltd:The

Safety monitoring method near the platform

US5844656A

( en )

1996-11-07

1998-12-01

Xybernaut Corporation

Head mounted display with adjustment components

US20020181115A1

( en )

2001-04-20

2002-12-05

John Hopkins University

Head mounted display with full field of view and high resolution

US20030030597A1

( en )

2001-08-13

2003-02-13

Geist Richard Edwin

Virtual display apparatus for mobile activities

US6590573B1

( en )

1983-05-09

2003-07-08

David Michael Geshwind

Interactive computer system for creating three-dimensional image information and for converting two-dimensional image information for three-dimensional display systems

US6611283B1

( en )

1997-11-21

2003-08-26

Canon Kabushiki Kaisha

Method and apparatus for inputting three-dimensional shape information

US20030184868A1

( en )

2001-05-07

2003-10-02

Geist Richard Edwin

Head-mounted virtual display apparatus with a near-eye light deflecting element in the peripheral field of view

JP2003307774A

( en )

2002-04-17

2003-10-31

Canon Inc

Optical equipment with gaze detection function

US6771424B1

( en )

1999-08-31

2004-08-03

Shimadzu Corporation

Head-mounted display device

US20040174496A1

( en )

2003-03-06

2004-09-09

Qiang Ji

Calibration-free gaze tracking under natural head movement

US20040238732A1

( en )

2001-10-19

2004-12-02

Andrei State

Methods and systems for dynamic virtual convergence and head mountable display

JP2005013752A

( en )

2000-07-28

2005-01-20

National Institute Of Advanced Industrial & Technology

Gaze detection system

US6850221B1

( en )

1995-09-05

2005-02-01

Interlink Electronics, Inc.

Trigger operated electronic device

US20060028436A1

( en )

1992-03-05

2006-02-09

Armstrong Brad A

Image controller

US20060250322A1

( en )

2005-05-09

2006-11-09

Optics 1, Inc.

Dynamic vergence and focus control for head-mounted displays

US20070081123A1

( en )

2005-10-07

2007-04-12

Lewis Scott W

Digital eyewear

US20070120986A1

( en )

2005-11-08

2007-05-31

Takashi Nunomaki

Imaging device, information processing method, and computer program

US20070273983A1

( en )

2006-05-26

2007-11-29

Hebert Raymond T

Devices, methods, and systems for image viewing

US20080002262A1

( en )

2006-06-29

2008-01-03

Anthony Chirieleison

Eye tracking head mounted display

US20080106489A1

( en )

2006-11-02

2008-05-08

Brown Lawrence G

Systems and methods for a head-mounted display

US20080117289A1

( en )

2004-08-06

2008-05-22

Schowengerdt Brian T

Variable Fixation Viewing Distance Scanned Light Displays

US20090160872A1

( en )

2007-12-20

2009-06-25

Raytheon Company

Imaging System

US20090167673A1

( en )

2007-12-26

2009-07-02

Kerofsky Louis J

Methods and Systems for Display Source Light Management with Variable Delay

US20100045932A1

( en )

2005-05-19

2010-02-25

The Johns Hopkins University

Wireless scleral search coil including systems for measuring eye movement and methods related thereto

US20100283969A1

( en )

2007-10-23

2010-11-11

Mcgill University

Binocular vision assessment and/or therapy

US20110075257A1

( en )

2009-09-14

2011-03-31

The Arizona Board Of Regents On Behalf Of The University Of Arizona

3-Dimensional electro-optical see-through displays

US20120022220A1

( en )

2005-07-19

2012-01-26

Nektar Therapeutics

Method for Preparing Polymer Maleimides

US20120033179A1

( en )

2009-02-26

2012-02-09

Timo Kratzer

Method and apparatus for determining the location of the ocular pivot point

US20120068913A1

( en )

2010-09-21

2012-03-22

Avi Bar-Zeev

Opacity filter for see-through head mounted display

US20120127062A1

( en )

2010-11-18

2012-05-24

Avi Bar-Zeev

Automatic focus improvement for augmented reality displays

US20120127284A1

( en )

2010-11-18

2012-05-24

Avi Bar-Zeev

Head-mounted display device which provides surround video

US20120139817A1

( en )

2009-08-13

2012-06-07

Display systems and methods for determining registration between a display and … — Magic Leap, Inc. (US12474575B2)

Related documents