ABSTRACT
Abstract
A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive image information outputted by the HMD. For example, the HAMID-to-eye vertical alignment may be different between the left and right eyes. The wearable device may determine if the HMD is level on the user's head and may then provide the user with a left-eye alignment marker and a right-eye alignment marker. Based on user feedback, the wearable device may determine if there is any left-right vertical misalignment and may take actions to reduce or minimize the effects of any misalignment.
Description
PRIORITY CLAIM
This application is a continuation of U.S. patent application Ser. No. 17/890,998, filed Aug. 18, 2022, which is a continuation of U.S. patent application Ser. No. 17/258,083, filed on Jan. 5, 2021, now U.S. Pat. No. 11,422,620, which is a 371 of International Application No. PCT/US2019/043302, filed Jul. 24, 2019, which claims benefit of U.S. Provisional Application No. 62/702,865, filed Jul. 24, 2018.
INCORPORATION BY REFERENCE
This application incorporates by reference the entirety of each of the following patent applications: U.S. Provisional Application No. 62/714,649, filed on Aug. 3, 2018; U.S. Provisional Application No. 62/875,474, filed on Jul. 17, 2019; U.S. application Ser. No. 16/251,017, filed on Jul. 17, 2019; and PCT Application No. PCT/US2019/043096, filed on Jul. 23, 2019.
FIELD
The present disclosure relates to display systems, including virtual reality and augmented reality display systems, and, more particularly, to systems and methods for aligning left and right displays in a wearable display system.
BACKGROUND
Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ, âaugmented realityâ, or âmixed realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or âVRâ, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or âARâ, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; a mixed reality, or âMRâ, related to merging real and virtual worlds to produce new environments where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR and MR technology.
SUMMARY
Various examples of display alignment in a mixed reality system are disclosed.
In some embodiments, an augment reality system is provided. The augmented reality system comprises a head-mounted display configured to present virtual content by outputting light to a user, an imaging device configured to capture images of eyes of the user, and at least one processor communicatively coupled to the head-mounted display and the imaging device. The at least one processor is configured to provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input. In some embodiments, the at least one processor is configured to determine if the head-mounted display system is level relative to the user's left and right eyes; and to provide the user with feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing the left or right-eye alignment marker.
In some other embodiments, a method is provided for vertically aligning, with a user's left and right eyes, displayed content of a left-eye display and a right-eye display of a head-mounted display system. The method includes providing, with the left-eye display, a left-eye alignment marker; providing, with the right-eye display, a right-eye alignment marker; receiving user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjusting image content in at least one of the left-eye and right-eye displays based on the received user input. In some embodiments, the method further comprises determining a level of the head-mounted display system relative to the user's left and right eyes; providing the user feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing the left or right-eye alignment marker.
In some embodiments, an augment reality system is provided. The augmented reality system comprises a head-mounted display (TIMID) configured to present virtual content by outputting light to a user and at least one processor communicatively coupled to the HMD. The HMD comprises a left-eye display configured to present virtual content to the user's left eye and a right-eye display configured to present virtual content to the user's right eye. The at least one processor is configured to provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input.
In some embodiments, an augment reality system is provided. The augmented reality system comprises a head-mounted display (TIMID) configured to present virtual content by outputting light to a user, an imaging system configured to capture images of eyes of the user, and at least one processor communicatively coupled to the HMD and the imaging device. The at least one processor is configured to determine an interocular axis of the user that extends between the user's left and right eyes based at least in part on one or more images captured by the imaging system; determine an orientation of the HMD relative to the interocular axis of the user; and provide the user with feedback based on the determined orientation of the HMD relative to the interocular axis of the user.
Additional examples of embodiments are enumerated below.
Example 1. An augmented reality system comprising:
a head-mounted display configured to present virtual content by outputting light to a user; an imaging device configured to capture images of eyes of the user; and at least one processor communicatively coupled to the head-mounted display and the imaging device, the at least one processor configured to:
provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input.
Example 2. The augmented reality system of Example 1, wherein the processor is configured to:
determine if the head-mounted display system is level relative to the user's left and right eyes; and provide the user with feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing, with the left-eye display, a left-eye alignment marker.
Example 3. The augmented reality system of Example 1, wherein the processor is configured to provide the user with feedback on whether the head-mounted display is level relative to the user's left and right eyes by presenting a level marker having an orientation that changes in relation to an orientation of the head-mounted display relative to the user's left and right eyes.
Example 4. The augmented reality system of Example 1, further comprising an eye-tracking system, wherein the processor is configured to determine if the head-mounted display system is level relative to the user's left and right eyes based on eye tracking data from the eye-tracking system.
Example 5. The augmented reality system of Example 1, further comprising an eye-tracking system, wherein the processor is further configured to determine an interocular axis of the user that extends between the user's left and right eyes based on eye tracking data from the eye-tracking system.
Example 6. The augmented reality system of Example 5, wherein the processor is further configured to determine if the head-mounted display system is level relative to the user's left and right eyes by determining an orientation of the eye-tracking system relative to the interocular axis of the user.
Example 7. The augmented reality system of Example 5, wherein the processor is configured to provide the user with feedback on whether the head-mounted display is level relative to the user's left and right eyes by presenting a level marker having an orientation that changes in relation with the orientation of the eye-tracking system relative to the interocular axis of the user.
Example 8. The augmented reality system of Example 5, wherein the processor is configured to provide the user with feedback on whether the head-mounted display is level relative to the user's left and right eyes by presenting a static level marker associated with an orientation of the head-mounted display and presenting a dynamic level marker associated with an orientation of the interocular axis, wherein the dynamic level marker moves relative to the static level marker as the orientation of the interocular axis changes relative to the head-mounted display.
Example 9. The augmented reality system of Example 8, wherein the dynamic level marker merges with the static level marker when the head-mounted display is level relative to the interocular axis.
Example 10. The augmented reality system of Example 1, wherein the left-eye alignment marker comprises a first horizontal line and wherein the right-eye alignment marker comprises a second horizontal line.
Example 11. The augmented reality system of Example 10, wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines.
Example 12. The augmented reality system of Example 10, wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines until the first and second horizontal lines are level from the perspective of the user.
Example 13. The augmented reality system of Example 1, wherein the head-mounted display comprises a first waveguide stack configured to pass light from the world into the left eye of the user and a second waveguide stack configured to pass light from the world into the right eye of the user and wherein each waveguide stack comprises a plurality of waveguides.
Example 14. The augmented reality system of Example 1, wherein the head-mounted display comprises a first waveguide stack configured to pass light from the world into the left eye of the user and a second waveguide stack configured to pass light from the world into the right eye of the user,
wherein each waveguide stack comprises a plurality of waveguides where one or more waveguides of that waveguide stack are configured to output light to the user with a different amount of wavefront divergence than one or more other waveguides of that waveguide stack, wherein different amounts of wavefront divergence are associated with different accommodation by the eye, and wherein the outputted light with different amounts of wavefront divergence forms virtual objects at different perceived depths away from the user.
Example 15. A method for vertically aligning, with a user's left and right eyes, displayed content of a left-eye display and a right-eye display of a head-mounted display system, the method comprising:
providing, with the left-eye display, a left-eye alignment marker; providing, with the right-eye display, a right-eye alignment marker; receiving user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjusting image content in at least one of the left-eye and right-eye displays based on the received user input.
Example 16. The method of Example 15, further comprising:
determining a level of the head-mounted display system relative to the user's left and right eyes; and providing the user feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing the left or right-eye alignment markers.
Example 17. The method of Example 15, wherein the head-mounted display system comprises an eye-tracking system, wherein determining the level of the head-mounted display system relative to the user's left and right eyes comprises utilizing the eye-tracking system to determine an interocular axis of the user that extends between the user's left and right eyes and determining a level of the head-mounted display system relative to the interocular axis.
Example 18. The method of Example 17, wherein utilizing the eye-tracking system to determine an interocular axis of the user comprises:
determining, with the eye-tracking system, a center of rotation of the user's left eye; determining, with the eye-tracking system, a center of rotation of the user's right eye; and determining a position of a line extending between the centers of rotation of the user's left and right eyes, wherein the line constitutes the interocular axis.
Example 19. The method of Example 15, wherein providing the left-eye alignment marker comprises providing, with the left-eye display, a first vertical alignment marker and a first horizontal alignment marker and wherein providing the right-eye alignment marker comprises providing, with the right-eye display, a second vertical alignment marker and a second horizontal alignment marker.
Example 20. The method of Example 19, wherein, when viewed by the user, the first and second vertical alignment markers fuse together in the user's vision and the first and second horizontal alignment markers remain unfused in the user's vision.
Example 21. The method of Example 19, receiving the user input to adjust the at least one of the left-eye and right-eye alignment markers comprises receiving user input to move at least one of the first and second horizontal alignment markers vertically.
Example 22. The method of Example 19, receiving the user input to adjust the at least one of the left-eye and right-eye alignment markers comprises receiving user input to move at least one of the first and second horizontal alignment markers vertically until the first and second horizontal alignment markers are vertically aligned with each other in the user's vision.
Example 23. The method of Example 15, further comprising:
determining that the user has worn the head-mounted display system for a given threshold of time; and <li id="ul0015-0002" num="00
PRIORITY CLAIM
This application is a continuation of U.S. patent application Ser. No. 17/890,998, filed Aug. 18, 2022, which is a continuation of U.S. patent application Ser. No. 17/258,083, filed on Jan. 5, 2021, now U.S. Pat. No. 11,422,620, which is a 371 of International Application No. PCT/US2019/043302, filed Jul. 24, 2019, which claims benefit of U.S. Provisional Application No. 62/702,865, filed Jul. 24, 2018.
INCORPORATION BY REFERENCE
This application incorporates by reference the entirety of each of the following patent applications: U.S. Provisional Application No. 62/714,649, filed on Aug. 3, 2018; U.S. Provisional Application No. 62/875,474, filed on Jul. 17, 2019; U.S. application Ser. No. 16/251,017, filed on Jul. 17, 2019; and PCT Application No. PCT/US2019/043096, filed on Jul. 23, 2019.
FIELD
The present disclosure relates to display systems, including virtual reality and augmented reality display systems, and, more particularly, to systems and methods for aligning left and right displays in a wearable display system.
BACKGROUND
Modern computing and display technologies have facilitated the development of systems for so called âvirtual realityâ, âaugmented realityâ, or âmixed realityâ experiences, wherein digitally reproduced images or portions thereof are presented to a user in a manner wherein they seem to be, or may be perceived as, real. A virtual reality, or âVRâ, scenario typically involves presentation of digital or virtual image information without transparency to other actual real-world visual input; an augmented reality, or âARâ, scenario typically involves presentation of digital or virtual image information as an augmentation to visualization of the actual world around the user; a mixed reality, or âMRâ, related to merging real and virtual worlds to produce new environments where physical and virtual objects co-exist and interact in real time. As it turns out, the human visual perception system is very complex, and producing a VR, AR, or MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements is challenging. Systems and methods disclosed herein address various challenges related to VR, AR and MR technology.
SUMMARY
Various examples of display alignment in a mixed reality system are disclosed.
In some embodiments, an augment reality system is provided. The augmented reality system comprises a head-mounted display configured to present virtual content by outputting light to a user, an imaging device configured to capture images of eyes of the user, and at least one processor communicatively coupled to the head-mounted display and the imaging device. The at least one processor is configured to provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input. In some embodiments, the at least one processor is configured to determine if the head-mounted display system is level relative to the user's left and right eyes; and to provide the user with feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing the left or right-eye alignment marker.
In some other embodiments, a method is provided for vertically aligning, with a user's left and right eyes, displayed content of a left-eye display and a right-eye display of a head-mounted display system. The method includes providing, with the left-eye display, a left-eye alignment marker; providing, with the right-eye display, a right-eye alignment marker; receiving user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjusting image content in at least one of the left-eye and right-eye displays based on the received user input. In some embodiments, the method further comprises determining a level of the head-mounted display system relative to the user's left and right eyes; providing the user feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing the left or right-eye alignment marker.
In some embodiments, an augment reality system is provided. The augmented reality system comprises a head-mounted display (TIMID) configured to present virtual content by outputting light to a user and at least one processor communicatively coupled to the HMD. The HMD comprises a left-eye display configured to present virtual content to the user's left eye and a right-eye display configured to present virtual content to the user's right eye. The at least one processor is configured to provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input.
In some embodiments, an augment reality system is provided. The augmented reality system comprises a head-mounted display (TIMID) configured to present virtual content by outputting light to a user, an imaging system configured to capture images of eyes of the user, and at least one processor communicatively coupled to the HMD and the imaging device. The at least one processor is configured to determine an interocular axis of the user that extends between the user's left and right eyes based at least in part on one or more images captured by the imaging system; determine an orientation of the HMD relative to the interocular axis of the user; and provide the user with feedback based on the determined orientation of the HMD relative to the interocular axis of the user.
Additional examples of embodiments are enumerated below.
Example 1. An augmented reality system comprising:
a head-mounted display configured to present virtual content by outputting light to a user; an imaging device configured to capture images of eyes of the user; and at least one processor communicatively coupled to the head-mounted display and the imaging device, the at least one processor configured to:
provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input.
Example 2. The augmented reality system of Example 1, wherein the processor is configured to:
determine if the head-mounted display system is level relative to the user's left and right eyes; and provide the user with feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing, with the left-eye display, a left-eye alignment marker.
Example 3. The augmented reality system of Example 1, wherein the processor is configured to provide the user with feedback on whether the head-mounted display is level relative to the user's left and right eyes by presenting a level marker having an orientation that changes in relation to an orientation of the head-mounted display relative to the user's left and right eyes.
Example 4. The augmented reality system of Example 1, further comprising an eye-tracking system, wherein the processor is configured to determine if the head-mounted display system is level relative to the user's left and right eyes based on eye tracking data from the eye-tracking system.
Example 5. The augmented reality system of Example 1, further comprising an eye-tracking system, wherein the processor is further configured to determine an interocular axis of the user that extends between the user's left and right eyes based on eye tracking data from the eye-tracking system.
Example 6. The augmented reality system of Example 5, wherein the processor is further configured to determine if the head-mounted display system is level relative to the user's left and right eyes by determining an orientation of the eye-tracking system relative to the interocular axis of the user.
Example 7. The augmented reality system of Example 5, wherein the processor is configured to provide the user with feedback on whether the head-mounted display is level relative to the user's left and right eyes by presenting a level marker having an orientation that changes in relation with the orientation of the eye-tracking system relative to the interocular axis of the user.
Example 8. The augmented reality system of Example 5, wherein the processor is configured to provide the user with feedback on whether the head-mounted display is level relative to the user's left and right eyes by presenting a static level marker associated with an orientation of the head-mounted display and presenting a dynamic level marker associated with an orientation of the interocular axis, wherein the dynamic level marker moves relative to the static level marker as the orientation of the interocular axis changes relative to the head-mounted display.
Example 9. The augmented reality system of Example 8, wherein the dynamic level marker merges with the static level marker when the head-mounted display is level relative to the interocular axis.
Example 10. The augmented reality system of Example 1, wherein the left-eye alignment marker comprises a first horizontal line and wherein the right-eye alignment marker comprises a second horizontal line.
Example 11. The augmented reality system of Example 10, wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines.
Example 12. The augmented reality system of Example 10, wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines until the first and second horizontal lines are level from the perspective of the user.
Example 13. The augmented reality system of Example 1, wherein the head-mounted display comprises a first waveguide stack configured to pass light from the world into the left eye of the user and a second waveguide stack configured to pass light from the world into the right eye of the user and wherein each waveguide stack comprises a plurality of waveguides.
Example 14. The augmented reality system of Example 1, wherein the head-mounted display comprises a first waveguide stack configured to pass light from the world into the left eye of the user and a second waveguide stack configured to pass light from the world into the right eye of the user,
wherein each waveguide stack comprises a plurality of waveguides where one or more waveguides of that waveguide stack are configured to output light to the user with a different amount of wavefront divergence than one or more other waveguides of that waveguide stack, wherein different amounts of wavefront divergence are associated with different accommodation by the eye, and wherein the outputted light with different amounts of wavefront divergence forms virtual objects at different perceived depths away from the user.
Example 15. A method for vertically aligning, with a user's left and right eyes, displayed content of a left-eye display and a right-eye display of a head-mounted display system, the method comprising:
providing, with the left-eye display, a left-eye alignment marker; providing, with the right-eye display, a right-eye alignment marker; receiving user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjusting image content in at least one of the left-eye and right-eye displays based on the received user input.
Example 16. The method of Example 15, further comprising:
determining a level of the head-mounted display system relative to the user's left and right eyes; and providing the user feedback on whether the head-mounted display system is level relative to the user's left and right eyes before providing the left or right-eye alignment markers.
Example 17. The method of Example 15, wherein the head-mounted display system comprises an eye-tracking system, wherein determining the level of the head-mounted display system relative to the user's left and right eyes comprises utilizing the eye-tracking system to determine an interocular axis of the user that extends between the user's left and right eyes and determining a level of the head-mounted display system relative to the interocular axis.
Example 18. The method of Example 17, wherein utilizing the eye-tracking system to determine an interocular axis of the user comprises:
determining, with the eye-tracking system, a center of rotation of the user's left eye; determining, with the eye-tracking system, a center of rotation of the user's right eye; and determining a position of a line extending between the centers of rotation of the user's left and right eyes, wherein the line constitutes the interocular axis.
Example 19. The method of Example 15, wherein providing the left-eye alignment marker comprises providing, with the left-eye display, a first vertical alignment marker and a first horizontal alignment marker and wherein providing the right-eye alignment marker comprises providing, with the right-eye display, a second vertical alignment marker and a second horizontal alignment marker.
Example 20. The method of Example 19, wherein, when viewed by the user, the first and second vertical alignment markers fuse together in the user's vision and the first and second horizontal alignment markers remain unfused in the user's vision.
Example 21. The method of Example 19, receiving the user input to adjust the at least one of the left-eye and right-eye alignment markers comprises receiving user input to move at least one of the first and second horizontal alignment markers vertically.
Example 22. The method of Example 19, receiving the user input to adjust the at least one of the left-eye and right-eye alignment markers comprises receiving user input to move at least one of the first and second horizontal alignment markers vertically until the first and second horizontal alignment markers are vertically aligned with each other in the user's vision.
Example 23. The method of Example 15, further comprising:
determining that the user has worn the head-mounted display system for a given threshold of time; and in response to determining that the user has worn the head-mounted display system for the given threshold of time, performing determining the level of the head-mounted display system.
Example 24. An augmented reality system comprising:
a head-mounted display configured to present virtual content by outputting light to a user, the head-mounted display comprising a left-eye display configured to present virtual content to the user's left eye and a right-eye display configured to present virtual content to the user's right eye; and at least one processor communicatively coupled to the head-mounted display, the at least one processor configured to:
provide, with the left-eye display, a left-eye alignment marker; provide, with the right-eye display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input.
Example 25. The augmented reality system of Example 24, wherein the left-eye alignment marker comprises a first horizontal line and wherein the right-eye alignment marker comprises a second horizontal line.
Example 26. The augmented reality system of Example 25, wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines.
Example 27. The augmented reality system of Example 25, wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines until the first and second horizontal lines are level from the perspective of the user.
Example 28. The augmented reality system of Example 24, wherein the left-eye display comprises a first waveguide stack configured to pass light from the world into the left eye of the user and the right-eye display comprises a second waveguide stack configured to pass light from the world into the right eye of the user, and wherein each waveguide stack comprises a plurality of waveguides.
Example 29. The augmented reality system of Example 24, wherein the left-eye display comprises a first waveguide stack configured to pass light from the world into the left eye of the user and the right-eye display comprises a second waveguide stack configured to pass light from the world into the right eye of the user,
wherein each waveguide stack comprises a plurality of waveguides where one or more waveguides of that waveguide stack are configured to output light to the user with a different amount of wavefront divergence than one or more other waveguides of that waveguide stack, wherein different amounts of wavefront divergence are associated with different accommodation by the eye, and wherein the outputted light with different amounts of wavefront divergence forms virtual objects at different perceived depths away from the user.
Example 30. The augmented reality system of Example 24, wherein to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, the at least one processor is configured to:
provide, with the left-eye display, a first vertical alignment marker and a first horizontal alignment marker; and provide, with the right-eye display, a second vertical alignment marker and a second horizontal alignment marker.
Example 31. The augmented reality system of Example 30, wherein, when viewed by the user, the first and second vertical alignment markers fuse together in the user's vision and the first and second horizontal alignment markers remain unfused in the user's vision.
Example 32. The augmented reality system of Example 30, wherein to receive user input to adjust at least one of the left-eye and the right-eye alignment markers, the at least one processor is configured to:
receive user input to move at least one of the first and second horizontal alignment markers vertically.
Example 33. The augmented reality system of Example 30, wherein to receive user input to adjust at least one of the left-eye and the right-eye alignment markers, the at least one processor is configured to:
receive user input to move at least one of the first and second horizontal alignment markers vertically until the first and second horizontal alignment markers are vertically aligned with each other in the user's vision.
Example 34. The augmented reality system of Example 24, wherein the at least one processor is further configured to:
select a first vertical position at which to present the left-eye alignment marker; and select a second vertical position at which to present the right-eye alignment marker, and wherein to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, the at least one processor is configured to:
provide, with the left-eye display, the left-eye alignment marker at the first vertical position; and provide, with the right-eye display, the right-eye alignment marker at the second vertical position.
Example 35. The augmented reality system of Example 24, wherein to select the first vertical position at which to present the left-eye alignment marker and select the second vertical position at which to present the right-eye alignment marker, the at least one processor is configured to:
pseudo- or quasi-randomly select the first vertical position at which to present the left-eye alignment marker; and pseudo- or quasi-randomly select the second vertical position at which to present the right-eye alignment marker.
Example 36. The augmented reality system of Example 24, wherein to vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input, the at least one processor is configured to:
vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input, the first vertical position selected, and the second vertical position selected.
Example 37. The augmented reality system of Example 24, wherein to vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input, the at least one processor is configured to:
adjust one or more extrinsic parameters of at least one of a first render camera that is associated with the left-eye display and a second render camera that is associated with the right-eye display.
Example 38. The augmented reality system of Example 37, wherein the one or more extrinsic parameters include at least one of a position and an orientation.
Example 39. The augmented reality system of Example 24, further comprising an imaging system configured to capture images of eyes of the user, and
wherein the at least one processor is communicatively coupled to the imaging system, the at least one processor further configured to:
determine whether the head-mounted display is level relative to the user's left and right eyes based at least in part on one or more images captured by the imaging system.
Example 40. The augmented reality system of Example 39, wherein the at least one processor is configured to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, in response to a determination that the head-mounted display is level relative to the user's left and right eyes.
Example 41. The augmented reality system of Example 24, wherein the left-eye display comprises a first waveguide stack and the right-eye display comprises a second waveguide stack, each of which comprises a plurality of waveguides configured to output light to the user, and
wherein to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, the at least one processor is configured to:
provide, with a single one of the plurality of waveguides in the first waveguide stack, the left-eye alignment marker; and provide, with a single one of the plurality of waveguides in the second waveguide stack, the right-eye alignment marker.
Example 42. An augmented reality system comprising:
a head-mounted display configured to present virtual content by outputting light to a user; an imaging system configured to capture images of eyes of the user; and at least one processor communicatively coupled to the head-mounted display and the imaging system, the at least one processor configured to:
determine an interocular axis of the user that extends between the user's left and right eyes based at least in part on one or more images captured by the imaging system; determine an orientation of the head-mounted display relative to the interocular axis of the user; and provide the user with feedback based on the determined orientation of the head-mounted display relative to the interocular axis of the user.
Example 43. The augmented reality system of Example 42, wherein to provide the user with feedback based on the determined orientation of the head-mounted display relative to the interocular axis of the user, the at least one processor is configured to:
present a level marker having an orientation that changes in relation to the determined orientation of the head-mounted display relative to the user's left and right eyes.
Example 44. The augmented reality system of Example 42, wherein to provide the user with feedback based on the determined orientation of the head-mounted display relative to the interocular axis of the user, the at least one processor is configured to:
present a static level marker associated with an orientation of the head-mounted display; and present a dynamic level marker associated with an orientation of the interocular axis, wherein the dynamic level marker moves relative to the static level marker as the orientation of the interocular axis changes relative to the head-mounted display.
Example 45. The augmented reality system of Example 44, wherein the dynamic level marker merges with the static level marker when the head-mounted display is level relative to the interocular axis.
Example 46. The augmented reality system of Example 42, wherein to determine the interocular axis of the user that extends between the user's left and right eyes based at least in part on one or more images captured by the imaging system, the at least one processor is configured to:
determine, based at least in part on one or more images of the user's left eye captured by the imaging system, a center of rotation of the user's left eye; determine, based at least in part on one or more images of the user's right eye captured by the imaging system, a center of rotation of the user's right eye; and determine a position of a line extending between the centers of rotation of the user's left and right eyes, wherein the line constitutes the interocular axis of the user.
Example 47. The augmented reality system of Example 42, wherein the at least one processor is further configured to:
determine whether the user has worn the head-mounted display for at least a threshold of amount of time, and wherein the at least one processor is configured to determine the interocular axis of the user that extends between the user's left and right eyes based at least in part on one or more images captured by the imaging system in response to a determination that the user has worn the head-mounted display for at least the threshold of amount of time.
Example 48. The augmented reality system of Example 42, wherein the at least one processor is further configured to:
determine whether the head-mounted display is level relative to the user's left and right eyes based on the determined orientation of the head-mounted display relative to the interocular axis of the user.
Example 47. The augmented reality system of Example 46, wherein the at least one processor is further configured to, in response to a determination that the head-mounted display is level relative to the user's left and right eyes:
provide, with a left-eye display of the head-mounted display, a left-eye alignment marker; provide, with a right-eye display of the head-mounted display, a right-eye alignment marker; receive user input to adjust at least one of the left-eye and the right-eye alignment markers; and vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input.
Example 48. The augmented reality system of Example 46, wherein to provide the user with feedback based on the determined orientation of the head-mounted display relative to the interocular axis of the user, the at least one processor is configured to:
provide the user with feedback indicating whether the head-mounted display is level relative to the user's left and right eye.
Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, the drawings, and the claims. Neither this summary nor the following detailed description limit the scope of the inventive subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person.
FIG. 2 schematically illustrates an example of a wearable system.
FIG. 3 schematically illustrates example components of a wearable system.
FIG. 4 schematically illustrates an example of a waveguide stack of a wearable device for outputting image information to a user.
FIG. 5 schematically illustrates an example of an eye and schematically illustrates an example coordinate system for determining an eye pose of an eye.
FIG. 6 is a schematic diagram of a wearable system that includes an eye tracking system.
FIG. 7 A is a block diagram of a wearable system that may include an eye tracking system.
FIG. 7 B is a block diagram of a render controller in a wearable system.
FIG. 7 C is a block diagram of a registration observer in a head-mounted display system.
FIG. 8 A is a schematic diagram of an eye showing the eye's corneal sphere.
FIG. 8 B illustrates an example corneal glint detected by an eye-tracking camera.
FIGS. 8 C- 8 E illustrate example stages of locating a user's corneal center with an eye tracking module in a wearable system.
FIGS. 9 A- 9 C illustrate an example normalization of the coordinate system of eye tracking images.
FIGS. 9 D- 9 G illustrate example stages of locating a user's pupil center with an eye tracking module in a wearable system.
FIG. 10 illustrates an example of an eye including the eye's optical and visual axes and the eye's center of rotation.
FIG. 11 is a process flow diagram of an example of a method for using eye tracking in rendering content and providing feedback on registration in a wearable device.
FIGS. 12 A and 12 B illustrate a nominal position of a display element relative to a user's eye and illustrate a coordinate system for describing the positions of the display element and the user's eye relative to one another.
FIG. 13 illustrates an example display screen that may be provided to a user of a head-mounted display system as part of leveling the head-mounted display system on the user's head.
FIG. 14 illustrates an example display screen that may be provided to the user of the head-mounted display system of FIG. 13 after the user has leveled the head-mounted display system on their head.
FIG. 15 illustrates an example display screen that may be provided to a user of a head-mounted display system as part of adjusting vertical alignment between left and right eye displays of the display system.
FIG. 16 illustrates examples of left-eye, right-eye, and fused images that may be provided to the user of FIG. 15 as part of adjusting the vertical alignment between the left and right displays of the display system.
FIG. 17 is a perspective view of a user's hand and a controller including various input devices.
FIG. 18 is a flow diagram illustrative of an embodiment of leveling a head-mounted display system on a user's head and of adjusting a vertical alignment of left and right displays of the display system.
Throughout the drawings, reference numbers may be re-used to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the disclosure.
DETAILED DESCRIPTION
Display systems may include a head-mounted display (HMD) which may display virtual objects such that the objects appear to be located within the user's ambient environment. In some embodiments, the virtual objects may be perceived by the user to be three-dimensional (3D).
The HMD may include two or more distinct displays or a single contiguous display. In any case, different images may be outputted to the left and right eyes of the user. As used herein, references to left-eye and right-eye displays or displays associated with the right and the left eyes of the viewer may be understood to indicate a display device configured to output different images to the left and right eyes of the viewer. Thus, left-eye and right-eye displays may indicate displays that are physically separated, or a single display configured to output different images to each eye. In some embodiments, the different images may be stereoscopic images that present slightly different views of the same object or scene, with the different views being fused together by the human visual system, which can create the perception of depth.
In some situations, the left and right displays may not be properly vertically aligned relative to one another and a user's eyes. Various factors may combine to determine the positions of the left and right eye displays relative to one another and to the user's eyes. For example, the HMD may become deformed or warped over time, leading to left-eye display, right-eye display vertical misalignments. The HMD can be deformed or become warped as the result of numerous factors including thermal cycling, physical shock (e.g., due to being dropped), elastic or inelastic bending from rough handling, aging of materials, etc. In some cases, one of the left or right-eye displays may become rotated and/or vertically translated relative to the other of the left or right-eye displays.
Such vertical misalignment may degrade the user experience and possibly cause user discomfort. Without being limited by theory, it is believed that the human visual system is not accustomed to receiving vertically misaligned light for forming corresponding left and right eye stereoscopic images, where the light propagating towards one eye may be vertically offset from light propagating towards the other eye to form the corresponding stereoscopic images. It will be appreciated that light propagating towards each eye may form stereoscopic images specific to those eyes. As an example, if light from one display is directed horizontally to reach one eye and light from another display is directed downwards to reach the other eye, then these displays may be considered vertically misaligned. It is believed that even relatively small vertical misalignments between the left and right eyes may cause discomfort such as harmful eye strain and headaches.
To reduce viewer discomfort due to vertical misalignment, differences in vertical alignment between a left-eye display and a right-eye display may be reduced. For example, the left-eye display and the right-eye display may be aligned vertically by physically moving those displays. In some cases, such physical movement of the displays is not practical or desirable. Advantageously, in some embodiments, vertical alignment between a left-eye display and a right-eye display is effectively achieved by causing the displays to shift image content vertically up or down on one or both of the displays. In some embodiments, the image shifting may allow the left-eye and right-eye displays to have similar vertical relationships with their corresponding left and right eyes.
Various systems and techniques described herein are at least in part directed to solving problems related to proper vertical alignment of left-eye and right-eye displays, which may allow the viewer to view image content comfortably. In some embodiments, a head-mounted display system may be configured to determine the positions of the eyes of the user (e.g., using a component such as an inward-facing imaging system, which may be an eye tracking system). Determining the position of the eyes may include determining the positions of representative points associated with the eyes such as the respective centers of rotation of the eyes and the position and/or orientation of the user's interocular axis (e.g., an axis extending between corresponding parts of the user's left and right eyes, such as an axis between the center of rotation of the user's left eye and the center of rotation of the user's right eye). The display system may then determine whether the HMD is level with respect to the user's eyes (e.g., level with the user's interocular axis). After leveling, the display system may present the user's left eye with a left-eye alignment marker and the user's right eye with a right-eye alignment marker. Preferably, the alignment markers include horizontal components that are not fused by the human visual system and which make apparent in vertical misalignments between the left-eye and right-eye displays. The display system may then solicit feedback from the user that identifies and/or adjusts vertical alignment differences between the left and right-eye displays (e.g., by asking the user to align the left-eye and right-eye alignment markers). Using such feedback, the display system may adjust content displayed through the system by compensating for any vertical alignment differences identified by the user, thus improving the user's comfort when viewing the HMD.
In some embodiments, the display system includes a plurality of waveguides formed in a stack for outputting image information to a user. An alignment process, as discussed herein, may be conducted for each waveguide. For example, different waveguides may be configured to output different component colors and/or different amounts of wavefront divergence corresponding to different depth planes, and alignments may be performed with each component color (for each of the corresponding component color waveguides) and/or each depth plane (for each waveguide with a corresponding amount of wavefront divergence).
Reference will now be made to the drawings, in which like reference numerals refer to like parts throughout. Unless indicated otherwise, the drawings are schematic and not necessarily drawn to scale.
Examples of 3D Display of a Wearable System
A wearable system (also referred to herein as a head-mounted display system or as an augmented reality (AR) system) may be configured to present 2D or 3D virtual images to a user. The images may be still images, frames of a video, or a video, in combination or the like. At least a portion of the wearable system may be implemented on a wearable device that may present a VR, AR, or MR environment, alone or in combination, for user interaction. The wearable device may be used interchangeably as an AR device (ARD). Further, for the purpose of the present disclosure, the term âARâ is used interchangeably with the term âMRâ.
FIG. 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects, and certain physical objects viewed by a person. In FIG. 1 , an MR scene 100 is depicted wherein a user of an MR technology sees a real-world park- like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120 . In addition to these items, the user of the MR technology also perceives that he âseesâ a robot statue 130 standing upon the real- world platform 120 , and a cartoon- like avatar character 140 flying by which seems to be a personification of a bumble bee, even though these elements do not exist in the real world.
In order for the 3D display to produce a true sensation of depth, and more specifically, a simulated sensation of surface depth, it may be desirable for each point in the display's visual field to generate an accommodative response corresponding to its virtual depth. If the accommodative response to a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues of convergence and stereopsis, the human eye may experience an accommodation conflict, resulting in unstable imaging, harmful eye strain, headaches, and, in the absence of accommodation information, almost a complete lack of surface depth.
VR, AR, and MR experiences may be provided by display systems having displays in which images corresponding to a plurality of depth planes are provided to a viewer. The images may be different for each depth plane (e.g., provide slightly different presentations of a scene or object) and may be separately focused by the viewer's eyes, thereby helping to provide the user with depth cues based on the accommodation of the eye required to bring into focus different image features for the scene located on different depth plane or based on observing different image features on different depth planes being out of focus. As discussed elsewhere herein, such depth cues provide credible perceptions of depth.
FIG. 2 illustrates an example of wearable system 200 which may be configured to provide an AR/VR/MR scene. The wearable system 200 may also be referred to as the AR system 200 . The wearable system 200 includes a display 220 , and various mechanical and electronic modules and systems to support the functioning of display 220 . The display 220 may be coupled to a frame 230 , which is wearable by a user, wearer, or viewer 210 . The display 220 may be positioned in front of the eyes of the user 210 . The display 220 may present AR/VR/MR content to a user. Because the display 220 may be worn on the head of the user 210 , it may also be referred to as a head-mounted display (HMD) and the wearable system 200 , comprising the display 220 , may also be referred to as a head-mounted display system.
In some embodiments, a speaker 240 is coupled to the frame 230 and positioned adjacent the ear canal of the user (in some embodiments, another speaker, not shown, is positioned adjacent the other ear canal of the user to provide for stereo/shapeable sound control). The display 220 may include an audio sensor (e.g., a microphone) 232 for detecting an audio stream from the environment and capture ambient sound. In some embodiments, one or more other audio sensors, not shown, are positioned to provide stereo sound reception. Stereo sound reception may be used to determine the location of a sound source. The wearable system 200 may perform voice or speech recognition on the audio stream.
The wearable system 200 may include an outward-facing imaging system 464 (shown in FIG. 4 ) which observes the world in the environment around the user. The wearable system 200 may also includ
CLAIMS
Claims ( 17 )
What is claimed is:
1. An augmented reality system comprising:
a head-mounted display configured to present virtual content by outputting light to a user, the head-mounted display comprising a left-eye display including a first waveguide stack comprising a plurality of waveguides configured to present virtual content to the user's left eye and a right-eye display including a second waveguide stack comprising a plurality of waveguides configured to present virtual content to the user's right eye; and
at least one processor communicatively coupled to the head-mounted display, the at least one processor configured to:
conduct a plurality of alignment processes for the first waveguide stack and the second waveguide stack, wherein an alignment process is conducted for each waveguide of the first and second waveguide stacks, wherein each alignment process comprises:
providing, with a waveguide of the left-eye display being aligned, a left-eye alignment marker;
providing, with a waveguide of the right-eye display being aligned, a right-eye alignment marker;
receiving user input to adjust at least one of the left-eye and the right-eye alignment markers; and
vertically adjusting image content in the waveguide being aligned of at least one of the left-eye and right-eye displays based on the received user input.
2. The augmented reality system of claim 1 , wherein the left-eye alignment marker comprises a first horizontal line and wherein the right-eye alignment marker comprises a second horizontal line.
3. The augmented reality system of claim 2 , wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines.
4. The augmented reality system of claim 2 , wherein the processor is configured to receive user input to adjust at least one of the left-eye and right-eye alignment markers in the form of user input to raise or lower at least one of the first and second horizontal lines until the first and second horizontal lines are level from the perspective of the user.
5. The augmented reality system of claim 1 , wherein the first waveguide stack is configured to pass light from the world into the left eye of the user and the second waveguide stack is configured to pass light from the world into the right eye of the user.
6. The augmented reality system of claim 1 , wherein the first waveguide stack is configured to pass light from the world into the left eye of the user and the second waveguide stack is configured to pass light from the world into the right eye of the user,
wherein one or more waveguides of each waveguide stack are configured to output light to the user with a different amount of wavefront divergence than one or more other waveguides of that waveguide stack,
wherein different amounts of wavefront divergence are associated with different accommodation by the eye, and
wherein the outputted light with different amounts of wavefront divergence forms virtual objects at different perceived depths away from the user.
7. The augmented reality system of claim 1 , wherein to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, the at least one processor is configured to:
provide, with the left-eye display, a first vertical alignment marker and a first horizontal alignment marker; and
provide, with the right-eye display, a second vertical alignment marker and a second horizontal alignment marker.
8. The augmented reality system of claim 7 , wherein, when viewed by the user, the first and second vertical alignment markers fuse together in the user's vision and the first and second horizontal alignment markers remain unfused in the user's vision.
9. The augmented reality system of claim 7 , wherein to receive user input to adjust at least one of the left-eye and the right-eye alignment markers, the at least one processor is configured to:
receive user input to move at least one of the first and second horizontal alignment markers vertically.
10. The augmented reality system of claim 7 , wherein to receive user input to adjust at least one of the left-eye and the right-eye alignment markers, the at least one processor is configured to:
receive user input to move at least one of the first and second horizontal alignment markers vertically until the first and second horizontal alignment markers are vertically aligned with each other in the user's vision.
11. The augmented reality system of claim 1 , wherein the at least one processor is further configured to:
select a first vertical position at which to present the left-eye alignment marker; and
select a second vertical position at which to present the right-eye alignment marker, and
wherein to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, the at least one processor is configured to:
provide, with the left-eye display, the left-eye alignment marker at the first vertical position; and
provide, with the right-eye display, the right-eye alignment marker at the second vertical position.
12. The augmented reality system of claim 11 , wherein to select the first vertical position at which to present the left-eye alignment marker and select the second vertical position at which to present the right-eye alignment marker, the at least one processor is configured to:
pseudo- or quasi-randomly select the first vertical position at which to present the left-eye alignment marker; and
pseudo- or quasi-randomly select the second vertical position at which to present the right-eye alignment marker.
13. The augmented reality system of claim 11 , wherein to vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input, the at least one processor is configured to:
vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input, the first vertical position selected, and the second vertical position selected.
14. The augmented reality system of claim 1 , wherein to vertically adjust image content in the at least one of the left-eye and right-eye displays based on the received user input, the at least one processor is configured to:
adjust one or more extrinsic parameters of at least one of a first render camera that is associated with the left-eye display and a second render camera that is associated with the right-eye display.
15. The augmented reality system of claim 14 , wherein the one or more extrinsic parameters include at least one of a position and an orientation.
16. The augmented reality system of claim 1 , further comprising an imaging system configured to capture images of eyes of the user, and
wherein the at least one processor is communicatively coupled to the imaging system, the at least one processor further configured to:
determine whether the head-mounted display is level relative to the user's left and right eyes based at least in part on one or more images captured by the imaging system.
17. The augmented reality system of claim 16 , wherein the at least one processor is configured to provide, with the left-eye display and the right-eye display, the left-eye alignment marker and the right-eye alignment marker, respectively, in response to a determination that the head-mounted display is level relative to the user's left and right eyes.
US18/486,848
2018-07-24
2023-10-13
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Active
US12105875B2
( en )
Priority Applications (2)
Application Number
Priority Date
Filing Date
Title
US18/486,848
US12105875B2
( en )
2018-07-24
2023-10-13
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US18/818,343
US12429946B2
( en )
2018-07-24
2024-08-28
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Applications Claiming Priority (5)
Application Number
Priority Date
Filing Date
Title
US201862702865P
2018-07-24
2018-07-24
PCT/US2019/043302
WO2020023672A1
( en )
2018-07-24
2019-07-24
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US202117258083A
2021-01-05
2021-01-05
US17/890,998
US11822718B2
( en )
2018-07-24
2022-08-18
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US18/486,848
US12105875B2
( en )
2018-07-24
2023-10-13
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Related Parent Applications (1)
Application Number
Title
Priority Date
Filing Date
US17/890,998
Continuation
US11822718B2
( en )
2018-07-24
2022-08-18
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Related Child Applications (1)
Application Number
Title
Priority Date
Filing Date
US18/818,343
Continuation
US12429946B2
( en )
2018-07-24
2024-08-28
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Publications (2)
Publication Number
Publication Date
US20240036645A1
US20240036645A1 ( en )
2024-02-01
US12105875B2
true
US12105875B2 ( en )
2024-10-01
Family
ID=69181195
Family Applications (4)
Application Number
Title
Priority Date
Filing Date
US17/258,083
Active
US11422620B2
( en )
2018-07-24
2019-07-24
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US17/890,998
Active
US11822718B2
( en )
2018-07-24
2022-08-18
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US18/486,848
Active
US12105875B2
( en )
2018-07-24
2023-10-13
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US18/818,343
Active
US12429946B2
( en )
2018-07-24
2024-08-28
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Family Applications Before (2)
Application Number
Title
Priority Date
Filing Date
US17/258,083
Active
US11422620B2
( en )
2018-07-24
2019-07-24
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US17/890,998
Active
US11822718B2
( en )
2018-07-24
2022-08-18
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Family Applications After (1)
Application Number
Title
Priority Date
Filing Date
US18/818,343
Active
US12429946B2
( en )
2018-07-24
2024-08-28
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
Country Status (5)
Country
Link
US
( 4 )
US11422620B2
( en )
EP
( 1 )
EP3827585A4
( en )
JP
( 3 )
JP7456995B2
( en )
CN
( 1 )
CN112655204A
( en )
WO
( 1 )
WO2020023672A1
( en )
Cited By (2)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20240419243A1
( en )
*
2018-07-24
2024-12-19
Magic Leap, Inc.
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US12524186B1
( en )
*
2025-03-17
2026-01-13
Apple Inc.
Visual output control for head-mountable devices
Families Citing this family (18)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11181977B2
( en )
*
2017-11-17
2021-11-23
Dolby Laboratories Licensing Corporation
Slippage compensation in eye tracking
GB2588774B
( en )
*
2019-11-05
2024-05-15
Arspectra Sarl
Augmented reality headset for medical imaging
CN113010125B
( en )
2019-12-20
2024-03-19
ææ¯è¡ä»½å ¬å¸
Method, computer program product, and binocular headset controller
CA3176950A1
( en )
*
2020-04-30
2021-11-04
Neurotologix Pty Ltd
Method, system and apparatus for investigating or assessing eye or pupil movement
CN116033864A
( en )
2020-07-15
2023-04-28
å¥è·å ¬å¸
Eye tracking using a non-spherical cornea model
GB2597533B
( en )
2020-07-28
2022-11-23
Sony Interactive Entertainment Inc
Gaze tracking system and method
US12446769B2
( en )
2020-10-28
2025-10-21
University Of Miami
Active calibration of head-mounted displays
JP2023550699A
( en )
*
2020-10-28
2023-12-05
ã¦ããã¼ã·ãã£ã¼ ãªã ãã¤ã¢ã
System and method for visual field testing in head-mounted displays
WO2023031933A1
( en )
*
2021-08-31
2023-03-09
Spectralics Ltd.
Dynamic parallel monocular projection
US12277725B2
( en )
*
2021-10-20
2025-04-15
Microsoft Technology Licensing, Llc
Enhanced eye tracking systems utilizing joint biological and hardware estimations
US12513280B1
( en )
2022-09-22
2025-12-30
Apple Inc.
Perceptual measure of end-to-end splay
US12327501B2
( en )
2022-12-22
2025-06-10
Microsoft Technology Licensing, Llc
Systems and methods for facilitating display misalignment correction based on vergence angle
WO2024155811A1
( en )
*
2023-01-19
2024-07-25
Google Llc
Determining a change to a frame for a head mounted wearable device
US12242063B2
( en )
*
2023-03-20
2025-03-04
Microsoft Technology Licensing, Llc
Vertical misalignment correction in binocular display systems
JP2024146552A
( en )
*
2023-03-31
2024-10-15
ãã¤ãã³æ ªå¼ä¼ç¤¾
Image processing device, image processing method, and program
US12627787B2
( en )
2023-04-03
2026-05-12
Microsoft Technology Licensing, Llc
Distance-based vertical misalignment correction
US12506856B2
( en )
2023-11-20
2025-12-23
Microsoft Technology Licensing, Llc
Changing FOV and resolution for calibrating scanning display system alignment
US20250260798A1
( en )
*
2024-02-14
2025-08-14
Microsoft Technology Licensing, Llc
Calibrating sensor alignment offset for stereoscopic sensor systems
Citations (45)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20020028045A1
( en )
*
1998-10-09
2002-03-07
Tetsuzo Yoshimura
Optical coupling structures and the fabrication processes
US6850221B1
( en )
1995-09-05
2005-02-01
Interlink Electronics, Inc.
Trigger operated electronic device
US20060028436A1
( en )
1992-03-05
2006-02-09
Armstrong Brad A
Image controller
US20070081123A1
( en )
2005-10-07
2007-04-12
Lewis Scott W
Digital eyewear
JP2011221235A
( en )
2010-04-08
2011-11-04
Sony Corp
Optical position adjusting method in head-mounted display
US20120127062A1
( en )
2010-11-18
2012-05-24
Avi Bar-Zeev
Automatic focus improvement for augmented reality displays
US20120162549A1
( en )
2010-12-24
2012-06-28
Chunyu Gao
Ergonomic Head Mounted Display Device And Optical System
WO2012172719A1
( en )
2011-06-16
2012-12-20
ããã½ããã¯æ ªå¼ä¼ç¤¾
Head-mounted display and misalignment correction method thereof
US20130082922A1
( en )
2011-09-29
2013-04-04
Samuel A. Miller
Tactile glove for human-computer interaction
US20130117377A1
( en )
2011-10-28
2013-05-09
Samuel A. Miller
System and Method for Augmented and Virtual Reality
US20130125027A1
( en )
2011-05-06
2013-05-16
Magic Leap, Inc.
Massive simultaneous remote digital presence world
US20130208234A1
( en )
2005-10-07
2013-08-15
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
US20130242262A1
( en )
2005-10-07
2013-09-19
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
WO2014017348A1
( en )
2012-07-24
2014-01-30
ã½ãã¼æ ªå¼ä¼ç¤¾
Image display device and image display method
US20140071539A1
( en )
2012-09-11
2014-03-13
Magic Leap, Inc.
Ergonomic head mounted display device and optical system
US20140177023A1
( en )
2012-04-05
2014-06-26
Augmented Vision Inc.
Apparatus for optical see-through head mounted display with mutual occlusion and opaqueness control capability
US20140240842A1
( en )
*
2013-02-22
2014-08-28
Ian Nguyen
Alignment-insensitive image input coupling
US20140267420A1
( en )
2013-03-15
2014-09-18
Magic Leap, Inc.
Display system and method
US20140375540A1
( en )
*
2013-06-24
2014-12-25
Nathan Ackerman
System for optimal eye fit of headset display device
US20150016777A1
( en )
2012-06-11
2015-01-15
Magic Leap, Inc.
Planar waveguide apparatus with diffraction element(s) and system employing same
US20150103306A1
( en )
2013-10-16
2015-04-16
Magic Leap, Inc.
Virtual or augmented reality headsets having adjustable interpupillary distance
US20150178939A1
( en )
2013-11-27
2015-06-25
Magic Leap, Inc.
Virtual and augmented reality systems and methods
US20150193980A1
( en )
2014-01-06
2015-07-09
Qualcomm Incorporated
Calibration of augmented reality (ar) optical see-through display using shape-based alignment
US20150205126A1
( en )
2013-11-27
2015-07-23
Magic Leap, Inc.
Virtual and augmented reality systems and methods
US9094677B1
( en )
2013-07-25
2015-07-28
Google Inc.
Head mounted display device with automated positioning
US9116545B1
( en )
*
2012-03-21
2015-08-25
Hayes Solos Raffle
Input detection
US20150326570A1
( en )
2014-05-09
2015-11-12
Eyefluence, Inc.
Systems and methods for discerning eye signals and continuous biometric identification
US20150346495A1
( en )
2014-05-30
2015-12-03
Magic Leap, Inc.
Methods and system for creating focal planes in virtual and augmented reality
US20160011419A1
( en )
2010-12-24
2016-01-14
Magic Leap, Inc.
Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
US20160026253A1
( en )
2014-03-11
2016-01-28
Magic Leap, Inc.
Methods and systems for creating virtual and augmented reality
US20160073092A1
( en )
2014-09-10
2016-03-10
Etron Technology, Inc.
Image calibration system and calibration method of a stereo camera
US20160187651A1
( en )
*
2014-03-28
2016-06-30
Osterhout Group, Inc.
Safety for a vehicle operator with an hmd
US20160353094A1
( en )
2015-05-29
2016-12-01
Seeing Machines Limited
Calibration of a head mounted eye tracking system
US20170000683A1
( en )
*
2015-03-16
2017-01-05
Magic Leap, Inc.
Methods and systems for modifying eye convergence for diagnosing and treating conditions including strabismus and/or amblyopia
US20170124928A1
( en )
2015-11-04
2017-05-04
Magic Leap, Inc.
Dynamic display calibration based on eye-tracking
US20170293145A1
( en )
2016-04-08
2017-10-12
Magic Leap, Inc.
Augmented reality systems and methods with variable focus lens elements
US20180088328A1
( en )
*
2016-09-28
2018-03-29
Intel Corporation
Replaceable optical element for near eye display
WO2018067357A2
( en )
2016-10-05
2018-04-12
Magic Leap, Inc.
Periocular test for mixed reality calibration
US20190220662A1
( en )
*
2018-01-12
2019-07-18
Microsoft Technology Licensing, Llc
Unguided passive biometric enrollment
WO2020023672A1
( en )
2018-07-24
2020-01-30
Magic Leap, Inc.
Display systems and methods for determining vertical alignment between left and right displays and a user's eyes
US10650785B1
( en )
*
2019-02-21
2020-05-12
Microsoft Technology Licensing, Llc
Color management of display device
US10678059B1
( en )
*
2019-04-02
2020-06-09
Microsoft Technology Licensing, Llc
Concealed shock protector for head-mounted device display
US20200209628A1
( en )
*
2018-12-27
2020-07-02
Facebook Technologies, Llc
Head mounted display calibration using portable docking station with calibration target
US20200310117A1
( en )
*
2019-03-27
2020-10-01
Lenovo (Singapore) Pte. Ltd.
Adjusting display settings of a head-mounted display
US20220082739A1
( en )
*
2020-09-17
2022-03-17
Facebook Technologies, Llc
Techniques for manufacturing variable etch depth gratings using gray-tone lithography
2019
2019-07-24
CN
CN201980058118.0A
patent/CN112655204A/en
active
Pending
2019-07-24
EP
EP19840270.3A
patent/EP3827585A4/en
active
Pending
2019-07-24
JP
JP2021502899A
patent/JP7456995B2/en
active
Active
2019-07-24
WO
PCT/US2019/043302
patent/WO2020023672A1/en
not_active
Ceased
2019-07-24
US
US17/258,083
patent/US11422620B2/en
active
Active
2022
2022-08-18
US
US17/890,998
patent/US11822718B2/en
active
Active
2023
2023-10-13
US
US18/486,848
patent/US12105875B2/en
active
Active
2024
2024-03-14
JP
JP2024039794A
patent/JP7699254B2/en
active
Active
2024-08-28
US
US18/818,343
patent/US12429946B2/en
active
Active
2025
2025-05-21
JP
JP2025084768A
patent/JP2025113382A/en
active
Pending
Patent Citations (50)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US20060028436A1
( en )
1992-03-05
2006-02-09
Armstrong Brad A
Image controller
US6850221B1
( en )
1995-09-05
2005-02-01
Interlink Electronics, Inc.
Trigger operated electronic device
US20020028045A1
( en )
*
1998-10-09
2002-03-07
Tetsuzo Yoshimura
Optical coupling structures and the fabrication processes
US20130208234A1
( en )
2005-10-07
2013-08-15
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
US20070081123A1
( en )
2005-10-07
2007-04-12
Lewis Scott W
Digital eyewear
US20130242262A1
( en )
2005-10-07
2013-09-19
Percept Technologies Inc.
Enhanced optical and perceptual digital eyewear
JP2011221235A
( en )
2010-04-08
2011-11-04
Sony Corp
Optical position adjusting method in head-mounted display
US20120127062A1
( en )
2010-11-18
2012-05-24
Avi Bar-Zeev
Automatic focus improvement for augmented reality displays
US20120162549A1
( en )
2010-12-24
2012-06-28
Chunyu Gao
Ergonomic Head Mounted Display Device And Optical System
US20160011419A1
( en )
2010-12-24
2016-01-14
Magic Leap, Inc.
Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
US20130125027A1
( en )
2011-05-06
2013-05-16
Magic Leap, Inc.
Massive simultaneous remote digital presence world
US20130235169A1
( en )
2011-06-16
2013-09-12
Panasonic Corporation
Head-mounted display and position gap adjustment method
WO2012172719A1
( en )
2011-06-16
2012-12-20
ããã½ããã¯æ ªå¼ä¼ç¤¾
Head-mounted display and misalignment correction method thereof
US20130082922A1
( en )
2011-09-29
2013-04-04
Samuel A. Miller
Tactile glove for human-computer interaction
US20130117377A1
( en )
2011-10-28
2013-05-09
Samuel A. Miller
System and Method for Augmented and Virtual Reality
US9116545B1
( en )
*
2012-03-21
2015-08-25
Hayes Solos Raffle
Input detection
US20140177023A1
( en )
2012-04-05
2014-06-26
Augmented Vision Inc.
Apparatus for optical see-through head mounted display with mutual occlusion and opaqueness control capability
US20140218468A1
( en )
2012-04-05
2014-08-07
Augmented Vision Inc.
Wide-field of view (fov) imaging devices with active foveation capability
US20150016777A1
( en )
2012-06-11
2015-01-15
Magic Leap, Inc.
Planar waveguide apparatus with diffraction element(s) and system employing same
US20150309263A2
( en )
2012-06-11
2015-10-29
Magic Leap, Inc.
Planar waveguide apparatus with diffraction element(s) and system employing same
WO2014017348A1
( en )
2012-07-24
2014-01-30
ã½ãã¼æ ªå¼ä¼ç¤¾
Image display device and image display method
US20140071539A1
( en )
2012-09-11
2014-03-13
Magic Leap, Inc.
Ergonomic head mounted display device and optical system
US20140240842A1
( en )
*
2013-02-22
2014-08-28
Ian Nguyen
Alignment-insensitive image input coupling
US20140267420A1
( en )
2013-03-15
2014-09-18
Magic Leap, Inc.
Display system and method
US20140375540A1
( en )
*
2013-06-24
2014-12-25
Nathan Ackerman
System for optimal eye fit of headset display device
US9094677B1
( en )
2013-07-25
2015-07-28
Google Inc.
Head mounted display device with automated positioning
US20150103306A1
( en )
2013-10-16
2015-04-16
Magic Leap, Inc.
Virtual or augmented reality headsets having adjustable interpupillary distance
US20150205126A1
( en )
2013-11-27
2015-07-23
Magic Leap, Inc.
Virtual and augmented reality systems and methods
US20150178939A1
( en )
2013-11-27
2015-06-25
Magic Leap, Inc.
Virtual and augmented reality systems and methods
US20150193980A1
( en )
2014-01-06
2015-07-09
Qualcomm Incorporated
Calibration of augmented reality (ar) optical see-through display using shape-based alignment
US20160026253A1
( en )
2014-03-11
2016-01-28
Magic Leap, Inc.
Methods and systems for creating virtual and augmented reality
US20160187651A1
( en )
*
2014-03-28
2016-06-30
Osterhout Group, Inc.
Safety for a vehicle operator with an hmd
US20150326570A1
( en )
2014-05-09
2015-11-12
Eyefluence, Inc.
Systems and methods for discerning eye signals and continuous biometric identification
US20150346495A1
( en )
2014-05-30
2015-12-03
Magic Leap, Inc.
Methods and system for creating focal planes in virtual and augmented reality
US20160073092A1
( en )
2014-09-10
2016-03-10
Etron Technology, Inc.
Image calibration system and calibration method of a stereo camera