ABSTRACT
Abstract
Described herein are embodiments of systems and a method to identify orthostatic hypotension and postural-orthostatic tachycardia syndrome. One system includes a head-mounted device configured to measure photoplethysmographic signal (PPG signal) at a region on a user's head, and a head-mounted camera configured to capture images indicative of the user's posture. Additionally, the system includes a computer that calculates systolic and diastolic blood pressure values based on the PPG signal, and identifies orthostatic hypotension based on a drop of systolic blood pressure below a first threshold, and/or a drop of diastolic blood pressure below a second threshold, within a predetermined duration from a transition in the posture from supine to sitting posture, or from sitting to standing posture.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a Continuation-In-Part of U.S. application Ser. No. 16/375,841, filed Apr. 4, 2019. U.S. Ser. No. 16/375,841 is a Continuation-In-Part of U.S. application Ser. No. 16/156,493, filed Oct. 10, 2018. U.S. Ser. No. 16/156,493, is a Continuation-In-Part of U.S. application Ser. No. 15/635,178, filed Jun. 27, 2017, now U.S. Pat. No. 10,136,856, which claims priority to U.S. Provisional Patent Application No. 62/354,833, filed Jun. 27, 2016, and U.S. Provisional Patent Application No. 62/372,063, filed Aug. 8, 2016.
U.S. Ser. No. 16/156,493 is also a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016, which claims priority to U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015, and U.S. Provisional Patent Application No. 62/236,868, filed Oct. 3, 2015.
U.S. Ser. No. 16/156,493 is also a Continuation-In-Part of U.S. application Ser. No. 15/832,855, filed Dec. 6, 2017, now U.S. Pat. No. 10,130,308, which claims priority to U.S. Provisional Patent Application No. 62/456,105, filed Feb. 7, 2017, and U.S. Provisional Patent Application No. 62/480,496, filed Apr. 2, 2017, and U.S. Provisional Patent Application No. 62/566,572, filed Oct. 2, 2017. U.S. Ser. No. 15/832,855 is a Continuation-In-Part of U.S. application Ser. No. 15/182,592, filed Jun. 14, 2016, now U.S. Pat. No. 10,165,949, a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016, a Continuation-In-Part of U.S. application Ser. No. 15/284,528, filed Oct. 3, 2016, now U.S. Pat. No. 10,113,913, a Continuation-In-Part of U.S. application Ser. No. 15/635,178, filed Jun. 27, 2017, now U.S. Pat. No. 10,136,856, and a Continuation-In-Part of U.S. application Ser. No. 15/722,434, filed Oct. 2, 2017.
U.S. Ser. No. 15/832,855 is a Continuation-In-Part of U.S. application Ser. No. 15/182,566, filed Jun. 14, 2016, now U.S. Pat. No. 9,867,546, which claims priority to U.S. Provisional Patent Application No. 62/175,319, filed Jun. 14, 2015, and U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015.
U.S. Ser. No. 15/182,592 claims priority to U.S. Provisional Patent Application No. 62/175,319, filed Jun. 14, 2015, and U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015.
U.S. Ser. No. 15/284,528 claims priority to U.S. Provisional Patent Application No. 62/236,868, filed Oct. 3, 2015, and U.S. Provisional Patent Application No. 62/354,833, filed Jun. 27, 2016, and U.S. Provisional Patent Application No. 62/372,063, filed Aug. 8, 2016.
U.S. Ser. No. 16/156,493 is also a Continuation-In-Part of U.S. application Ser. No. 15/833,115, filed Dec. 6, 2017, now U.S. Pat. No. 10,130,261. U.S. Ser. No. 15/833,115 is a Continuation-In-Part of U.S. application Ser. No. 15/182,592, a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016, a Continuation-In-Part of U.S. application Ser. No. 15/284,528, a Continuation-In-Part of U.S. application Ser. No. 15/635,178, and a Continuation-In-Part of U.S. application Ser. No. 15/722,434, filed Oct. 2, 2017.
This application is also a Continuation-In-Part of U.S. application Ser. No. 16/147,695, filed Sep. 29, 2018. U.S. Ser. No. 16/147,695 is a Continuation of U.S. application Ser. No. 15/182,592, filed Jun. 14, 2016, which claims priority to U.S. Provisional Patent Application No. 62/175,319, filed Jun. 14, 2015, and U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015.
ACKNOWLEDGMENTS
Gil Thieberger would like to thank his holy and beloved teacher, Lama Dvora-hla, for her extraordinary teachings and manifestation of wisdom, love, compassion and morality, and for her endless efforts, support, and skills in guiding him and others on their paths to freedom and ultimate happiness. Gil would also like to thank his beloved parents for raising him exactly as they did.
BACKGROUND
Continuous monitoring of various physiological signals, such as blood pressure, tissue perfusion, and cardiac output, can provide important data for physicians when monitoring and treating various conditions. For example, keeping track of conditions such as elevated blood pressure is vital in the prevention and treatment of various blood-pressure-related diseases. However, continuous monitoring of some physiological signals with existing commonly used technology (e.g., cuff-based devices) can be difficult, uncomfortable, and impractical to perform in real-world settings (e.g., at work, while commuting, etc.).
Some current monitoring technologies are based on measuring a photoplethysmographic signal (PPG signal), and utilizing the PPG signal to calculate physiological signals, such as blood pressure. While such technologies may be more comfortable and portable than the cuff-based blood pressure monitors, they tend to be effected by the posture of the monitored person. For example, the posture may introduce artifacts that may unduly influence the shape of a cardiac waveform, which can lead to inaccurate calculations of physiological signals.
Thus, there is a need for systems that enable comfortable and practical monitoring of physiological signals, such as blood pressure, tissue perfusion, and cardiac output, in a manner that adjusts for various postures that may be assumed.
SUMMARY
Some aspects of this disclosure include various head-mounted systems (e.g., smartglasses) that measure physiological signals, such as blood pressure, tissue perfusion, and cardiac output by analyzing a photoplethysmographic signal (PPG signal) at a region on the user's head, which is measured using a head-mounted device. Additionally, the systems utilize a head-mounted camera that captures images indicative of posture of the user. The systems further include a computer that calculates a value of a physiological signal of the user based on the PPG signal and the posture of the user (which is identifiable in the images), in a manner that adjusts for effects of the posture of the user on the measurements.
BRIEF DESCRIPTION OF THE DRAWINGS
The embodiments are herein described by way of example only, with reference to the following drawings:
FIG. 1 a and FIG. 1 b illustrate various inward-facing head-mounted cameras coupled to an eyeglasses frame;
FIG. 2 illustrates inward-facing head-mounted cameras coupled to an augmented reality device;
FIG. 3 illustrates head-mounted cameras coupled to a virtual reality device;
FIG. 4 illustrates a side view of head-mounted cameras coupled to an augmented reality device;
FIG. 5 illustrates a side view of head-mounted cameras coupled to a sunglasses frame;
FIG. 6 , FIG. 7 , FIG. 8 and FIG. 9 illustrate head-mounted systems (HMSs) configured to measure various ROIs relevant to some of the embodiments describes herein;
FIG. 10 , FIG. 11 , FIG. 12 and FIG. 13 illustrate various embodiments of systems that include inward-facing head-mounted cameras having multi-pixel sensors (FPA sensors);
FIG. 14 a , FIG. 14 b , and FIG. 14 c illustrate embodiments of two right and left clip-on devices that are configured to attached/detached from an eyeglasses frame;
FIG. 15 a and FIG. 15 b illustrate an embodiment of a clip-on device that includes inward-facing head-mounted cameras pointed at the lower part of the face and the forehead;
FIG. 16 a and FIG. 16 b illustrate an embodiment of a single-unit clip-on device that is configured to be attached behind an eyeglasses frame;
FIG. 17 a and FIG. 17 b illustrate embodiments of right and left clip-on devices that are configured to be attached behind an eyeglasses frame;
FIG. 18 illustrates embodiments of right and left clip-on devices, which are configured to be attached/detached from an eyeglasses frame, and have protruding arms to hold inward-facing head-mounted cameras;
FIG. 19 a is a schematic illustration of an inward-facing head-mounted camera embedded in an eyeglasses frame, which utilizes the Scheimpflug principle;
FIG. 19 b is a schematic illustration of a camera that is able to change the relative tilt between its lens and sensor planes according to the Scheimpflug principle;
FIG. 20 illustrates an embodiment of a system configured to calculate a physiological signal;
FIG. 21 illustrates an embodiment of a system configured to calculate blood pressure that includes at least two inward-facing HCAMs;
FIG. 22 illustrates one embodiment of a system configured to calculate blood pressure, which includes inward-facing HCAMs as well as outward-facing HCAMs;
FIG. 23 and FIG. 24 illustrate brainwave headsets having at least two inward facing cameras that capture the user's facial expressions;
FIG. 25 illustrates an HMD having head mounted cameras able to capture both the user's face and the user's back;
FIG. 26 illustrates a HMD having head mounted cameras around the head;
FIG. 27 illustrates a HMD having head mounted cameras able to capture portions of the user's torso, hands, and legs;
FIG. 28 illustrates a HMD having head mounted a camera able to capture the user's shoulder;
FIG. 29 , FIG. 30 , FIG. 31 , and FIG. 32 illustrate HMDs having head mounted cameras able to capture both the user's face and the user's back;
FIG. 33 and FIG. 34 illustrate HMDs having head mounted cameras able to capture both the user's facial expressions and hand gestures with the same camera; and
FIG. 35 a and FIG. 35 b are schematic illustrations of possible embodiments for computers.
DETAILED DESCRIPTION
âVisible-light cameraâ refers to a non-contact device designed to detect at least some of the visible spectrum, such as a video camera with optical lenses and CMOS or CCD sensor. A âthermal cameraâ refers herein to a non-contact device that measures electromagnetic radiation having wavelengths longer than 2500 nanometer (nm) and does not touch its region of interest (ROI). A thermal camera may include one sensing element (pixel), or multiple sensing elements that are also referred to herein as âsensing pixelsâ, âpixelsâ, and/or focal-plane array (FPA). A thermal camera may be based on an uncooled thermal sensor, such as a thermopile sensor, a microbolometer sensor (where microbolometer refers to any type of a bolometer sensor and its equivalents), a pyroelectric sensor, or a ferroelectric sensor.
A reference to a âcameraâ herein may relate to various types of devices. In one example, a camera is a visible-light camera. In another example, a camera may capture light in the ultra-violet range. And in another example, a camera may capture near infrared radiation (e.g., wavelengths between 750 and 2000 nm).
In some embodiments, a device, such as a camera, may be positioned such that it occludes an ROI on the user's face, while in other embodiments, the device may be positioned such that it does not occlude the ROI. Sentences in the form of âthe system/camera does not occlude the ROIâ indicate that the ROI can be observed by a third person located in front of the user and looking at the ROI, such as illustrated by all the ROIs in FIG. 7 and FIG. 11 . Sentences in the form of âthe system/camera occludes the ROIâ indicate that some of the ROIs cannot be observed directly by that third person, such as ROIs
19 and 37 that are occluded by the lenses in FIG. 1 a , and ROIs
97 and 102 that are occluded by cameras
91 and 96 , respectively, in FIG. 9 .
Although some of the disclosed embodiments can use occluding cameras successfully, in certain scenarios, such as when using a head-mounted system (HMS) on a daily basis and/or in a normal day-to-day setting, using cameras that do not occlude their ROIs on the face may provide one or more advantages to the user, to the HMS, and/or to the cameras, which may relate to one or more of the following: esthetics, better ventilation of the face, reduced weight, simplicity to wear, ability to operate without active illumination, and reduced likelihood to being tarnished.
The term âinward-facing head-mounted cameraâ refers to a camera configured to be worn on a user's head and to remain pointed at its ROI, which is on the user's face, also when the user's head makes angular and lateral movements (such as movements with an angular velocity above 0.1 rad/sec, above 0.5 rad/sec, and/or above 1 rad/sec). A head-mounted camera (which may be inward-facing and/or outward-facing) may be physically coupled to a frame worn on the user's head, may be attached to eyeglass using a clip-on mechanism (configured to be attached to and detached from the eyeglasses), or may be mounted to the user's head using any other known device that keeps the camera in a fixed position relative to the user's head also when the head moves. Sentences in the form of âcamera physically coupled to the frameâ mean that the camera moves with the frame, such as when the camera is fixed to
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a Continuation-In-Part of U.S. application Ser. No. 16/375,841, filed Apr. 4, 2019. U.S. Ser. No. 16/375,841 is a Continuation-In-Part of U.S. application Ser. No. 16/156,493, filed Oct. 10, 2018. U.S. Ser. No. 16/156,493, is a Continuation-In-Part of U.S. application Ser. No. 15/635,178, filed Jun. 27, 2017, now U.S. Pat. No. 10,136,856, which claims priority to U.S. Provisional Patent Application No. 62/354,833, filed Jun. 27, 2016, and U.S. Provisional Patent Application No. 62/372,063, filed Aug. 8, 2016.
U.S. Ser. No. 16/156,493 is also a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016, which claims priority to U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015, and U.S. Provisional Patent Application No. 62/236,868, filed Oct. 3, 2015.
U.S. Ser. No. 16/156,493 is also a Continuation-In-Part of U.S. application Ser. No. 15/832,855, filed Dec. 6, 2017, now U.S. Pat. No. 10,130,308, which claims priority to U.S. Provisional Patent Application No. 62/456,105, filed Feb. 7, 2017, and U.S. Provisional Patent Application No. 62/480,496, filed Apr. 2, 2017, and U.S. Provisional Patent Application No. 62/566,572, filed Oct. 2, 2017. U.S. Ser. No. 15/832,855 is a Continuation-In-Part of U.S. application Ser. No. 15/182,592, filed Jun. 14, 2016, now U.S. Pat. No. 10,165,949, a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016, a Continuation-In-Part of U.S. application Ser. No. 15/284,528, filed Oct. 3, 2016, now U.S. Pat. No. 10,113,913, a Continuation-In-Part of U.S. application Ser. No. 15/635,178, filed Jun. 27, 2017, now U.S. Pat. No. 10,136,856, and a Continuation-In-Part of U.S. application Ser. No. 15/722,434, filed Oct. 2, 2017.
U.S. Ser. No. 15/832,855 is a Continuation-In-Part of U.S. application Ser. No. 15/182,566, filed Jun. 14, 2016, now U.S. Pat. No. 9,867,546, which claims priority to U.S. Provisional Patent Application No. 62/175,319, filed Jun. 14, 2015, and U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015.
U.S. Ser. No. 15/182,592 claims priority to U.S. Provisional Patent Application No. 62/175,319, filed Jun. 14, 2015, and U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015.
U.S. Ser. No. 15/284,528 claims priority to U.S. Provisional Patent Application No. 62/236,868, filed Oct. 3, 2015, and U.S. Provisional Patent Application No. 62/354,833, filed Jun. 27, 2016, and U.S. Provisional Patent Application No. 62/372,063, filed Aug. 8, 2016.
U.S. Ser. No. 16/156,493 is also a Continuation-In-Part of U.S. application Ser. No. 15/833,115, filed Dec. 6, 2017, now U.S. Pat. No. 10,130,261. U.S. Ser. No. 15/833,115 is a Continuation-In-Part of U.S. application Ser. No. 15/182,592, a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016, a Continuation-In-Part of U.S. application Ser. No. 15/284,528, a Continuation-In-Part of U.S. application Ser. No. 15/635,178, and a Continuation-In-Part of U.S. application Ser. No. 15/722,434, filed Oct. 2, 2017.
This application is also a Continuation-In-Part of U.S. application Ser. No. 16/147,695, filed Sep. 29, 2018. U.S. Ser. No. 16/147,695 is a Continuation of U.S. application Ser. No. 15/182,592, filed Jun. 14, 2016, which claims priority to U.S. Provisional Patent Application No. 62/175,319, filed Jun. 14, 2015, and U.S. Provisional Patent Application No. 62/202,808, filed Aug. 8, 2015.
ACKNOWLEDGMENTS
Gil Thieberger would like to thank his holy and beloved teacher, Lama Dvora-hla, for her extraordinary teachings and manifestation of wisdom, love, compassion and morality, and for her endless efforts, support, and skills in guiding him and others on their paths to freedom and ultimate happiness. Gil would also like to thank his beloved parents for raising him exactly as they did.
BACKGROUND
Continuous monitoring of various physiological signals, such as blood pressure, tissue perfusion, and cardiac output, can provide important data for physicians when monitoring and treating various conditions. For example, keeping track of conditions such as elevated blood pressure is vital in the prevention and treatment of various blood-pressure-related diseases. However, continuous monitoring of some physiological signals with existing commonly used technology (e.g., cuff-based devices) can be difficult, uncomfortable, and impractical to perform in real-world settings (e.g., at work, while commuting, etc.).
Some current monitoring technologies are based on measuring a photoplethysmographic signal (PPG signal), and utilizing the PPG signal to calculate physiological signals, such as blood pressure. While such technologies may be more comfortable and portable than the cuff-based blood pressure monitors, they tend to be effected by the posture of the monitored person. For example, the posture may introduce artifacts that may unduly influence the shape of a cardiac waveform, which can lead to inaccurate calculations of physiological signals.
Thus, there is a need for systems that enable comfortable and practical monitoring of physiological signals, such as blood pressure, tissue perfusion, and cardiac output, in a manner that adjusts for various postures that may be assumed.
SUMMARY
Some aspects of this disclosure include various head-mounted systems (e.g., smartglasses) that measure physiological signals, such as blood pressure, tissue perfusion, and cardiac output by analyzing a photoplethysmographic signal (PPG signal) at a region on the user's head, which is measured using a head-mounted device. Additionally, the systems utilize a head-mounted camera that captures images indicative of posture of the user. The systems further include a computer that calculates a value of a physiological signal of the user based on the PPG signal and the posture of the user (which is identifiable in the images), in a manner that adjusts for effects of the posture of the user on the measurements.
BRIEF DESCRIPTION OF THE DRAWINGS
The embodiments are herein described by way of example only, with reference to the following drawings:
FIG. 1 a and FIG. 1 b illustrate various inward-facing head-mounted cameras coupled to an eyeglasses frame;
FIG. 2 illustrates inward-facing head-mounted cameras coupled to an augmented reality device;
FIG. 3 illustrates head-mounted cameras coupled to a virtual reality device;
FIG. 4 illustrates a side view of head-mounted cameras coupled to an augmented reality device;
FIG. 5 illustrates a side view of head-mounted cameras coupled to a sunglasses frame;
FIG. 6 , FIG. 7 , FIG. 8 and FIG. 9 illustrate head-mounted systems (HMSs) configured to measure various ROIs relevant to some of the embodiments describes herein;
FIG. 10 , FIG. 11 , FIG. 12 and FIG. 13 illustrate various embodiments of systems that include inward-facing head-mounted cameras having multi-pixel sensors (FPA sensors);
FIG. 14 a , FIG. 14 b , and FIG. 14 c illustrate embodiments of two right and left clip-on devices that are configured to attached/detached from an eyeglasses frame;
FIG. 15 a and FIG. 15 b illustrate an embodiment of a clip-on device that includes inward-facing head-mounted cameras pointed at the lower part of the face and the forehead;
FIG. 16 a and FIG. 16 b illustrate an embodiment of a single-unit clip-on device that is configured to be attached behind an eyeglasses frame;
FIG. 17 a and FIG. 17 b illustrate embodiments of right and left clip-on devices that are configured to be attached behind an eyeglasses frame;
FIG. 18 illustrates embodiments of right and left clip-on devices, which are configured to be attached/detached from an eyeglasses frame, and have protruding arms to hold inward-facing head-mounted cameras;
FIG. 19 a is a schematic illustration of an inward-facing head-mounted camera embedded in an eyeglasses frame, which utilizes the Scheimpflug principle;
FIG. 19 b is a schematic illustration of a camera that is able to change the relative tilt between its lens and sensor planes according to the Scheimpflug principle;
FIG. 20 illustrates an embodiment of a system configured to calculate a physiological signal;
FIG. 21 illustrates an embodiment of a system configured to calculate blood pressure that includes at least two inward-facing HCAMs;
FIG. 22 illustrates one embodiment of a system configured to calculate blood pressure, which includes inward-facing HCAMs as well as outward-facing HCAMs;
FIG. 23 and FIG. 24 illustrate brainwave headsets having at least two inward facing cameras that capture the user's facial expressions;
FIG. 25 illustrates an HMD having head mounted cameras able to capture both the user's face and the user's back;
FIG. 26 illustrates a HMD having head mounted cameras around the head;
FIG. 27 illustrates a HMD having head mounted cameras able to capture portions of the user's torso, hands, and legs;
FIG. 28 illustrates a HMD having head mounted a camera able to capture the user's shoulder;
FIG. 29 , FIG. 30 , FIG. 31 , and FIG. 32 illustrate HMDs having head mounted cameras able to capture both the user's face and the user's back;
FIG. 33 and FIG. 34 illustrate HMDs having head mounted cameras able to capture both the user's facial expressions and hand gestures with the same camera; and
FIG. 35 a and FIG. 35 b are schematic illustrations of possible embodiments for computers.
DETAILED DESCRIPTION
âVisible-light cameraâ refers to a non-contact device designed to detect at least some of the visible spectrum, such as a video camera with optical lenses and CMOS or CCD sensor. A âthermal cameraâ refers herein to a non-contact device that measures electromagnetic radiation having wavelengths longer than 2500 nanometer (nm) and does not touch its region of interest (ROI). A thermal camera may include one sensing element (pixel), or multiple sensing elements that are also referred to herein as âsensing pixelsâ, âpixelsâ, and/or focal-plane array (FPA). A thermal camera may be based on an uncooled thermal sensor, such as a thermopile sensor, a microbolometer sensor (where microbolometer refers to any type of a bolometer sensor and its equivalents), a pyroelectric sensor, or a ferroelectric sensor.
A reference to a âcameraâ herein may relate to various types of devices. In one example, a camera is a visible-light camera. In another example, a camera may capture light in the ultra-violet range. And in another example, a camera may capture near infrared radiation (e.g., wavelengths between 750 and 2000 nm).
In some embodiments, a device, such as a camera, may be positioned such that it occludes an ROI on the user's face, while in other embodiments, the device may be positioned such that it does not occlude the ROI. Sentences in the form of âthe system/camera does not occlude the ROIâ indicate that the ROI can be observed by a third person located in front of the user and looking at the ROI, such as illustrated by all the ROIs in FIG. 7 and FIG. 11 . Sentences in the form of âthe system/camera occludes the ROIâ indicate that some of the ROIs cannot be observed directly by that third person, such as ROIs
19 and 37 that are occluded by the lenses in FIG. 1 a , and ROIs
97 and 102 that are occluded by cameras
91 and 96 , respectively, in FIG. 9 .
Although some of the disclosed embodiments can use occluding cameras successfully, in certain scenarios, such as when using a head-mounted system (HMS) on a daily basis and/or in a normal day-to-day setting, using cameras that do not occlude their ROIs on the face may provide one or more advantages to the user, to the HMS, and/or to the cameras, which may relate to one or more of the following: esthetics, better ventilation of the face, reduced weight, simplicity to wear, ability to operate without active illumination, and reduced likelihood to being tarnished.
The term âinward-facing head-mounted cameraâ refers to a camera configured to be worn on a user's head and to remain pointed at its ROI, which is on the user's face, also when the user's head makes angular and lateral movements (such as movements with an angular velocity above 0.1 rad/sec, above 0.5 rad/sec, and/or above 1 rad/sec). A head-mounted camera (which may be inward-facing and/or outward-facing) may be physically coupled to a frame worn on the user's head, may be attached to eyeglass using a clip-on mechanism (configured to be attached to and detached from the eyeglasses), or may be mounted to the user's head using any other known device that keeps the camera in a fixed position relative to the user's head also when the head moves. Sentences in the form of âcamera physically coupled to the frameâ mean that the camera moves with the frame, such as when the camera is fixed to (or integrated into) the frame, or when the camera is fixed to (or integrated into) an element that is physically coupled to the frame.
Sentences in the form of âa frame configured to be worn on a user's headâ or âa frame worn on a user's headâ refer to a mechanical structure that loads more than 50% of its weight on the user's head. For example, an eyeglasses frame may include two temples connected to two rims connected by a bridge; the frame in Oculus Rift⢠includes the foam placed on the user's face and the straps; and the frames in Google Glass⢠and Spectacles by Snap Inc. are similar to eyeglasses frames. Additionally or alternatively, the frame may connect to, be affixed within, and/or be integrated with, a helmet (e.g., sports, motorcycle, bicycle, and/or combat helmets) and/or a brainwave-measuring headset.
When a camera is inward-facing and head-mounted, challenges faced by systems known in the art that are used to acquire images, which include non-head-mounted cameras, may be simplified and even eliminated with some of the embodiments described herein. Some of these challenges may involve dealing with complications caused by movements of the user, image registration, ROI alignment, tracking based on markers, and/or motion compensation.
In various embodiments, cameras are located close to a user's face, such as at most 2 cm, 5 cm, 10 cm, 15 cm, or 20 cm from the face (herein âcmâ denotes to centimeters). The distance from the face/head in sentences such as âa camera located less than 10 cm from the face/headâ refers to the shortest possible distance between the camera and the face/head. The head-mounted cameras used in various embodiments may be lightweight, such that each camera weighs below 10 g, 5 g, 1 g, and/or 0.5 g (herein âgâ denotes to grams).
The following figures show various examples of HMSs equipped with head-mounted cameras. FIG. 1 a illustrates various inward-facing head-mounted cameras coupled to an eyeglasses frame 15 . Cameras 10 and 12 measure regions 11 and 13 on the forehead, respectively. Cameras
18 and 36 measure regions on the periorbital areas
19 and 37 , respectively. The HMS further includes an optional computer 16 , which may include a processor, memory, a battery and/or a communication module. FIG. 1 b illustrates a similar HMS in which inward-facing head-mounted cameras
48 and 49
measure regions
41 and 41 , respectively. Cameras
22 and 24
measure regions
23 and 25 , respectively. Camera 28 measures region 29 . And cameras 26 and 43
measure regions
38 and 39 , respectively.
FIG. 2 illustrates inward-facing head-mounted cameras coupled to an augmented reality device such as Microsoft HoloLensâ¢. FIG. 3 illustrates head-mounted cameras coupled to a virtual reality device such as Facebook's Oculus Riftâ¢. FIG. 4 is a side view illustration of head-mounted cameras coupled to an augmented reality device such as Google Glassâ¢. FIG. 5 is another side view illustration of head-mounted cameras coupled to a sunglasses frame.
FIG. 6 to FIG. 9 illustrate HMSs configured to measure various ROIs relevant to some of the embodiments describes herein. FIG. 6 illustrates a frame 35 that mounts inward-facing head-mounted cameras
30 and 31 that measure regions
32 and 33 on the forehead, respectively. FIG. 7 illustrates a frame 75 that mounts inward-facing head-mounted cameras
70 and 71 that measure regions
72 and 73 on the forehead, respectively, and inward-facing head-mounted cameras
76 and 77 that measure regions
78 and 79 on the upper lip, respectively. FIG. 8 illustrates a frame 84 that mounts inward-facing head-mounted cameras
80 and 81 that measure regions
82 and 83 on the sides of the nose, respectively. And FIG. 9 illustrates a frame 90 that includes (i) inward-facing head-mounted cameras
91 and 92 that are mounted to protruding arms and measure regions
97 and 98 on the forehead, respectively, (ii) inward-facing head-mounted cameras
95 and 96 , which are also mounted to protruding arms, which measure regions
101 and 102 on the lower part of the face, respectively, and (iii) head-mounted cameras
93 and 94 that measure regions on the periorbital areas
99 and 100 , respectively.
FIG. 10 to FIG. 13 illustrate various inward-facing head-mounted cameras having multi-pixel sensors (FPA sensors), configured to measure various ROIs relevant to some of the embodiments describes herein. FIG. 10 illustrates head-mounted cameras
120 and 122 that measure regions
121 and 123 on the forehead, respectively, and mounts head-mounted camera 124 that measure region 125 on the nose. FIG. 11 illustrates head-mounted cameras
126 and 128 that measure regions
127 and 129 on the upper lip, respectively, in addition to the head-mounted cameras already described in FIG. 10 . FIG. 12 illustrates head-mounted cameras
130 and 132 that measure larger regions
131 and 133 on the upper lip and the sides of the nose, respectively. And FIG. 13 illustrates head-mounted cameras
134 and 137 that measure regions
135 and 138 on the right and left cheeks and right and left sides of the mouth, respectively, in addition to the head-mounted cameras already described in FIG. 12 .
In some embodiments, the head-mounted cameras may be physically coupled to the frame using a clip-on device configured to be attached/detached from a pair of eyeglasses in order to secure/release the device to/from the eyeglasses, multiple times. The clip-on device holds at least an inward-facing camera, a processor, a battery, and a wireless communication module. Most of the clip-on device may be located in front of the frame (as illustrated in FIG. 14 b , FIG. 15 b , and FIG. 18 ), or alternatively, most of the clip-on device may be located behind the frame, as illustrated in FIG. 17 b and FIG. 16 b.
FIG. 14 a , FIG. 14 b , and FIG. 14 c illustrate two right and left clip-on devices
141 and 142 , respectively, configured to attached/detached from an eyeglasses frame 140 . The clip-on device 142 includes an inward-facing head-mounted camera 143 pointed at a region on the lower part of the face (such as the upper lip, mouth, nose, and/or cheek), an inward-facing head-mounted camera 144 pointed at the forehead, and other electronics 145 (such as a processor, a battery, and/or a wireless communication module). The clip-on devices
141 and 142 may include additional cameras illustrated in the drawings as black circles.
FIG. 15 a and FIG. 15 b illustrate a clip-on device 147 that includes an inward-facing head-mounted camera 148 pointed at a region on the lower part of the face (such as the nose), and an inward-facing head-mounted camera 149 pointed at the forehead. The other electronics (such as a processor, a battery, and/or a wireless communication module) is located inside the box 150 , which also holds the cameras
148 and 149 .
FIG. 17 a and FIG. 17 b illustrate two right and left clip-on devices
160 and 161 , respectively, configured to be attached behind an eyeglasses frame 165 . The clip-on device 160 includes an inward-facing head-mounted camera 162 pointed at a region on the lower part of the face (such as the upper lip, mouth, nose, and/or cheek), an inward-facing head-mounted camera 163 pointed at the forehead, and other electronics 164 (such as a processor, a battery, and/or a wireless communication module). The clip-on devices
160 and 161 may include additional cameras illustrated in the drawings as black circles.
FIG. 16 a and FIG. 16 b illustrate a single-unit clip-on device 170 , configured to be attached behind an eyeglasses frame 176 . The single-unit clip-on device 170 includes inward-facing head-mounted cameras
171 and 172 pointed at regions on the lower part of the face (such as the upper lip, mouth, nose, and/or cheek), inward-facing head-mounted cameras
173 and 174 pointed at the forehead, a spring 175 configured to apply force that holds the clip-on device 170 to the frame 176 , and other electronics 177 (such as a processor, a battery, and/or a wireless communication module). The clip-on device 170 may include additional cameras illustrated in the drawings as black circles.
FIG. 18 illustrates two right and left clip-on devices 153 and 154 , respectively, configured to attached/detached from an eyeglasses frame, and having protruding arms to hold the inward-facing head-mounted cameras. Head-mounted camera 155 measures a region on the lower part of the face, head-mounted camera 156 measures regions on the forehead, and the left clip-on device 154 further includes other electronics 157 (such as a processor, a battery, and/or a wireless communication module). The clip-on devices 153 and 154 may include additional cameras illustrated in the drawings as black circles.
It is noted that the elliptic and other shapes of the ROIs in some of the drawings are just for illustration purposes, and the actual shapes of the ROIs are usually not as illustrated. It is possible to calculate the accurate shape of an ROI using various methods, such as a computerized simulation using a 3D model of the face and a model of a head-mounted system (HMS) to which a camera is physically coupled, or by placing a LED instead of the sensor, while maintaining the same field of view (FOV) and observing the illumination pattern on the face. Furthermore, illustrations and discussions of a camera represent one or more cameras, where each camera may have the same FOV and/or different FOVs. Unless indicated to the contrary, the cameras may include one or more sensing elements (pixels), even when multiple sensing elements do not explicitly appear in the figures; when a camera includes multiple sensing elements then the illustrated ROI usually refers to the total ROI captured by the camera, which is made of multiple regions that are respectively captured by the different sensing elements. The positions of the cameras in the figures are just for illustration, and the cameras may be placed at other positions on the HMS.
Sentences in the form of an âROI on an areaâ, such as ROI on the forehead or an ROI on the nose, refer to at least a portion of the area. Depending on the context, and especially when using a camera having a small number of pixels, the ROI may cover another area (in addition to the area). For example, a sentence in the form of âan ROI on the noseâ may refer to either: 100% of the ROI is on the nose, or some of the ROI is on the nose and some of the ROI is on the upper lip.
FIG. 20 illustrates a system for calculating a physiological signal of a user. In one embodiment, the system includes at least a head-mounted device 612 , a head-mounted camera 614 , and a computer 610 . Optionally, the system may include additional elements, such as a frame 620 , a second head-mounted device 616 , and/or an inertial measurement unit (IMU) 618 .
The head-mounted device 612 measures photoplethysmographic signal (PPG signal) 613 at a region on the user's head. Some examples of regions at which measurements of the PPG signal 613 may be taken include a temple, the forehead, a cheek, the nose, and behind an ear. Optionally, the PPG signal 613 represents changes in the concentration levels of hemoglobin and blood oxygenation due to the dynamics of the user's blood flow. Various techniques may be utilized in order to quantify these changes, in order to produce the PPG signal.
In one embodiment, the head-mounted device 612 is a photoplethysmographic device (PPG device). In one example, the PPG device includes a light source and a photodetector. Optionally, the light source emits light to the region on the face, and the photodetector measures the reflected light from the tissue. Optionally, the reflected light is proportional to blood volume variations. In some embodiments, the PPG device utilizes light having a single wavelength (e.g., green light with a wavelength of â530 nm). In another example, the PPG device utilizes light having multiple wavelengths, which may be emitted by multiple LEDs.
In another embodiment, the head-mounted device 612 is a second camera located more than 10 mm away from the region on the user's head, and the PPG signal 613 is recognizable from color changes in a region in images taken by the second camera. Thus, the second camera may be considered an inward-facing camera. Optionally, the second camera weighs less than 10 g (grams). Optionally, the second camera does not occlude the region on the user's head.
Known imaging photoplethysmographic techniques may be used to calculate the PPG signal 613 from the color changes that are recognizable in the region in the images taken by the second camera. Herein, sentences of the form âthe PPG signal is recognizable from color changes in a region in the imagesâ refer to effects of color changes due to variations in blood flow that may be identified and/or utilized by the computer 610 , which are usually not recognized by the naked eye. Herein, âcolor changesâ includes changes to amplitudes of one or more of the color channels in the images, and/or changes to ratios between amplitudes of two or more color channels in the images. There are various signal processing and/or analytical techniques known in the art that may be utilized by the computer 610 to extract the PPG signal 613 from the images taken by the second camera.
In some embodiments, the computer 610 may employ one or more of the following preprocessing techniques in order to obtain the PPG signal 613 from images taken by the second camera: normalization of pixel intensities (e.g., to obtain a zero-mean unit variance time series signal), and conditioning a time series signal by constructing a square wave, a sine wave, or a user defined shape, such as that obtained from an ECG signal or a PPG signal as described in U.S. Pat. No. 8,617,081. Various preprocessing techniques known in the art that may assist in extracting the PPG signal 613 from the images 615 are discussed in Zaunseder et al. (2018), âCardiovascular assessment by imaging photoplethysmographyâa reviewâ, Biomedical Engineering 63(5), 617-634. Additional examples of processing known in the art, which may be utilized by the computer 610 , are given below.
In one example, U.S. Pat. No. 8,768,438, titled âDetermining cardiac arrhythmia from a video of a subject being monitored for cardiac functionâ, describes how to obtain a PPG signal from video of the user. In this example, a time series signal is generated from video images of a subject's exposed skin, and a reference signal is used to perform a constrained source separation (which is a variant of ICA) on the time series signals to obtain the PPG signal. Peak-to-peak pulse points are detected in the PPG signal, which may be analyzed to determine parameters such as heart rate, heart rate variability, and/or to obtain peak-to-peak pulse dynamics that can be indicative of conditions such as cardiac arrhythmia.
In another example, U.S. Pat. No. 8,977,347, titled âVideo-based estimation of heart rate variabilityâ, describes how a times-series signal similar to the one described above may be subjected to a different type of analysis to detect the heart rate variability. In this example, the time series data are de-trended to remove slow non-stationary trends from the signal and filtered (e.g., using bandpass filtering). Following that, low frequency and high frequency components of the integrated power spectrum within the time series signal are extracted using Fast Fourier Transform (FFT). A ratio of the low and high frequency of the integrated power spectrum within these components is computed. And analysis of the dynamics of this ratio over time is used to estimate heart rate variability.
In yet another example, U.S. Pat. No. 9,020,185, titled âSystems and methods for non-contact heart rate sensingâ, describes how to obtain a PPG signal from video of a user, which can be filtered and processed to separate an underlying pulsing signal by, for example, using an ICA algorithm. The separated pulsing signal from the algorithm can be transformed into frequency spacing data using FFT, in which the heart rate can be extracted or estimated.
Due to the proximity of the second camera to the face, in some embodiments, there may be an acute angle between the optical axis of second camera and the region on the face (e.g., when the region includes a portion on the forehead). In order to improve the sharpness of the images taken by the second camera, the second camera may be configured to operate in a way that takes advantage of the Scheimpflug principle. In one embodiment, the second camera includes a sensor and a lens; the sensor plane is tilted by a fixed angle greater than 2° relative to the lens plane according to the Scheimpflug principle in order to capture a sharper image when the second camera is worn by the user (where the lens plane refers to a plane that is perpendicular to the optical axis of the lens, which may include one or more lenses). Optionally, camera does not occlude the region on the user's face. In another embodiment, the second camera includes a sensor, a lens, and a motor; the motor tilts the lens relative to the sensor according to the Scheimpflug principle. The tilt improves the sharpness of images when the second camera is worn by the user. Additional details regarding utilization of the Scheimpflug principle are provided further below.
The head-mounted camera 614 captures images 615 indicative of posture of the user. For example, the head-mounted camera 614 is configured such that its field of view (FOV) includes portions of the user's body (e.g., feet or a shoulder) when the user stands upright and looks ahead (thus, the head-mounted camera 614 may be considered a down-pointing camera). In another embodiment, the head-mounted camera 614 has a FOV that is similar to the user's, e.g., it is oriented such that it has a frontal view when the user stands upright and looks ahead. Optionally, in this embodiment, the FOV of the head-mounted camera 614 does not include the feet and/or shoulders of the user when the user stands upright and looks ahead. Optionally, the head-mounted camera 614 is an outward-facing camera that is utilized by an extended reality device, such as an augmented reality device, a virtual reality device, or a mixed reality device. Optionally, the outward-facing camera is utilized by the extended reality device to capture images of the user's surroundings.
In one embodiment, the head-mounted camera 614 is a visible light camera and/or a near-IR camera. Optionally, the head-mounted camera 614 features an extended depth of field such as: (i) a camera that operates according to Scheimpflug principle, (ii) a light field camera, and/or (iii) a camera that utilizes at least one of the following techniques to achieve an extended depth of field: wavefront coding, diffusion coding, coded aperture, multiple apertures, and/or a lens array.
It is to be noted that some embodiments may involve utilization of multiple head-mounted cameras to generate images indicative of the user's posture. Some examples of head mounted cameras and their locations and/or orientations that may be utilized by embodiments of the system illustrated in FIG. 20 are illustrated herein in FIG. 24 to FIG. 34 .
Various elements of the system illustrated in FIG. 20 may be coupled to the frame 620 , which is configured to be worn on the user's head. In some embodiments, the head-mounted device 612 and/or the head-mounted camera 614 may be physically coupled to the frame 620 . Optionally, the frame 620 may be an eyeglasses frame, or a frame of smartglasses or an extended reality device.
In one embodiment, the system illustrated in FIG. 20 includes an IMU 618 , which provides a signal indicative of the movement and/or orientation of the user. Optionally, IMU 618 is physically coupled to the frame 620 and/or attached to the user's head in some other fashi
CLAIMS
Claims ( 20 )
We claim:
1. A system configured to identify orthostatic hypotension, comprising:
a head-mounted device configured to measure photoplethysmographic signal (PPG signal) at a region on a user's head;
a head-mounted camera configured to capture images indicative of the user's posture; and
a computer configured to:
calculate systolic and diastolic blood pressure values based on the PPG signal; and
identify orthostatic hypotension based on a drop of systolic blood pressure below a first threshold, and/or a drop of diastolic blood pressure below a second threshold, within a predetermined duration from a transition in the posture from supine to sitting posture, or from sitting to standing posture.
2. The system of claim 1 , wherein the computer adjusts calculations of the blood pressure values based on the user's posture, such that for the same PPG signal the computer outputs different values for the blood pressure values for the following different postures: standing, sitting, and lying down.
3. The system of claim 2 , wherein the computer is further configured to utilize a model to calculate the blood pressure values; wherein the model was generated based on samples comprising: feature values generated from PPG signals of multiple users and images of the multiple users, and labels generated based on corresponding values of blood pressure values of the multiple users; and wherein a first non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were sitting, and a second non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were standing.
4. The system of claim 2 , wherein the computer is further configured to utilize a model to calculate the blood pressure values; wherein the model was generated based on samples comprising: feature values generated from PPG signals of multiple users and images of the multiple users, and labels generated based on corresponding values of blood pressure values of the multiple users; and wherein a first non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were sitting, and a second non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were lying down.
5. The system of claim 2 , wherein the head-mounted device is a contact photoplethysmographic device.
6. The system of claim 2 , wherein the head-mounted device is a second camera located more than 10 mm away from the region on the user's head, and the PPG signal is recognizable from color changes in a region in images taken by the second camera.
7. The system of claim 2 , wherein the computer is further configured to generate feature values based on data comprising the PPG signal and the images, and to utilize a model to calculate the blood pressure values based on the feature values; wherein one or more of the feature values are indicative of the user's posture, and the PPG signal is indicative of cardiac pulse wave arrival times at the region on the user's head; wherein the computer is further configured to: (i) receive a second photoplethysmographic signal (second PPG signal) indicative of pulse wave arrival times at a second region on the user's body, which is at least 25 mm away from the region on the user's head, and (ii) generate at least one of the feature values based on the second PPG signal; and wherein the at least one of the feature values are indicative of a difference in cardiac pulse wave arrival times at the region and the second region.
8. The system of claim 7 , wherein the second region is located on a wrist of the user, and further comprising a wrist-mounted device configured to measure the second PPG signal.
9. The system of claim 2 , wherein the computer is further configured to generate feature values based on data comprising the PPG signal and the images, and to utilize a model to calculate the blood pressure values based on the feature values; wherein one or more of the feature values are indicative of the user's posture, and the PPG signal is indicative of cardiac pulse wave arrival times at the region on the user's head; wherein the computer is further configured to: (i) receive a signal indicative of the user's heart's electrical activity (EA signal), and (ii) generate at least one of the feature values based on the signal indicative of the user's heart's electrical activity; wherein the EA signal is indicative of times at which one or more of the following cardiac activity phases occur: atrial systole, ventricular systole, and ventricular repolarization; and wherein the at least one of the feature values are indicative of a difference in time between when a certain cardiac activity phase of the user and when a corresponding pulse wave arrives at the region on the user's head.
10. The system of claim 1 , wherein the computer is further configured to generate feature values based on data comprising the PPG signal and the images, and to utilize a model to calculate the blood pressure values based on the feature values; and wherein a feature value, from among the feature values, which is based on the images, is indicative of the vertical distance between the head-mounted device and the user's heart.
11. The system of claim 1 , wherein the computer is further configured to generate feature values based on data comprising the PPG signal and the images, and to utilize a model to calculate the blood pressure values based on the feature values; and wherein a feature value, from among the feature values, which is based on the images, is indicative of the vertical distance between the head-mounted device and the brachial artery of the user.
12. The system of claim 1 , wherein the computer is further configured to generate feature values based on data comprising the PPG signal and the images, and to utilize a model to calculate the blood pressure values based on the feature values; and wherein a feature value, from among of the feature values, is generated based on the images and is indicative of whether the user's legs are crossed.
13. A system configured to identify Postural-Orthostatic Tachycardia Syndrome, comprising:
a head-mounted device configured to measure photoplethysmographic signal (PPG signal) at a region on a user's head;
a head-mounted camera configured to capture images indicative of the user's posture; and
a computer configured to detect tachycardia based on the PPG signal, and to identify Postural-Orthostatic Tachycardia Syndrome based on detecting occurrence of the tachycardia within a predetermined duration from a transition in the posture from supine or sitting posture to standing posture.
14. The system of claim 13 , wherein the head-mounted device is a contact photoplethysmographic device.
15. A method for identifying orthostatic hypotension, comprising:
measuring, by a head-mounted device, a photoplethysmographic signal (PPG signal) at a region on a user's head;
capturing, by a head-mounted camera, images indicative of the user's posture;
calculating systolic and diastolic blood pressure values based on the PPG signal; and
identifying orthostatic hypotension based on a drop of systolic blood pressure below a first threshold, and/or a drop of diastolic blood pressure below a second threshold, within a predetermined duration from a transition in the posture from supine to sitting posture, or from sitting to standing posture.
16. The method of claim 15 , further comprising generating feature values based on data comprising the PPG signal and the images, and utilizing a model to calculate the blood pressure values based on the feature values; wherein the model was generated based on samples comprising: feature values generated from PPG signals of multiple users and images of the multiple users, and labels generated based on corresponding values of blood pressure values of the multiple users; and wherein a first non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were sitting, and a second non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were lying down.
17. The method of claim 15 , further comprising a 4-non-transitory computer-readable medium having instructions stored thereon that, in response to execution by a system including a processor and memory, causes the system to perform operations described in the method of claim 15 .
18. The method of claim 15 , further comprising generating feature values based on data comprising the PPG signal and the images, and utilizing a model to calculate the blood pressure values based on the feature values; wherein the model was generated based on samples comprising: feature values generated from PPG signals of multiple users and images of the multiple users, and labels generated based on corresponding values of blood pressure values of the multiple users; and wherein a first non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were sitting, and a second non-empty subset of the samples were generated based on PPG signals and images taken while at least some of the multiple users were standing.
19. The method of claim 15 , further comprising generate feature values based on data comprising the PPG signal and the images, and utilizing a model to calculate the blood pressure values based on the feature values; wherein one or more of the feature values are indicative of the user's posture, and the PPG signal is indicative of cardiac pulse wave arrival times at the region on the user's head; and further comprising: (i) receiving a second photoplethysmographic signal (second PPG signal) indicative of pulse wave arrival times at a second region on the user's body, which is at least 25 mm away from the region on the user's head, and (ii) generating at least one of the feature values based on the second PPG signal; and wherein the at least one of the feature values are indicative of a difference in cardiac pulse wave arrival times at the region and the second region.
20. The method of claim 15 , further comprising generating feature values based on data comprising the PPG signal and the images, and utilizing a model to calculate the blood pressure values based on the feature values; wherein one or more of the feature values are indicative of the user's posture, and the PPG signal is indicative of cardiac pulse wave arrival times at the region on the user's head; and further comprising: (i) receiving a signal indicative of the user's heart's electrical activity (EA signal), and (ii) generating at least one of the feature values based on the signal indicative of the user's heart's electrical activity; wherein the EA signal is indicative of times at which one or more of the following cardiac activity phases occur: atrial systole, ventricular systole, and ventricular repolarization; and wherein the at least one of the feature values are indicative of a difference in time between when a certain cardiac activity phase of the user and when a corresponding pulse wave arrives at the region on the user's head.
US16/453,993
2015-06-14
2019-06-26
Identification of posture-related syncope using head-mounted sensors
Active
US10667697B2
( en )
Priority Applications (15)
Application Number
Priority Date
Filing Date
Title
US16/453,993
US10667697B2
( en )
2015-06-14
2019-06-26
Identification of posture-related syncope using head-mounted sensors
US16/551,654
US10638938B1
( en )
2015-06-14
2019-08-26
Eyeglasses to detect abnormal medical events including stroke and migraine
US16/689,929
US11064892B2
( en )
2015-06-14
2019-11-20
Detecting a transient ischemic attack using photoplethysmogram signals
US16/689,959
US10799122B2
( en )
2015-06-14
2019-11-20
Utilizing correlations between PPG signals and iPPG signals to improve detection of physiological responses
US16/831,413
US10791938B2
( en )
2015-06-14
2020-03-26
Smartglasses for detecting congestive heart failure
US16/854,883
US10813559B2
( en )
2015-06-14
2020-04-21
Detecting respiratory tract infection based on changes in coughing sounds
US17/005,259
US11103139B2
( en )
2015-06-14
2020-08-27
Detecting fever from video images and a baseline
US17/009,655
US11154203B2
( en )
2015-06-14
2020-09-01
Detecting fever from images and temperatures
US17/027,677
US11103140B2
( en )
2015-06-14
2020-09-21
Monitoring blood sugar level with a comfortable head-mounted device
US17/320,012
US12232850B2
( en )
2015-06-14
2021-05-13
Doorway system that utilizes wearable-based health state verifications
US17/319,634
US11903680B2
( en )
2015-06-14
2021-05-13
Wearable-based health state verification for physical access authorization
US17/319,817
US12293844B2
( en )
2015-06-14
2021-05-13
Wearable-based certification of a premises as contagion-safe
US17/381,222
US11986273B2
( en )
2015-06-14
2021-07-21
Detecting alcohol intoxication from video images
US18/538,234
US12193792B2
( en )
2015-06-14
2023-12-13
Reservation booking using wearable-based health state verifications
US19/032,007
US20250160668A1
( en )
2015-06-14
2025-01-18
Eyewear-Mounted Dual-Sided PPG System for Enhanced Blood Flow Measurement
Applications Claiming Priority (22)
Application Number
Priority Date
Filing Date
Title
US201562175319P
2015-06-14
2015-06-14
US201562202808P
2015-08-08
2015-08-08
US201562236868P
2015-10-03
2015-10-03
US15/182,566
US9867546B2
( en )
2015-06-14
2016-06-14
Wearable device for taking symmetric thermal measurements
US15/182,592
US10165949B2
( en )
2015-06-14
2016-06-14
Estimating posture using head-mounted cameras
US201662354833P
2016-06-27
2016-06-27
US201662372063P
2016-08-08
2016-08-08
US15/231,276
US20170035344A1
( en )
2015-08-08
2016-08-08
Detection of an Allergic Reaction Using Thermal Measurements of the Face
US15/284,528
US10113913B2
( en )
2015-10-03
2016-10-03
Systems for collecting thermal measurements of the face
US201762456105P
2017-02-07
2017-02-07
US201762480496P
2017-04-02
2017-04-02
US15/635,178
US10136856B2
( en )
2016-06-27
2017-06-27
Wearable respiration measurements system
US201762566572P
2017-10-02
2017-10-02
US15/722,434
US10523852B2
( en )
2015-06-14
2017-10-02
Wearable inward-facing camera utilizing the Scheimpflug principle
US15/833,115
US10130261B2
( en )
2015-06-14
2017-12-06
Detecting physiological responses while taking into account consumption of confounding substances
US15/832,855
US10130308B2
( en )
2015-06-14
2017-12-06
Calculating respiratory parameters from thermal measurements
US201862652348P
2018-04-04
2018-04-04
US201862667453P
2018-05-05
2018-05-05
US16/147,695
US10376153B2
( en )
2015-06-14
2018-09-29
Head mounted system to collect facial expressions
US16/156,493
US10524667B2
( en )
2015-06-14
2018-10-10
Respiration-based estimation of an aerobic activity parameter
US16/375,841
US10376163B1
( en )
2015-06-14
2019-04-04
Blood pressure from inward-facing head-mounted cameras
US16/453,993
US10667697B2
( en )
2015-06-14
2019-06-26
Identification of posture-related syncope using head-mounted sensors
Related Parent Applications (3)
Application Number
Title
Priority Date
Filing Date
US16/147,695
Continuation-In-Part
US10376153B2
( en )
2015-06-14
2018-09-29
Head mounted system to collect facial expressions
US16/375,841
Continuation-In-Part
US10376163B1
( en )
2015-06-14
2019-04-04
Blood pressure from inward-facing head-mounted cameras
US16/831,413
Continuation-In-Part
US10791938B2
( en )
2015-06-14
2020-03-26
Smartglasses for detecting congestive heart failure
Related Child Applications (6)
Application Number
Title
Priority Date
Filing Date
US15/635,178
Continuation
US10136856B2
( en )
2015-06-14
2017-06-27
Wearable respiration measurements system
US15/635,178
Continuation-In-Part
US10136856B2
( en )
2015-06-14
2017-06-27
Wearable respiration measurements system
US16/551,654
Continuation-In-Part
US10638938B1
( en )
2015-06-14
2019-08-26
Eyeglasses to detect abnormal medical events including stroke and migraine
US16/551,564
Continuation-In-Part
US20210065310A1
( en )
2019-08-26
2019-08-26
User interfaces for instant payroll with bulk approval
US16/689,959
Continuation-In-Part
US10799122B2
( en )
2015-06-14
2019-11-20
Utilizing correlations between PPG signals and iPPG signals to improve detection of physiological responses
US16/854,883
Continuation-In-Part
US10813559B2
( en )
2015-06-14
2020-04-21
Detecting respiratory tract infection based on changes in coughing sounds
Publications (2)
Publication Number
Publication Date
US20190313915A1
US20190313915A1 ( en )
2019-10-17
US10667697B2
true
US10667697B2 ( en )
2020-06-02
Family
ID=68160833
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/453,993
Active
US10667697B2
( en )
2015-06-14
2019-06-26
Identification of posture-related syncope using head-mounted sensors
Country Status (1)
Country
Link
US
( 1 )
US10667697B2
( en )
Cited By (1)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11468713B2
( en )
2021-03-02
2022-10-11
Bank Of America Corporation
System and method for leveraging a time-series of microexpressions of users in customizing media presentation based on users# sentiments
Families Citing this family (62)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11154203B2
( en )
2015-06-14
2021-10-26
Facense Ltd.
Detecting fever from images and temperatures
US11103140B2
( en )
2015-06-14
2021-08-31
Facense Ltd.
Monitoring blood sugar level with a comfortable head-mounted device
CA2933799A1
( en )
*
2016-06-21
2017-12-21
John G. Robertson
System and method for automatically generating a facial remediation design and application protocol to address observable facial deviations
US11184574B2
( en )
2017-07-17
2021-11-23
Facebook, Inc.
Representing real-world objects with a virtual reality environment
US10860103B2
( en )
*
2018-09-19
2020-12-08
Dell Products, L.P.
Enhancing virtual reality with physiological sensing
US11830182B1
( en )
2019-08-20
2023-11-28
Apple Inc.
Machine learning-based blood flow tracking
TWI750504B
( en )
*
2019-08-29
2021-12-21
é æ¡æºæ §è¡ä»½æéå ¬å¸
Method of liveness detection and related device
WO2021084488A1
( en )
*
2019-10-31
2021-05-06
Facense Ltd.
Smartglasses for detecting physiological parameters
KR102405416B1
( en )
*
2019-11-25
2022-06-07
íêµì ì기ì ì°êµ¬ì
Method for posture estimation of user wearing Head Mounted Display
US11587276B2
( en )
2019-12-03
2023-02-21
Disney Enterprises, Inc.
Data-driven extraction and composition of secondary dynamics in facial performance capture
US11810271B2
( en )
*
2019-12-04
2023-11-07
Align Technology, Inc.
Domain specific image quality assessment
CA3161710A1
( en )
*
2019-12-31
2021-07-08
William Xavier Kerber
Proxy controller suit with optional dual range kinematics
US11475639B2
( en )
2020-01-03
2022-10-18
Meta Platforms Technologies, Llc
Self presence in artificial reality
CN111222498A
( en )
*
2020-03-19
2020-06-02
æ¡æçµåç§æ大å¦
Identity recognition method based on photoplethysmography
US11604367B2
( en )
2020-04-08
2023-03-14
Facense Ltd.
Smartglasses with bendable temples
US11482049B1
( en )
2020-04-14
2022-10-25
Bank Of America Corporation
Media verification system
CN111582082B
( en )
*
2020-04-24
2022-04-01
ç山大å¦
Two-classification motor imagery electroencephalogram signal identification method based on interpretable clustering model
US12474365B2
( en )
*
2020-06-20
2025-11-18
Apple Inc.
User posture transition detection and classification
US11151767B1
( en )
*
2020-07-02
2021-10-19
Disney Enterprises, Inc.
Techniques for removing and synthesizing secondary dynamics in facial performance capture
US20230301523A1
( en )
*
2020-07-09
2023-09-28
NeuroSteer Ltd.
Systems and apparatuses for physiological and psychological parameter monitoring from a subject's head and methods of use thereof
US20230284912A1
( en )
*
2020-09-11
2023-09-14
Stat Health Informatics, Inc.
Long-term continuous biometric monitoring using in-ear pod
US20220100988A1
( en )
*
2020-09-25
2022-03-31
Canon Kabushiki Kaisha
Information processing apparatus, information processing method, and storage medium
US12219344B2
( en )
2020-09-25
2025-02-04
Apple Inc.
Adaptive audio centering for head tracking in spatial audio applications
US12373991B2
( en )
*
2020-10-07
2025-07-29
Electronics And Telecommunications Research Institute
Method, apparatus, system and computer-readable recording medium for feature information
CN112364694B
( en )
*
2020-10-13
2023-04-18
å®æ³¢å¤§å¦
Human body sitting posture identification method based on key point detection
WO2022093247A1
( en )
*
2020-10-29
2022-05-05
Hewlett-Packard Development Company, L.P.
Facial gesture mask
CN112464012B
( en )
*
2020-10-31
2022-06-17
æµæ±å·¥ä¸å¤§å¦
Automatic scenic spot photographing system capable of automatically screening photos and automatic scenic spot photographing method
US11803237B2
( en )
*
2020-11-14
2023-10-31
Facense Ltd.
Controlling an eye tracking camera according to eye movement velocity
CN114521874A
( en )
2020-11-23
2022-05-24
æ·±å³è¿ççç©å»ççµåè¡ä»½æéå ¬å¸
Orthostatic hypotension measurement method and related equipment
US12223104B2
( en )
2020-12-22
2025-02-11
Meta Platforms Technologies, Llc
Partial passthrough in virtual reality
KR102573068B1
( en )
*
2021-02-05
2023-08-31
ê³ëª ëíêµ ì°ííë ¥ë¨
Advanced visual intelligence based non-contact image based blood pressure measurement method and system
US11594032B1
( en )
2021-02-17
2023-02-28
Bank Of America Corporation
Media player and video verification system
US11790694B1
( en )
2021-02-17
2023-10-17
Bank Of America Corporation
Video player for secured video stream
US11527106B1
( en )
2021-02-17
2022-12-13
Bank Of America Corporation
Automated video verification
US11928187B1
( en )
2021-02-17
2024-03-12
Bank Of America Corporation
Media hosting system employing a secured video stream
US12183035B1
( en )
2021-03-08
2024-12-31
Meta Platforms, Inc.
System and method for positioning a 3D eyeglasses model
CN113065485A
( en )
*
2021-04-09
2021-07-02
æé½ç¿ç ç§ææéè´£ä»»å ¬å¸
Simultaneous-engraving multi-angle chef cap detection method based on frame alignment
CN113057617B
( en )
*
2021-04-30
2022-08-26
éåºç工大å¦
Non-invasive monitoring system for cardiac output
US11716456B2
( en )
*
2021-05-21
2023-08-01
Microsoft Technology Licensing, Llc
Autocalibrated near-eye display
US12032377B2
( en )
*
2021-05-25
2024-07-09
Ubkang (Qingdao) Technology Co., Ltd.
Mobility aid robot navigating method and mobility aid robot using the same
US11526548B1
( en )
2021-06-24
2022-12-13
Bank Of America Corporation
Image-based query language system for performing database operations on images and videos
US11941051B1
( en )
2021-06-24
2024-03-26
Bank Of America Corporation
System for performing programmatic operations using an image-based query language
US11295503B1
( en )
2021-06-28
2022-04-05
Facebook Technologies, Llc
Interactive avatars in artificial reality
US12028319B1
( en )
2021-07-06
2024-07-02
Bank Of America Corporation
Image-based firewall for synthetic media prevention
US11784975B1
( en )
2021-07-06
2023-10-10
Bank Of America Corporation
Image-based firewall system
CN113538644B
( en )
*
2021-07-19
2023-08-29
å京ç¾åº¦ç½è®¯ç§ææéå ¬å¸
Character dynamic video generation method, device, electronic device and storage medium
CN117377425A
( en )
*
2021-08-03
2024-01-09
å为ææ¯æéå ¬å¸
Electronic devices and methods for assessing carotid artery disease risk
US12159352B2
( en )
*
2021-08-30
2024-12-03
Bsset Llc
Extended reality movement platform
CN113887332B
( en )
*
2021-09-13
2024-04-05
ååç工大å¦
Skin operation safety monitoring method based on multi-mode fusion
CN114187393A
( en )
*
2021-12-08
2022-03-15
åå·å¯ç¿å ç§ææéå ¬å¸
ResNet-LSTM-based movement data redirection method
US12418696B1
( en )
2022-03-08
2025-09-16
Bank Of America Corporation
Action oriented detection of synthetic media objects
US12207014B2
( en )
*
2022-05-06
2025-01-21
Google Llc
Cloud-based application of visual effects to video
CN115171158B
( en )
*
2022-07-05
2025-07-15
é½é²å·¥ä¸å¤§å¦(å±±ä¸çç§å¦é¢)
A fingertip detection method based on visual features
US12097427B1
( en )
2022-08-26
2024-09-24
Meta Platforms Technologies, Llc
Alternate avatar controls
US20240104830A1
( en )
*
2022-09-24
2024-03-28
International Business Machines Corporation
Augmenting data used to train computer vision model with images of different perspectives
CN115826767B
( en )
*
2023-02-24
2023-06-30
é¿æ¥ç工大å¦
Multi-mode upper limb movement recognition model crossing tested as well as construction method and application method thereof
CN117275069B
( en )
*
2023-09-26
2024-06-04
åä¸ç§æ大å¦
End-to-end head gesture estimation method based on learnable vector and attention mechanism
KR20250121741A
( en )
*
2024-02-05
2025-08-12
íêµì ìíµì ì°êµ¬ì
Techniques for Gesture Recognition in Egocentric Vision Sensors with multiple users
CN117809354B
( en )
*
2024-02-29
2024-06-21
ååç工大å¦
Emotion recognition method, medium and device based on head wearable device perception
CN119091219B
( en )
*
2024-09-06
2025-09-02
ä¸é建工éå¢æéå ¬å¸
Computer vision-based construction site safety helmet wearing detection system and method
CN119851185B
( en )
*
2025-03-19
2025-05-20
åå·ç§ç¹æ£æµææ¯æéå ¬å¸
Attention-based multi-mode well site video safety analysis method
CN120938380A
( en )
*
2025-09-29
2025-11-14
ä¸å½äººæ°è§£æ¾åæ»å»é¢ç¬¬äºå»å¦ä¸å¿
Wearable cardiovascular stress warning system
Citations (48)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US5757002A
( en )
1995-08-30
1998-05-26
Kyoto Dai-Ichi Kagaku Co., Ltd.
Method of and apparatus for measuring lactic acid in organism
US6819950B2
( en )
2000-10-06
2004-11-16
Alexander K. Mills
Method for noninvasive continuous determination of physiologic characteristics
US20070276632A1
( en )
2006-05-26
2007-11-29
Triage Wireless, Inc.
System for measuring vital signs using bilateral pulse transit time
US7384395B2
( en )
2002-10-09
2008-06-10
Cardiac Pacemakers, Inc.
Detection of congestion from monitoring patient response to a recumbent position
US20100168589A1
( en )
2007-06-12
2010-07-01
Sotera Wireless, Inc.
BODY-WORN SYSTEM FOR MEASURING CONTINUOUS NON-INVASIVE BLOOD PRESSURE (cNIBP)
US20100241011A1
( en )
2009-03-20
2010-09-23
Massachusetts Institute Of Technology
Calibration of Pulse Transit Time Measurements to Arterial Blood Pressure using External Arterial Pressure Applied along the Pulse Transit Path
US20120022349A1
( en )
2008-10-07
2012-01-26
Orsan Medical Technologies Ltd.
Diagnosis of acute strokes
US20120029320A1
( en )
2010-07-30
2012-02-02
Nellcor Puritan Bennett Llc
Systems and methods for processing multiple physiological signals
US8175670B2
( en )
2004-03-09
2012-05-08
Nellcor Puritan Bennett Llc
Pulse oximetry signal correction using near infrared absorption by water
US8251903B2
( en )
2007-10-25
2012-08-28
Valencell, Inc.
Noninvasive physiological analysis using excitation-sensor modules and related devices and methods
US8512253B2
( en )
2002-01-15
2013-08-20
Orsan Medical Technologies, Ltd
Cerebral perfusion monitor
US20130215244A1
( en )
2012-02-21
2013-08-22
Lalit Keshav MESTHA
Removing environment factors from signals generated from video images captured for biomedical measurements
US8617081B2
( en )
2011-09-28
2013-12-31
Xerox Corporation
Estimating cardiac pulse recovery from multi-channel source data via constrained source separation
US8768438B2
( en )
2012-06-25
2014-07-01
Xerox Corporation
Determining cardiac arrhythmia from a video of a subject being monitored for cardiac function
US20140213917A1
( en )
2011-06-23
2014-07-31
Butterfleye SAL
Waterproof heart rate measuring apparatus
US8855384B2
( en )
2012-06-20
2014-10-07
Xerox Corporation
Continuous cardiac pulse rate estimation from multi-channel source video data
US20150005646A1
( en )
2013-06-26
2015-01-01
Massachusetts Institute Of Technology
Pulse detection from head motions in video
US8977347B2
( en )
2012-06-25
2015-03-10
Xerox Corporation
Video-based estimation