ABSTRACT
Abstract
An OCT system includes a machine learning (ML) model trained to receive a single OCT scan/image and provide an image translation and/or denoise function. The ML model may be based on a neural network (NN) architecture including a series of encoding modules in a contracting path followed by a series of decoding modules in an expanding path leading to an output convolution module. An intermediate error module determines a deep error measure, e.g., between a training output image and at least one encoding module and/or decoding module, and an error from the output convolution module is combined with the deep error measure. The NN may be trained using true averaged images as ground truth, training outputs. Alternatively, the NN may be trained using randomly selected, individual OCT images/scans as training outputs.
Description
PRIORITY
This application is a National Phase application under 35 U.S.C. § 371 of International Application No. PCT/EP2020/053515, filed Feb. 12, 2020, which claims priority to U.S. Provisional Application Ser. No. 62/805,835, filed Feb. 14, 2019, the contents of which are hereby incorporated by reference in their entirety.
FIELD OF INVENTION
The present invention is generally directed to the field of optical coherence tomography (OCT). More specifically, it is directed toward improving the quality of OCT scans/images, and other ophthalmic images.
BACKGROUND
Early diagnosis is critical for the successful treatment of various eye diseases. Optical imaging is the preferred method for non-invasive examination of the retina. Although age-related macular degeneration, diabetic retinopathy and glaucoma are known to be the major causes of vision loss, diagnosis is often not made until after damage has manifested itself. This is primarily due to the relatively poor resolution of some retinal imaging techniques. Therefore, a goal of advanced sensitive and accurate ophthalmic imaging and diagnostic tools is to provide imaging modalities able to resolve/display (e.g., for detection and monitoring) pathological variations of retinal microstructures at a pre-clinical stage of disease.
One such advanced ophthalmic imaging and diagnostic tool is the optical coherence tomography (OCT) system. Depending upon the technology used, OCT may provide an axial resolution in the range of about 1 to 15 μm, and a lateral resolution in the range of a few microns to a few tens of microns. As it would be understood, achieving higher resolutions typically requires extensive research, cost, and complexity.
Attempts to achieve improved image quality through software solutions have had limited success. For example, the Hessian-based vesselness filter, known in the art, may provide improved vascular connectivity, but it has been found to introduce imaginary (e.g., fictitious) structures not found in the original (e.g., true) scan. Because the Hessian-based vesselness filter is not faithful to the true vascular structure of the original scan, its use in examining ophthalmic images for pathologies is limited.
It is an object of the present invention to provide an OCT system with improved resolution.
It is an object of the present invention to provide a system and method for improving the imaging capability of an existing OCT, or OCT angiography (OCTA), system with minimal hardware modification to the OCT or OCTA system.
It is a further object of the present invention to provide a system and method for enhancing the image quality of pre-existing OCT images.
SUMMARY OF INVENTION
The above objects are met in an optical coherence tomography (OCT) system having a light source for generating a beam of light; a beam splitter having a beam-splitting surface for directing a first portion of the light into a reference arm and a second portion of the light into a sample arm; optics for directing the light in the sample arm to one or more locations on a sample; a detector for receiving light returning from the sample and reference arms and generating signals in response thereto; a processor for converting the signals into a first image and submitting the first image to an image translation module that translates the first image to a second image characterized by one or more of decreased jitter and minimized creation of fictional structures as compared to the first image; and an output display for displaying an output image based on the second image. The image translation module preferably includes a machine learning module (e.g., a deep learning, neural network) trained using a set of training input images and a target set of training output images, where the training input images are generated independent of the training output images. For example, the training input are not based on training output images with added, known types of noise (e.g., Gaussian noise, Poisson noise, speckle noise, salt & pepper noise, etc.). Nonetheless, the training output images may be of higher image quality than the training input images so that the trained machine learning module (e.g., in operation) produces second images that are higher quality representations of the first images. For example, individual training output images may be constructed by averaging a set of training input images. Alternatively, the training input images and training output images may be generated by OCTs of different modalities, where an OCT modality capable of creating higher quality images is used to generate training output images, and an OCT modality that generates lower quality images is used to generate input training images. For instance, an adaptive optics OCT system may be used to generate the training output images, and one or more of a non-adaptive OCT system (e.g., a time domain OCT, frequency-domain (FD) OCT, spectral domain (SD) OCT, and/or swept source (SS) OCT) may be used to generate the training input image. In this manner, the translation module effectively converts OCT images of a first modality to images resembling those generated by an OCT of a second, different modality.
Typically, such an image translation module would require a large number of training samples (e.g. a larger number of training input image and training output images) for effective deep learning. This may not be a problem when taking images of nature scenes, for example, but it is problematic when attempting to gather a large library of ophthalmic images (particularly a large number of OCT images) for training a machine learning model. Creating a large library of ophthalmic images for deep learning can be economically prohibitive. The present invention provides a novel neural network (NN) architecture that provides deep learning results with a smaller library of training samples than typical. Additionally, the present neural network deviates from prior known image translation neural network architectures to define a compact form with fewer learning layers, or modules. The present NN is suitable for multiple imaging modalities, e.g., different types of ophthalmic images such as images from fundus imaging systems and OCT systems, but is herein illustratively described within the context of OCT images. Thus, the present NN may be recited as incorporated within an OCT system, but it is to be understood that the present NN architecture may also be incorporated into other types of ophthalmic imaging systems and may be applied to the processing of other types of ophthalmic images. For example, the present NN may be used to process, and improve the image quality of, an existing library of previously generated ophthalmic images (e.g., a memory store of preexisting OCT images or fundus images).
The known U-Net architecture is traditionally limited to image classification and image segmentation. The present NN architecture is based on the U-Net, but extends it functionality to image translation. Previously, a U-Net would be combined with another NN, such as an adversarial network (GAN) to implement image translation. In this prior art case, the U-Net would provide image segmentation, and the GAN would provide image translation. The present architecture, however, builds on the basic U-Net architecture so that it provides image translation directly without the need for a GAN, or any other secondary NN, to achieve image translation. The present NN architecture may include: an input module for receiving a first image (e.g., an input OCT or Fundus image); a contracting path following the input module, where the contracting path includes multiple of encoding modules with each encoding module having a convolution stage (e.g., one or more convolution layers), an activation function, and a max pooling operation; an expanding path following the contracting path, where the expanding path includes multiple decoding modules (e.g., one or more decoding layers) with each decoding module concatenating its current value with that of a corresponding encoding module; an output convolution module excluding a pooling layer and excluding an activation function, where the output convolution module receives the output from the last decoding module in the expanding path. In a traditional U-Net, each decoding module in the expanding path would include an activation function (e.g., a sigmoid) layer. In the present invention, however, one or more, and preferably all, decoding modules in the expanding path do not have any activation layer. This lack of activation layer(s) in the decoding module(s) aids the present architecture to achieve image translation functionality. In a traditional neural network, the output from the output convolution module would typically be compared with a target training output image to determine a loss error, and this loss error would be fed back through the NN (e.g., in a back-propagation process) to adjust the NN's weights and biases so as to produce an output with smaller error in a subsequent training cycle. The present invention deviates from this practice. The present NN further includes at least one intermediate error module that determines an error measure for at least one encoding module and/or one decoding module. This intermediate error module takes the current results of its at least one encoding module and/or one decoding module, upscale the current results to the resolution of a current training output image, and compares it with the current training output image to define one or more deep error measures. The additional deep error measures are then combined with the loss error from the output convolution module to define a total loss error for the system that can then be fed back through the NN to adjust its internal weights and biases. These multiple sources of error may be combined, for example, by direct addition, by a weighted combination, and/or by averaging. By introducing the training output image into various internal stages of the NN, the NN is prevented from deviating too far off the target output and thereby also assists in achieving images translation.
The present NN may be used for additional purposes, such reduction of noise artifacts in ophthalmic images. It is to be understood, however, that other NN architectures may likewise be used to implement some of the present image translation and noise reduction functionalities.
The above objects are further met in an ophthalmic imaging system or method (e.g., a fundus imaging system or OCT system) for reducing noise artifacts in an ophthalmic image, or for generating ophthalmic images of reduced noise artifacts. The system or method may include using a processor for acquiring a first ophthalmic image, submitting the first ophthalmic image to an image modification module that creates a second ophthalmic image based on the first image and having reduced noise artifacts as compared with the first ophthalmic image; and displaying on an electronic display an output image based on the second image. Preferably, the image modification module includes a neural network whose training includes: collecting multiple test ophthalmic images of at least one sample (e.g., an eye), the collected test ophthalmic images being noisy images; randomly selecting one of the test ophthalmic images as a training output image; randomly selecting one or more of the remaining test ophthalmic images as a training set of training input images; and separately and individually submitting each training input image to the neural network and providing the training output image as a target output for the neural network.
Other objects and attainments together with a fuller understanding of the invention will become apparent and appreciated by referring to the following description and claims taken in conjunction with the accompanying drawings.
The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Any embodiment feature mentioned in one claim category, e.g. system, can be claimed in another claim category, e.g. method, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However, any subject matter resulting from a deliberate reference back to any previous claims can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings wherein like reference symbols/characters refer to like parts:
FIG. 1 illustrates a neural network architecture in accord with the present invention.
FIG. 2 illustrates a process for creating a ground truth, true averaged image.
FIG. 3 illustrates a method for defining a set of training sample pairs for training a neural network in accord with the present invention.
FIG. 4 illustrates the training of a neural network in accord with the present invention.
FIG. 5 provides another illustration of the training of a (e.g., convolutional) neural network in accord with the present invention.
FIG. 6 illustrates an example operation of a convolutional neural network, either with live data input after training or with test data input during an evaluation phase.
FIG. 7 illustrates the implementation of a trained neural network into a workflow for averaging multiple scans/images similar to the workflow of FIG. 8 .
FIG. 8 illustrates a patching-based scheme in accord with the present invention for training a neural network using pairs of corresponding image patches.
FIG. 9 illustrates the use of pairs of corresponding patches to train a neural network in accord with the present invention.
FIG. 10 illustrates the operation of a neural network trained on pairs of image patches in accord with the present invention.
FIG. 11 illustrates the use of a set of individual AO-OCT images produced by an AO-OCT system to train a neural network in accord with the present invention.
FIG. 12 illustrates the operation of a neural network trained to output images/scans having characteristics of an AO-OCT.
FIGS. 13 A and 13 B compare a true, averaged image with a âpredicted averagedâ (e.g., averaged-looking) image produced with a trained neural network in accord with the present invention.
FIG. 14 illustrates another application of the present invention where a 6 mm input image/scan is submitted to a trained neural network to produce a predicted averaged-looking 6 mm image.
FIG. 15 shows an example ophthalmic (e.g., en face) image that is input to a NN ML model that is trained on averaged images and/or averaged patches.
FIG. 16 shows the resultant output image produced by the trained NN ML model in response to receiving the input image of FIG. 21 .
FIGS. 17 A and 17 B illustrate that the presently trained NN model learns the structure of vessels of an eye.
FIG. 18 shows the result of inputting an image of a diseased eye to a neural network trained in accord with the present invention.
FIG. 19 illustrates the training of a neural network to denoise OCT scans/images in acc
PRIORITY
This application is a National Phase application under 35 U.S.C. § 371 of International Application No. PCT/EP2020/053515, filed Feb. 12, 2020, which claims priority to U.S. Provisional Application Ser. No. 62/805,835, filed Feb. 14, 2019, the contents of which are hereby incorporated by reference in their entirety.
FIELD OF INVENTION
The present invention is generally directed to the field of optical coherence tomography (OCT). More specifically, it is directed toward improving the quality of OCT scans/images, and other ophthalmic images.
BACKGROUND
Early diagnosis is critical for the successful treatment of various eye diseases. Optical imaging is the preferred method for non-invasive examination of the retina. Although age-related macular degeneration, diabetic retinopathy and glaucoma are known to be the major causes of vision loss, diagnosis is often not made until after damage has manifested itself. This is primarily due to the relatively poor resolution of some retinal imaging techniques. Therefore, a goal of advanced sensitive and accurate ophthalmic imaging and diagnostic tools is to provide imaging modalities able to resolve/display (e.g., for detection and monitoring) pathological variations of retinal microstructures at a pre-clinical stage of disease.
One such advanced ophthalmic imaging and diagnostic tool is the optical coherence tomography (OCT) system. Depending upon the technology used, OCT may provide an axial resolution in the range of about 1 to 15 μm, and a lateral resolution in the range of a few microns to a few tens of microns. As it would be understood, achieving higher resolutions typically requires extensive research, cost, and complexity.
Attempts to achieve improved image quality through software solutions have had limited success. For example, the Hessian-based vesselness filter, known in the art, may provide improved vascular connectivity, but it has been found to introduce imaginary (e.g., fictitious) structures not found in the original (e.g., true) scan. Because the Hessian-based vesselness filter is not faithful to the true vascular structure of the original scan, its use in examining ophthalmic images for pathologies is limited.
It is an object of the present invention to provide an OCT system with improved resolution.
It is an object of the present invention to provide a system and method for improving the imaging capability of an existing OCT, or OCT angiography (OCTA), system with minimal hardware modification to the OCT or OCTA system.
It is a further object of the present invention to provide a system and method for enhancing the image quality of pre-existing OCT images.
SUMMARY OF INVENTION
The above objects are met in an optical coherence tomography (OCT) system having a light source for generating a beam of light; a beam splitter having a beam-splitting surface for directing a first portion of the light into a reference arm and a second portion of the light into a sample arm; optics for directing the light in the sample arm to one or more locations on a sample; a detector for receiving light returning from the sample and reference arms and generating signals in response thereto; a processor for converting the signals into a first image and submitting the first image to an image translation module that translates the first image to a second image characterized by one or more of decreased jitter and minimized creation of fictional structures as compared to the first image; and an output display for displaying an output image based on the second image. The image translation module preferably includes a machine learning module (e.g., a deep learning, neural network) trained using a set of training input images and a target set of training output images, where the training input images are generated independent of the training output images. For example, the training input are not based on training output images with added, known types of noise (e.g., Gaussian noise, Poisson noise, speckle noise, salt & pepper noise, etc.). Nonetheless, the training output images may be of higher image quality than the training input images so that the trained machine learning module (e.g., in operation) produces second images that are higher quality representations of the first images. For example, individual training output images may be constructed by averaging a set of training input images. Alternatively, the training input images and training output images may be generated by OCTs of different modalities, where an OCT modality capable of creating higher quality images is used to generate training output images, and an OCT modality that generates lower quality images is used to generate input training images. For instance, an adaptive optics OCT system may be used to generate the training output images, and one or more of a non-adaptive OCT system (e.g., a time domain OCT, frequency-domain (FD) OCT, spectral domain (SD) OCT, and/or swept source (SS) OCT) may be used to generate the training input image. In this manner, the translation module effectively converts OCT images of a first modality to images resembling those generated by an OCT of a second, different modality.
Typically, such an image translation module would require a large number of training samples (e.g. a larger number of training input image and training output images) for effective deep learning. This may not be a problem when taking images of nature scenes, for example, but it is problematic when attempting to gather a large library of ophthalmic images (particularly a large number of OCT images) for training a machine learning model. Creating a large library of ophthalmic images for deep learning can be economically prohibitive. The present invention provides a novel neural network (NN) architecture that provides deep learning results with a smaller library of training samples than typical. Additionally, the present neural network deviates from prior known image translation neural network architectures to define a compact form with fewer learning layers, or modules. The present NN is suitable for multiple imaging modalities, e.g., different types of ophthalmic images such as images from fundus imaging systems and OCT systems, but is herein illustratively described within the context of OCT images. Thus, the present NN may be recited as incorporated within an OCT system, but it is to be understood that the present NN architecture may also be incorporated into other types of ophthalmic imaging systems and may be applied to the processing of other types of ophthalmic images. For example, the present NN may be used to process, and improve the image quality of, an existing library of previously generated ophthalmic images (e.g., a memory store of preexisting OCT images or fundus images).
The known U-Net architecture is traditionally limited to image classification and image segmentation. The present NN architecture is based on the U-Net, but extends it functionality to image translation. Previously, a U-Net would be combined with another NN, such as an adversarial network (GAN) to implement image translation. In this prior art case, the U-Net would provide image segmentation, and the GAN would provide image translation. The present architecture, however, builds on the basic U-Net architecture so that it provides image translation directly without the need for a GAN, or any other secondary NN, to achieve image translation. The present NN architecture may include: an input module for receiving a first image (e.g., an input OCT or Fundus image); a contracting path following the input module, where the contracting path includes multiple of encoding modules with each encoding module having a convolution stage (e.g., one or more convolution layers), an activation function, and a max pooling operation; an expanding path following the contracting path, where the expanding path includes multiple decoding modules (e.g., one or more decoding layers) with each decoding module concatenating its current value with that of a corresponding encoding module; an output convolution module excluding a pooling layer and excluding an activation function, where the output convolution module receives the output from the last decoding module in the expanding path. In a traditional U-Net, each decoding module in the expanding path would include an activation function (e.g., a sigmoid) layer. In the present invention, however, one or more, and preferably all, decoding modules in the expanding path do not have any activation layer. This lack of activation layer(s) in the decoding module(s) aids the present architecture to achieve image translation functionality. In a traditional neural network, the output from the output convolution module would typically be compared with a target training output image to determine a loss error, and this loss error would be fed back through the NN (e.g., in a back-propagation process) to adjust the NN's weights and biases so as to produce an output with smaller error in a subsequent training cycle. The present invention deviates from this practice. The present NN further includes at least one intermediate error module that determines an error measure for at least one encoding module and/or one decoding module. This intermediate error module takes the current results of its at least one encoding module and/or one decoding module, upscale the current results to the resolution of a current training output image, and compares it with the current training output image to define one or more deep error measures. The additional deep error measures are then combined with the loss error from the output convolution module to define a total loss error for the system that can then be fed back through the NN to adjust its internal weights and biases. These multiple sources of error may be combined, for example, by direct addition, by a weighted combination, and/or by averaging. By introducing the training output image into various internal stages of the NN, the NN is prevented from deviating too far off the target output and thereby also assists in achieving images translation.
The present NN may be used for additional purposes, such reduction of noise artifacts in ophthalmic images. It is to be understood, however, that other NN architectures may likewise be used to implement some of the present image translation and noise reduction functionalities.
The above objects are further met in an ophthalmic imaging system or method (e.g., a fundus imaging system or OCT system) for reducing noise artifacts in an ophthalmic image, or for generating ophthalmic images of reduced noise artifacts. The system or method may include using a processor for acquiring a first ophthalmic image, submitting the first ophthalmic image to an image modification module that creates a second ophthalmic image based on the first image and having reduced noise artifacts as compared with the first ophthalmic image; and displaying on an electronic display an output image based on the second image. Preferably, the image modification module includes a neural network whose training includes: collecting multiple test ophthalmic images of at least one sample (e.g., an eye), the collected test ophthalmic images being noisy images; randomly selecting one of the test ophthalmic images as a training output image; randomly selecting one or more of the remaining test ophthalmic images as a training set of training input images; and separately and individually submitting each training input image to the neural network and providing the training output image as a target output for the neural network.
Other objects and attainments together with a fuller understanding of the invention will become apparent and appreciated by referring to the following description and claims taken in conjunction with the accompanying drawings.
The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Any embodiment feature mentioned in one claim category, e.g. system, can be claimed in another claim category, e.g. method, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However, any subject matter resulting from a deliberate reference back to any previous claims can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings wherein like reference symbols/characters refer to like parts:
FIG. 1 illustrates a neural network architecture in accord with the present invention.
FIG. 2 illustrates a process for creating a ground truth, true averaged image.
FIG. 3 illustrates a method for defining a set of training sample pairs for training a neural network in accord with the present invention.
FIG. 4 illustrates the training of a neural network in accord with the present invention.
FIG. 5 provides another illustration of the training of a (e.g., convolutional) neural network in accord with the present invention.
FIG. 6 illustrates an example operation of a convolutional neural network, either with live data input after training or with test data input during an evaluation phase.
FIG. 7 illustrates the implementation of a trained neural network into a workflow for averaging multiple scans/images similar to the workflow of FIG. 8 .
FIG. 8 illustrates a patching-based scheme in accord with the present invention for training a neural network using pairs of corresponding image patches.
FIG. 9 illustrates the use of pairs of corresponding patches to train a neural network in accord with the present invention.
FIG. 10 illustrates the operation of a neural network trained on pairs of image patches in accord with the present invention.
FIG. 11 illustrates the use of a set of individual AO-OCT images produced by an AO-OCT system to train a neural network in accord with the present invention.
FIG. 12 illustrates the operation of a neural network trained to output images/scans having characteristics of an AO-OCT.
FIGS. 13 A and 13 B compare a true, averaged image with a âpredicted averagedâ (e.g., averaged-looking) image produced with a trained neural network in accord with the present invention.
FIG. 14 illustrates another application of the present invention where a 6 mm input image/scan is submitted to a trained neural network to produce a predicted averaged-looking 6 mm image.
FIG. 15 shows an example ophthalmic (e.g., en face) image that is input to a NN ML model that is trained on averaged images and/or averaged patches.
FIG. 16 shows the resultant output image produced by the trained NN ML model in response to receiving the input image of FIG. 21 .
FIGS. 17 A and 17 B illustrate that the presently trained NN model learns the structure of vessels of an eye.
FIG. 18 shows the result of inputting an image of a diseased eye to a neural network trained in accord with the present invention.
FIG. 19 illustrates the training of a neural network to denoise OCT scans/images in accord with the present invention.
FIG. 20 shows an example live input en face image to a trained neural network in accord with the present invention, and the resultant denoised en face image.
FIGS. 21 A and 21 B show two separate examples of a live B-scan input to trained neural network, and their respectively output denoised B-scans.
FIG. 22 illustrates a generalized frequency domain optical coherence tomography (FD-OCT) system used to collect 3-D image data of an eye suitable for use with the present invention.
FIG. 23 shows an example of an en face vasculature image.
FIG. 24 illustrates a generalized adaptive optics optical coherence tomography (AO-OCT) system comprised of an AO subsystem and an OCT/OCTA subsystem.
FIG. 25 illustrates an example of a multilayer perceptron (MLP) neural network.
FIG. 26 shows a simplified neural network consisting of an input layer, a hidden layer, and an output layer.
FIG. 27 illustrates an example convolutional neural network architecture.
FIG. 28 illustrates an example U-Net architecture.
FIG. 29 illustrates an example computer system (or computing device or computer device).
DESCRIPTION OF THE PREFERRED EMBODIMENTS
There are several different types of ophthalmic images. For example, ophthalmic images may be created by fundus photography, fluorescein angiography (FA), fundus auto-fluorescence (FAF), optical coherence tomography (OCT), OCT angiography (OCTA), ocular ultrasonography. Each provides different information (or information gathered in a different manner) of an eye, and each may emphasize a different aspect of the eye. Ophthalmic images, in general, are an integral part of diagnosing a particular eye-related malady, and their effectiveness is dependent upon their ability to produce good quality images, e.g., images with sufficient resolution, focus, magnification, and signal-to-noise ratio (SNR). Examples of fundus imagers are provided in U.S. Pat. Nos. 8,967,806 and 8,998,411, examples of OCT systems are provided in U.S. Pat. Nos. 6,741,359 and 9,706,915, and examples of an OCTA imaging system may be found in U.S. Pat. Nos. 9,700,206 and 9,759,544, all of which are herein incorporated in their entirety by reference. The present invention seeks to use machine learning (ML) techniques (e.g. decision tree learning, support vector machines, artificial neural networks (ANN), deep learning, etc.) to provide an ophthalmic image-translation tool and/or a denoising tool that improves the quality of an ophthalmic image produced by any of a select type of imaging systems. The following exemplary embodiments describe the invention as applied to OCT systems, but it is to be understood that the present invention may also be applied to other ophthalmic imaging modalities, such as fundus imaging.
Furthermore, the present invention provides for translating OCT images of a first OCT modality into images of a second OCT modality. In particular, there are different types of OCT/OCTA imaging modalities, such as time domain (TD) OCT/OCTA and Frequency-domain (FD) OCT/OCTA. Other more specific OCT modalities may include Spectral Domain (SD) OCT/OCTA, Swept Source (SS) OCT/OCTA, Adaptive optics (AO) OCT/OCTA, etc., and each has its advantages and disadvantages. For example, AO-OCT generally provides better image quality and higher resolution than more conventional OCT systems, but is much more complicated, requires expensive components, and generally provides a much-reduced field-of-view (FOV) than conventional OCT systems. The present invention provides an OCT system of a first modality that may selectively produce OCT images that are translated into (e.g., mimic, simulate, or resemble) those of another, different modality. For example, an SD-OCT may be made to provide images that resemble those provided by an SS-OCT or an AO-OCT. Alternatively, the present invention may provide an OCT system that translates a single captured OCT image of a given sample region of an eye into an image that resembles that produce by an averaging of multiple OCT images of the same sample region without the need for repeated scanning of the same sample region.
Additionally, the present invention provides an improved method/tool for reducing noise (e.g., denoising) in an ophthalmic image, e.g., an OCT image. The present denoising tool may be applied separately, or in addition to, the image-translation tool. That is, the present denoising tool may be applied independent of, or in combination with, the present image-translation tool. Furthermore, the present denoising technique may optionally be combined with the training of the present image translation tool.
The present ophthalmic image enhancement tool(s) may be incorporated into an ophthalmic imaging device such as a fundus imager or OCT system, or may be provided as a network-accessible service. For example, the image enhancement tool(s) may be implemented as an application executable on a mobile device, such as computing tablet or smart phone, that accesses a remote host over the Internet, and the remote host provides the present invention as an ophthalmic image enhancing service. In this manner, higher computing requirements may be offloaded from the mobile device (or OCT system) onto the remote host. Alternatively, the ophthalmic image enhancement tool may be provided entirely on the Internet as a website accessible over the Internet by use of a web browser. In this manner a physician or technician may make use of the present ophthalmic image enhancement tool from anywhere using any device having Internet access and capable of running a web browser.
For illustration purposes, various imaging systems, including OCT and AO-OCT systems, are described below with reference to FIGS. 22 - 24 .
The following embodiments describe a system and method for providing an OCT/OCTA system capable of converting an OCT/OCTA image of first modality into another (preferably higher quality and/or higher resolution) OCT image of a second modality. For example, the converted image may have characteristics of an image created by averaging multiple OCT/OCTA images (hereinafter termed an average-simulating image) and/or an image created by AO-OCT system (hereinafter termed an AO-simulating image) without the difficulties associated with the generation of true averaged OCT/OCTA images or true AO-OCT/OCTA images. Alternatively, or in addition to the image translation capability, the present system may further provide a noise reduction utility. In some embodiments, these added capabilities are provided by a data processing module based on machine learning. Various types of machine learning techniques are envisioned by the present invention (e.g. nearest neighbor, naive Bayes, decision tree learning, support vector machines, artificial neural networks (ANN), deep learning, etc.), but a currently preferred implementation is based on neural networks (NN), and in particular is based on a new NN architecture that builds on a U-Net architecture and provides for simplified training, a reduced number of network levels, and a smaller training set than typical.
Various neural network architectures, such as the multilayer perceptron (MLP) neural network, convolutional neural network (CNN), and U-Net, are discussed below with reference to FIGS. 25 - 28 . Optionally, the present invention may be implemented using any one, or a combination, of these neural networks, but a preferred embodiment may use a specialized image-translation neural network architecture based on the U-Net architecture to implement an image translation function and/or a denoising function.
A typical neural network (NN) may have multiple hidden layers and be made up of neurons having learnable weights and biases, where each neuron may receive inputs, perform an operation and be optionally followed by a non-linearity (e.g., an activation function). Typically, a deep learning NN requires a large training set and is not well suited to manipulate images larger than a few pixels. A convolutional neural network (CNN) is similar to a (deep learning) neural network (NN), but the CNN may be optimized to work with images more efficiently. The CNN assumes that data close together is more related than data far apart, which permits making forward functions more efficient and reducing the number of parameters. CNNs, however, may still require a large training set. As it would be understood by one versed in the art, obtaining a larger number of relevant medical images (particularly ophthalmic images) to compile a large training set can be problematic for various economic and regulatory reasons. The U-Net neural network architecture may use a smaller training set than a traditional NN, and like a CNN, the U-Net may also be optimized for training on images. The primary use of the U-Net has traditionally been for image segmentation (e.g., identifying the shape of foreground objects in an image) as a pre-processing step for further image processing. For example, a U-Net may be coupled as a preprocessing step to an adversarial network (GAN) to implement image translation. In this case, the GAN would receive segmentation information output from U-Net, and the GAN would apply image translation to the segmented items. The present invention provides an architecture based on the U-Net that eliminates the need for a GAN, such that instead of (or in addition to) providing image segmentation, the present U-Net-based architecture provides image translation, directly.
With reference to FIG. 1 , the present architecture builds on the typical U-Net architecture (e.g., see for example, FIG. 28 ) and includes an input module (or input layer or input stage) 12 that receives an input Uin (e.g., input image or image patch) of any given size (e.g., 128 by 128 pixels in size). For illustration purposes, the image size at any stage, or layer, is indicated within a box that represents the image, e.g., box 12 encloses number â128Ã128â to indicate that input image Uin is comprised of 128 by 128 pixels. The input image may be an OCT/OCTA en face or B-scan image. It is to be understood, however, that the input may be of any size and dimension. For example, the input image may be an RGB color image, as illustrated in FIG. 27 . The input image undergoes a series of processing layers, each of which is illustrated with exemplary sizes, but these sizes are for illustration purposes only and would depend upon multiple factors, such as the size of the image, convolution filter, pooling stages, etc. The present architecture includes a contracting path (herein exemplarily comprised of five encoding modules 14 , 16 , 18 , 20 , and 22 ) followed by an expanding path (similarly comprised of five corresponding decoding modules 24 , 26 , 28 , 30 , and 34 ), and five copy-and-crop links (e.g., skip connections) CL 1 to CL 5 between corresponding modules/stages that copy the output of one encoding module in the contracting path and concatenates it to (e.g., appends it to the back of) the up-converted input of a correspond decoding module in the expanding path. The contracting path is similar to an encoder, and generally captures context (or feature) information by the use of feature maps. Each encoding module in the contracting path may include one or more convolutional layers, illustratively indicated by an asterisk symbol â*â, which may be followed by a max pooling layer (e.g., embodied within a DownSampling layer). For example, input image, Uin, is illustratively shown to undergoes two convolution layers, each producing 32 feature maps. As it would be understood, each convolution kernel produces a feature map (e.g., the output from a convolution operation with a given kernel is an image typically termed a âfeature mapâ). For example, input Uin undergoes a first convolution that applies 32 convolution kernels (not shown) to produce an output 36 consisting of 32 respective feature maps. However, as it is known in the art, the number of feature maps produced by a convolution operation may be adjusted (up or down). For example, the number of feature maps may be reduced by averaging groups of feature maps, dropping some feature maps, or other known method of feature map reduction. In the present case, the first convolution is followed by a second convolution whose output 38 is limited to 32 feature maps. Another way to envision feature maps may be to think of the output of a convolution layer as a 3D image whose 2D dimension is given by the listed X-Y planar pixel dimension (e.g., 128Ã128 pixels), and whose depth is given by the number of feature maps (e.g., 32 planar images deep). Following this analogy, the output 38 of encoding module 14 may be described as a 128Ã128Ã32 image. The output 38 from the second convolution then undergoes a pooling operation, which reduces the 2D dimension of each feature map (e.g., the X and Y dimensions may each be reduced by half). The pooling operation may be embodied within the DownSampling operation, as indicated by a downward arrow 40 . Several pooling methods, such as max pooling, are known in the art and the specific pooling method is not critical to the present invention. The contracting path thus forms a convolutional network, herein shown with five encoding modules (or stages or layers) 14 , 16 , 18 , 20 , and 22 . As is typical of convolutional networks (e.g., see FIGS. 27 and 28 ), each encoding module preferably provides at least one convolution stage, followed by an activation function (e.g., a rectified linear unit, ReLU or sigmoid layer), not shown, and a max pooling operation. Generally, an activation function introduces non-linearity into a layer (which helps avoid overfitting issues), receives the results of a layer, and determines whether to âactivateâ the output (e.g., determines whether the value of a given node meets a criteria to have an output forwarded to a next layer/node).
The expanding path is similar to a decoder, and among other things, may provide localization and spatial information for the results of the contracting path. The expanding path is herein shown to have five decoding modules, 24 , 26 , 28 , 30 , and 34 , where each decoding module concatenates its current up-converted input with the output of a corresponding encoding module. For example, the up-converted input 42 of decoding module 24 is shown concatenated with the output 44 of corresponding encoding module 22 . More specifically, output 44 (whose dimensions are 8Ã8Ã512) may be appended to up-converted input 42 (whose dimensions are 8Ã8Ã1024) to define a concatenated image whose dimensions are 8Ã8Ã1536. In this manner, the feature information (from encoding module 22 ) is combined with spatial information (from decoding module 24 ). This combining of feature and spatial information continues in the expanding path through a sequence of up-convolutions (e.g., UpSampling or transpose convolutions or deconvolutions) and concatenations with high-resolution features from the contracting path (e.g., via CL 1 to CL 5 ). The output of a deconvolution layer is concatenated with the corresponding (optionally cropped) feature map(s) from the contracting path, followed by two (or more) convolutional layers. Between the contracting path and the expanding path is typically a bottleneck module, BNK, which may consist of two, or more, convolutional layers.
The present architecture of FIG. 1 is based on the U-Net (see FIG. 28 ), but has some distinguishing features. Firstly, the expanding path in the present architecture may lack any activation function. That is, one or more, and preferably all, decoding modules 24 , 26 , 28 , 30 , and/or 34 in the expanding path may lack any activation layer, e.g., sigmoid layer. This lack of activation function(s) in the expanding path may aid the present architecture to achieve image translation functionality. Secondly, the present neural network incorporates a novel training architecture. As it is known in the art, a neural network may be trained by back-propagation wherein, during a training cycle, a current neural network output (e.g. Uout) is compared with a current training output image Tout to determine a current loss error E 1 . Typically, this loss error E 1 would be fed back through the neural network to adjust its internal weights and biases so as to produce a better output with smaller error in a subsequent training cycle. The present architecture, however, incorporates deep supervision links to determine a deep error from any stage in the contracting path and/or the expanding path and combines this deep error with the loss error at the output module/layer 50 to define a total loss error for the system. The deep supervision may be in the form of an intermediate error module (e.g., EM 1 /EM 2 ) that determines a deep error measure for at least one encoding module and/or at least one decoding module. Optionally, the selected encoding module(s) and decoding module(s) may be randomly selected, or may correspond to each other according to their corresponding copy-and-crop link(s). The current state of a selected encoding module(s) and/or decoding module(s) is compared with a current training output image to assure that the output does not deviate too much from the input. This results in faster convergence, and thus a shorter training time. For example, during training, the current output from encoding stage 18 may be up-converted to the resolution of the of the current training output image Tout, and compared with Tout to determine a deep error E 2 . Similarly, during training, the current output from the decoding stage 28 may be up-converted to the resolution of the of the current training output image Tout, and compared with Tout to determine a deep error E 3 . Deep errors E 2 and E 3 are then combined with the error E 1 , such as by a weighted linear combination. Optionally, the input Uin may be fed forward to (e.g., combined with) the output Uout prior to determining loss error E 1 . For example, input Uin may be added to output Uout (such as by use of a feedforward link from Uin to Uout, not shown) to define a combination network output, Uin+Uout, (or a weighted combination of both) prior to determining error loss E 1 . The combination network output, Uin+Uout, may take the place of Uout in FIG. 1 and be compared with the current training output image Tout to determine the current loss error E 1 . By combining the input with the output, the network does not need to âlearnâ to do nothing when the input already matches the desired output (Tout), (e.g., makes the Identity âInput=Outputâ trivial). In practice, this may speed up training and produce better results. Additionally, the present architecture may use a customized loss function that determines an Lâ1 loss (e.g., a sum of absolute differences) and an Lâ2 loss (e.g., a sum of square differences) when comparing Uout with Tout, and combines the two losses (e.g., by weighted summation) to define E 1 . Preferably, the Lâ1 loss is weighted more heavily than the Lâ2 loss when defining E 1 to ensure faithful representation of the original image/scan. Error measures E 2 and E 3 may be determined by Lâ1 loss, Lâ2 loss, or a combination of both. Error measures E 1 , E 2 , and/or E 3 may be weighted by a gain (e.g. α 1 , α 2 , and/or α 3 respectively). For example, E 2 may be weighted by a gain of α 2 =0.2, E 3 may be weighted by a gain of α 3 =0.2, and E 1 (which may be a combination of Lâ1 loss and Lâ2 loss) may be weighted by a gain of α 1 =0.6, such that all the gains sum up to 1 and the loss error of the output module (e.g., E 1 ) is weighted more heavily. The resultant total loss error may then be fed back through the neural network in a back-propagation process. Using deep supervision limits the output so that it does not create (or substantially limits the creation of) fictional structures (e.g. structures not found in the input image). When dealing with medical images, avoiding the creation of fictitious structures in an output image is of premium importance to clinicians. Thus, the present deep supervision architecture makes for faster convergence and better feature representation than a traditional U-net structure. It is to be understood that the present deep supervision architecture would be used in the training of a ML model in accord with the present invention, but once the ML model is trained, the weights and biases of the various stages/modules would be set (e.g., fixed) and it may not be necessary to use the deep supervision portion during of the architecture in an operational stage of the trained ML model.
The present NN architecture of FIG. 1 may be used to train an ML model to accomplish multiple tasks. A first task may be image translation and a second task may be noise reduction. For ease of discussion, each task is discussed separately, as follows.
Image Translation
Individual OCT/OCTA scans suffer from jitter, drop outs, and speckle noise, among other issues. These issues can affect the quality of en face images both qualitatively and quantitatively, as they are used in the quantification of vasculature density. The present invention seeks to improve the quality of an ophthalmic image by use of a trained neural network. As is explained below, this requires multiple training pair sets, e.g., a training input image paired with a corresponding ground truth, target training output image. A difficulty with using deep learning is obtaining ground truth outputs for use in the training set. The quality (e.g. vessel continuity, noise level) of true averaged images is generally far superior to that of an individual scan. Thus one approached in the present invention is to translate a single ophthalmic input image to an average-simulating image, e.g., an image that has characteristics of a true averaged image. In this approach, true averaged images are used as ground truth images (e.g., as training output images) in the training of the present neural network. Another approach is to translate an ophthalmic input image of a first modality to an output ophthalmic image simulating a different modality that typically has a higher quality. For example, AO-OCT images may be used as ground truth, training output images to train a neural network to produce AO-simulating image. For ease of discussion, much of the following discussion describes the use of true averaged images as ground truth, target output images in a training set, but unless otherwise stated, it is be understood that a similar description applies to using AO-OCT images (or other higher quality and/or higher resolution images) as ground truth target outputs in a training set.
FIG. 2 illustrates a process for creating a ground truth, true averaged image Avg 1 . An ophthalmic imaging system such as an OCT/OCTA system 13 (such as illustrated in FIGS. 22 and 24 ) captures an image set S 1 consisting of n individual images/scans S 1 S 1 to S 1 Sn of the same region of a sample (e.g., an eye). For example, the first scan of image set S 1 is labeled S 1 S 1 , and the n th scan of image set S 1 is labeled S 1 Sn. Similarly, if a second image set S 2 (not shown) of p scans were collected, then the individual scans in second image set S 2 may be labeled S 2 S 1 to S 2 Sp. Irrespective, the individual scans (e.g., S 1 S 1 to S 1 Sn) of an image set may optionally be subjected to image pre-processing (e.g., illumination correction, blur and focus correction, filtering and noise removal, edge enhancement, etc.) by an image pre-processor module/block 15 before being submitted to an image registration (e.g. alignment) module/block 17 , which essentially aligns the pre-processed scans within an image set (e.g., S 1 ). The individual scans (S 1 S 1 to S 1 Sn) may be registered by identifying characteristic features (e.g., polygon shape descriptors (e.g., vessel structures), spectra descriptor (e.g., SIFT, SURF), etc.) within each individual scan, identifying matching characteristic features within multiple individual scans, and aligning the individual scans by aligning their respectively matched characteristic features. This process generates multiple image alignment settings (e.g., registration parameters). The aligned scans/images are then averaged (e.g., by image averaging module/block 19 ) to produce true averaged image Avg 1 . Registering and averaging the individual scans/images within each of multiple image sets, generates a respective averaged image for each image set, where each averaged image is of higher quality than the constituent images within its corresponding image set. However, generating a true averaged image takes considerably more time, which limits its application. The present invention may use multiple image sets of individual scans and their corresponding averaged images to train a neural network.
FIG. 3 illustrates a method for defining a set of training sample pairs. In the present example, each of the individual scans S 1 S 1 to S 1 Sn is separately paired with their corresponding true averaged image Avg 1 to define n separate training pairs TP 1 to TPn, where each of scan S 1 to Sn is a separate training input to a neural network, and their corresponding true averaged image Avg 1 is each input's corresponding target training output. As it would be understood, multiple training pair sets may be defined by obtaining multiples image sets (e.g., S 1 , S 2 , . . . , Sm) of various regions of an eye, and averaging the constituent scans of each image set to define a corresponding true averaged image for each image set, as illustrated in FIG. 2 .
FIG. 4 illustrates the training of a neural network 25 (such as that shown in FIG. 1 ) in accord with the present invention. In the present example, multiple training pair sets TS 1 to TSm are collected. Each training set consist of a set of constituent OCT/OCTA scans 21 and used to define their corresponding true averaged image 23 . Each training set (TS 1 to TSm) may be submitted to neural network 25 for training, as described below. Optionally, pre-processor module/block 15 may be omitted and neural network 25 may be trained on raw data (scans/image), instead of pre-processed (e.g., filtered) data.
FIG. 5 provides another illustration of the training of a (convolutional) neural network 25 in accord with the present invention. As before, each training pair TP 1 to TPn consists of an individual image/scan S 1 S 1 to S 1 Sn supplied as a training input sample paired with its true averaged image Avg 1 , supplied as a target training output sample. During the training phase, the convolutional network (e.g., NN of FIG. 1 ) 25 is thus trained on single scan images S 1 S 1 to S 1 Sn as training inputs and their true averaged image Avg 1 as corresponding training outputs.
FIG. 6 illustrates an example operation of the present CNN 25 , either with live data input after training or with test data input during an evaluation phase. The present CNN 25 receives a single ophthalmic image 31 (e.g., a live OCT/OCTA scan) as input, and predicts (e.g. generates) a corresponding output image 33 (e.g. an average-simulating image with characteristics of a true averaged image, or an output image with characteristics of a different imaging modality, e.g. AO-simulating image, according to training). In the present example, the output image 33 is an average-simulating image, and has characteristics of a true averaged en face image. In other words, output image 33 simulates the averaging of the input image with multiple hypothetical images of a similar region as taken by the input image. It is noted that input image 31 is not an image used in training, or an image derived from any image used in training. That is, an input test scan (e.g., 31 ) not seen before by the network 25 is selected for the testing/evaluation/operation phase, and the network 25 predicts a possible output corresponding to a situation where the input image was part of a set of images that was averaged together.
As is explained above, creating a true averaged image requires registering multiple, individual scans/images that are to be averaged together. Establishing good registration among the individual scans may be complicated by the individual scans not being of sufficient quality. Consequently, the resultant averaged images may be less than optimal, e.g., it may show haziness and/or blurring. Thus, sometimes a true averaged image is not necessarily of higher quality than an individual image if the true averaged image is the result of a bad registration of multiple images. The present invention may also be used to improve the registration of individual images to define true averaged images of higher quality.
With reference to FIG. 7 , a present trained NN 25 may be incorporated into a workflow for averaging multiple scans/images similar to the workflow of FIG. 2 . For example, the present workflow may begin with data collection, wherein multiples OCT/OCTA scans I- 1 to I-q of the same region of a sample are collected. Optionally, the multiple scans I- 1 to I-q may be en face images, B-scans, cube scans, etc. Irrespective, the input scans I- 1 to I-q are submitted to a trained neural network 25 (e.g., based on any, or combination, of the neural networks of FIGS. 1 and 25 - 28 ), and the neural network 25 produces a separate and corresponding translated image A- 1 to A-q for each input image I- 1 to I-q. Each translated image A- 1 to A-q may be a higher quality image than its corresponding input image I- 1 to I-q, which facilitates their registration (e.g., alignment) by image registration block 17 . Because of the better registration of images, the result of averaging will be better than if the original scans I- 1 to I-q were averaged directly, as described above in reference to FIG. 2 . Optionally, output images A- 1 to A-q from neural network 25 may be submitted to an image pre-processor block 15 prior to being submitted to image registration block 17 , which applies at least one of multiple image registration techniques known in the art and generates image registration settings (e.g., registration parameters or transform parameters). Because images A- 1 to A-q output from neural network 25 are of better quality than original images I- 1 to I-q, image registration block 17 can provide better registration, and image averaging module 19 can thus provide a higher quality true averaged image 45 . That is, averaging module 19 may average together the transformed images A- 1 to A-q. Alternatively, since there is a one-to-one correspondence between translated images A- 1 to A-q and input images I- 1 to I-q, the registration/transform parameters generated by image registration module 17 for each individual image A- 1 to A-q may be applied to individually corresponding input images I- 1 to I-q so as to register (e.g., image align) input images I- 1 to I-q using registration parameters from images A- 1 to A-q, as indicated by dotted line 47 . In this manner, the thus registered (e.g. aligned) input images I- 1 to I-q may then be averaged by image averaging module 19 to produce a true averaged image of input images I- 1 to I-q. Optionally, irrespective of whether averaged image 45 is the result of averaging registered images A- 1 to A-q, the result of averaging input images I- 1 to I-q, or a combination of both, average image 45 may optionally be used to generate new training sample pairs, which may then be submitted to neural network 25 to provide additional training, or may be used to train another neural network, or may be used for further processing.
In the above discussed embodiments, the present image translation neural network is trained on pairs of corresponding input and output images. For example, to train the neural network to create average-simulating images, multiple OCT/OCTA scans of the same region of an eye are taken, registered/aligned to each other, and averaged together. This creates multiple training sample pairs where each of the multiple scans may be paired to its corresponding true averaged image. The present network thus learns the weights needed to translate the image style of single input images to the image style/characteristics of images of different imaging conditions or a different imaging modality (e.g., average-simulating image or AO-simulating image). Using true averaged OCT images (or true AO-OCT images) forces the network to learn from real image characteristics rather than smoothing or coherence-based approaches which look for auxiliary image characteristics.
Deep learning networks may benefit from very large datasets. Optionally the number of training sample pairs may be increased by using a patching-based scheme to train the network on smaller patches of the training images.
FIG. 8 illustrates a patching-based scheme in accord with the present invention for training a neural network using pairs of corresponding image patches. A full-size, single image S 1 S 3 and a corresponding full-size true averaged image Avg 1 is shown. For example, full-size image S 1 S 3 may be part of a set of images that are averaged together to define true averaged image Avg 1 , as is discussed above. Although full-size images may be used for training, the images in each training pair may optionally be divided into multiple, similarly sized and corresponding image-segments (e.g., patches). For example, full-size image S 1 S 3 may be divided into a first patch group S 1 S 3 PG of twenty-five patches (S 1 S 3 P 1 to S 1 S 3 P 25 ), and full-size, true averaged image Avg 1 may similarly be divided into a second patch group Avg 1 PG of twenty-file patches (Avg 1 P 1 to Avg 1 P 25 ). In the present example, the patches are numbered consecutively in five rows such that the top, first row of patches of patch group S 2 S 3 PG are numbered from S 1 S 3 P 1 to S 1 S 3 P 5 , and so on down to the last row of patches numbered S 1 S 3 P 21 to S 1 S 3 P 25 . It is to be understood that full-size images S 1 S 3 and Avg 1 may be divided into any number of corresponding patches, such as 64Ã64 tiles/patches or any other number of patches. Irrespective, each patch in patch group S 1 S 3 PG has a corresponding patch in second group Avg 1 PG, and each pair of corresponding patches may define a separate training pair. For illustration purposes, three training pairs of corresponding patches (TP 1 â², TP 2 â², and TP 3 â²) are shown. In the present example, training pair TP 1 â² consist of patch S 1 S 37 as training input and corresponding patch AvgP 7 as training target output. Similarly, training pair TP 2 â² consist of training input patch S 1 S 3 P 17 and training target output patch Avg 1 P 17 , and training pair TP 3 â² consists of training input patch S 1 S 3 P 19 and training output patch Avg 1 P 19 . In this manner, the neural network may be trained on pairs of patches from single scans and corresponding patches from true averaged scans (or true AO-OCT scans).
FIG. 9 illustrates the use of pairs of corresponding patches to train a neural network in accord with the present invention. Each full-size, single image/scan S 1 S 1 to S 1 Sn is divided into a plurality of patches (e.g., P 1 to P 25 ), and their corresponding true averaged image Avg 1 is divided into a similar number of similarly sized patches. Individual patches from each single image/scan are paired with their corresponding patch from the averaged image to define training patch pairs. For example, the upper left patch (S 1 S 1 P 1 to S 1 SnP 1 ) from each sample image (S 1 S 1 to S 1 Sn) is paired with the upper left patch Avg 1 P
CLAIMS
Claims ( 27 )
The invention claimed is:
1. An optical coherence tomography (OCT) system comprising:
a light source for generating a beam of light;
a beam splitter having a beam-splitting surface for directing a first portion of the light into a reference arm and a second portion of the light into a sample arm;
optics for directing the light in the sample arm to one or more locations on a sample;
a detector for receiving light returning from the sample and reference arms and generating signals in response thereto;
a processor for converting the signals into a first image and submitting the first image to an image translation module that translates the first image to a second image characterized by one or more of decreased noise artifacts, jitter and minimized creation of fictional structures as compared to the first image; and
an output display for displaying a system output image based on the second image;
wherein the image translation module includes a machine learning module based on a neural network trained using a set of training input images and a target set of training output images,
wherein the neural network has an input module configured to receive a current training input image, a plurality of intermediate processing modules following the input module, an intermediate error module that determines an intermediate error based on an output of at least one select intermediate processing module and a current training output image, and an output module following the plurality of intermediate processing modules that determines a preliminary output error based on its output and the current training output image, and
wherein a total loss error for a current training cycle is defined based on a combination of the preliminary output error and the intermediate error.
2. The system of claim 1 , wherein the processor further combines a current second image with one or more previously obtained second images to generate the system output image.
3. The system of claim 2 , wherein the current second image is combined with the one or more previously obtained second images by one of direct averaging or weighted averaging with second images of higher image quality being weighted more heavily.
4. The system of claim 1 , wherein:
the processor defines a plurality of the first images, submits the plurality first images to the image translation module to produce a corresponding plurality of second images, and calculates motion contrast information from the plurality of second images using an OCT angiography (OCTA) processing technique; and
the system output image displays the motion contrast information.
5. The system of claim 1 , wherein the processor:
defines a plurality of the first images;
submits the plurality of first images to the image translation module to produce a corresponding plurality of second images;
applies an image registration technique to the plurality of second images to produce image alignment settings; and
aligns the plurality of first images based at least in part on the image alignment settings of the plurality of second images.
6. The system of claim 1 , wherein:
the first image is divided into a plurality of first image segments; and
the image translation module individually translates each first image segment into a corresponding second image segment, and combines the second image segments to construct the second image.
7. The system of claim 1 , wherein:
at least one of the training output images is defined as the averaging of a set of OCT test images of the same region of a test sample; and
at least a fraction of the training input images is included in the set of OCT test images.
8. The system of claim 1 , wherein:
the first image is of a first region of the sample; and
the second image has characteristics defined as the averaging of multiple hypothetical OCT scans of the first region with the first image.
9. The system of claim 1 , wherein the training of the neural network includes:
collecting a plurality of OCT test images of a target ophthalmic region;
averaging the plurality of OCT test images to define a corresponding averaged-image of the target ophthalmic region;
separately and individually inputting the OCT test images of the target ophthalmic region as training input images to the neural network, and providing their corresponding averaged-image as their individually corresponding training output image for the neural network.
10. The system of claim 9 , wherein the training of the neural network further includes:
dividing each OCT test image into a plurality of test segments;
dividing their corresponding averaged-image into a plurality of corresponding ground truth segments;
correlating test segments to corresponding ground truth segments;
separately and individually submitting the correlated test segments to the neural network as training input images and providing their correlated ground truth segments as training output images for the neural network.
11. The system of claim 9 , further including combining a currently inputted OCT test image with a corresponding current output of the neural network to define a combination network output, and comparing the combination network output with the corresponding training output image to determine the total loss error for a current training cycle.
12. The system of claim 1 , wherein the training input images and training output images include a mixture of images of healthy eyes and images of diseased eyes.
13. The system of claim 1 , wherein:
the first image is of a first imaging modality; and
the second image simulates a second imaging modality different than the first imaging modality.
14. The system of claim 13 , wherein the first and second modalities are a mixture including one or more of time domain OCT, spectral domain OCT, swept source OCT, and adaptive optics OCT (AO-OCT).
15. The system of claim 1 , wherein:
the OCT system is of a first modality;
the machine learning module is trained using third images taken with a first OCT device of the first modality as the set of training input images and fourth images taken with a second OCT device of a second modality as the target set of training output images, the second modality being different than the first modality; and
the second image has features characteristic of an image generated by an OCT system of the second modality.
16. The system of claim 15 , wherein:
the first OCT device of the first modality is of a non-adaptive optics OCT type;
the second OCT device of the second modality is of an adaptive optics OCT type;
the third images obtained by the first OCT device are bigger than the fourth images obtained by the second OCT device, the third images are divided into third image segments of similar size as the fourth images and each third image segment is correlated to a corresponding fourth image; and
the correlated third segments are separately and individually submitted to the neural network as training input images and their correspondingly correlated fourth images are provided as training output images for the neural network.
17. The system of claim 1 , wherein
the plurality of intermediate processing modules form a contracting path and an expanding path, the contracting path following the input module and including a plurality of encoding modules, each encoding module having a convolution stage, an activation function, and a max pooling operation,
the expanding path following the contracting path and having a plurality of decoding modules, each decoding module concatenates its current value with that of a corresponding encoding module;
the output module is an output convolution module excluding a pooling layer and a sigmoid layer activation function, the output convolution module receiving the output from the last decoding module in the expanding path and producing the preliminary output error; and
the intermediate error module determines the intermediate error for at least one encoding module and/or one decoding module.
18. The system of claim 17 , wherein during training of the neural network:
the intermediate error module determines the error measure as an error between the current training output image and the current value of the intermediate error module's corresponding encoding module and/or decoding module; and
the preliminary output error from the output convolution module is based on the current training output image and the current value of the output convolution module.
19. An ophthalmic imaging system comprising:
a processor for acquiring a first image, and submitting the first image to an image modification module that defines a second image based on the first image; and
an output display for displaying an output image based on the second image;
wherein the image modification module includes a machine model based on a neural network trained using a set of training input images and a target set of training output images, the neural network having:
a) an input module for receiving a current training input image;
b) a contracting path following the input module, the contracting path including a plurality of encoding modules, each encoding module having a convolution stage, an activation function, and a max pooling operation;
c) an expanding path following the contracting path, the expanding path having a plurality of decoding modules, each decoding module concatenating its current value with that of a corresponding encoding module;
d) an output convolution module excluding a pooling layer and an activation function, the output convolution module receiving the output from the last decoding module in the expanding path and producing a preliminary output error; and
e) an intermediate error module that determines an error measure for at least one encoding module and/or one decoding module; and
during training of the neural network, the preliminary output error of the output convolution module is combined with the output error of the intermediate error module.
20. The system of claim 19 , wherein the activation function is a rectifier linear unit or sigmoid layer.
21. The system of claim 19 , wherein the preliminary output error is based on a current training output image and the error measure is based on a current output of the at least one encoding module and/or one decoding module and the current training output image.
22. The system of claim 21 , wherein
a training cycle error for a current training cycle is based on the combined errors from the output convolution module and the intermediate error module.
23. The system of claim 19 , wherein the error measure is based on a square loss function.
24. The system of claim 19 , wherein the ophthalmic imaging system is an optical coherence tomography (OCT) angiography system, and the first image is a vasculature image.
25. The system of claim 19 , wherein the ophthalmic imaging system is an optical coherence tomography (OCT) system of a first modality, and training of the neural network includes:
collecting one or more of third images of different target ophthalmic regions using a first OCT system of the first modality;
collecting one or more of fourth images of the same target ophthalmic regions using a second OCT system of a second modality different than the first modality;
defining one or more of the training output images from the one or more fourth images; and
defining one or more of the training input images from the one or more third images, wherein each input training image has a corresponding training output image.
26. The system of claim 25 , wherein:
the first OCT system of a non-adaptive optics OCT type; and
the second OCT system is of an adaptive optics OCT type.
27. The system of claim 19 , wherein one of the set of training input images or target set of training output images are obtained by an optical coherence tomography (OCT) system, and the other of the set of training input images or target set of training output images are obtained by a fundus imaging system.
US17/428,122
2019-02-14
2020-02-12
System for OCT image translation, ophthalmic image denoising, and neural network therefor
Active
2042-02-21
US12444051B2
( en )
Priority Applications (1)
Application Number
Priority Date
Filing Date
Title
US17/428,122
US12444051B2
( en )
2019-02-14
2020-02-12
System for OCT image translation, ophthalmic image denoising, and neural network therefor
Applications Claiming Priority (3)
Application Number
Priority Date
Filing Date
Title
US201962805835P
2019-02-14
2019-02-14
PCT/EP2020/053515
WO2020165196A1
( en )
2019-02-14
2020-02-12
System for oct image translation, ophthalmic image denoising, and neural network therefor
US17/428,122
US12444051B2
( en )
2019-02-14
2020-02-12
System for OCT image translation, ophthalmic image denoising, and neural network therefor
Publications (2)
Publication Number
Publication Date
US20220058803A1
US20220058803A1 ( en )
2022-02-24
US12444051B2
true
US12444051B2 ( en )
2025-10-14
Family
ID=69570686
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US17/428,122
Active
2042-02-21
US12444051B2
( en )
2019-02-14
2020-02-12
System for OCT image translation, ophthalmic image denoising, and neural network therefor
Country Status (5)
Country
Link
US
( 1 )
US12444051B2
( en )
EP
( 1 )
EP3924931B1
( en )
JP
( 1 )
JP7518086B2
( en )
CN
( 1 )
CN113396440A
( en )
WO
( 1 )
WO2020165196A1
( en )
Families Citing this family (58)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2018091486A1
( en )
2016-11-16
2018-05-24
Ventana Medical Systems, Inc.
Convolutional neural networks for locating objects of interest in images of biological samples
WO2020183799A1
( en )
*
2019-03-11
2020-09-17
ãã¤ãã³æ ªå¼ä¼ç¤¾
Medical image processing device, medical image processing method, and program
US11763553B2
( en )
*
2019-04-17
2023-09-19
The Regents Of The University Of California
Artificial intelligence advance imagingâprocessing conditioned light photography and videography to reveal features detectable by other advanced imaging and functional testing technologies
WO2021041772A1
( en )
2019-08-30
2021-03-04
The Research Foundation For The State University Of New York
Dilated convolutional neural network system and method for positron emission tomography (pet) image denoising
US12062151B2
( en )
*
2020-03-11
2024-08-13
Mediatek Inc.
Image-guided adjustment to super-resolution operations
US12064281B2
( en )
*
2020-03-18
2024-08-20
The Regents Of The University Of California
Method and system for denoising CT images using a neural network
CN111862255B
( en )
*
2020-07-17
2024-07-26
ä¸æµ·èå½±å»çç§æè¡ä»½æéå ¬å¸
Regularized image reconstruction method, regularized image reconstruction system, readable storage medium and regularized image reconstruction device
CN112184576B
( en )
*
2020-09-17
2024-01-19
æ¡æçµåç§æ大å¦
A method for enhancing high-reflection bright spots in spectral domain optical coherence tomography
CN112163541A
( en )
*
2020-10-09
2021-01-01
ä¸æµ·äºç» æºè½ç§ææéå ¬å¸
3D target detection method and device, electronic equipment and storage medium
CN112330724B
( en )
*
2020-10-15
2024-04-09
è´µå·å¤§å¦
An unsupervised multimodal image registration method based on ensemble attention enhancement
US12136197B2
( en )
*
2020-11-05
2024-11-05
Massachusetts Institute Of Technology
Neural network systems and methods for removing noise from signals
CN112434749A
( en )
*
2020-12-02
2021-03-02
çµåç§æ大å¦ä¸å±±å¦é¢
Multimode fiber speckle pattern reconstruction and identification method based on U-Net
EP4256519A2
( en )
*
2020-12-04
2023-10-11
Carl Zeiss Meditec, Inc.
Using multiple sub-volumes, thicknesses, and curvatures for oct/octa data registration and retinal landmark detection
CN112541876B
( en )
*
2020-12-15
2023-08-04
å京ç¾åº¦ç½è®¯ç§ææéå ¬å¸
Satellite image processing method, network training method, related device and electronic equipment
CN112699950B
( en )
*
2021-01-06
2023-03-24
è ¾è®¯ç§æï¼æ·±å³ï¼æéå ¬å¸
Medical image classification method, image classification network processing method, device and equipment
CN112819705B
( en )
*
2021-01-13
2023-04-18
西å®äº¤é大å¦
Real image denoising method based on mesh structure and long-distance correlation
CN112700439B
( en )
*
2021-01-14
2025-03-11
广ä¸å¯ä»å»çç§ææéå ¬å¸
A neural network-based OCT human eye image acquisition and registration method and system
CN112712520A
( en )
*
2021-01-18
2021-04-27
ä½å±±ç§å¦ææ¯å¦é¢
Choroid layer segmentation method based on ARU-Net
US12354235B2
( en )
*
2021-01-26
2025-07-08
Samsung Electronics Co., Ltd.
Method and apparatus with image restoration
CN113256531A
( en )
*
2021-06-11
2021-08-13
äºåçµç½æéè´£ä»»å ¬å¸çµåç§å¦ç 究é¢
Noise reduction method for hyperspectral image of power equipment
CN113962953B
( en )
*
2021-10-20
2025-03-21
ä¸æµ·èå½±å»çç§æè¡ä»½æéå ¬å¸
Image processing method and system
JP7594827B2
( en )
2021-10-13
2024-12-05
å½ç«å¤§å¦æ³äººæ±å大å¦
Biometric image processing program, biological image processing device, and biological image processing method
CA3235419A1
( en )
*
2021-10-14
2023-04-20
Exo Imaging, Inc.
Method and system for image processing based on convolutional neural network
CN113971677B
( en )
*
2021-10-21
2025-08-01
泰康ä¿é©éå¢è¡ä»½æéå ¬å¸
Image segmentation method, device, electronic equipment and readable medium
TWI796837B
( en )
2021-11-17
2023-03-21
å®ç¢è¡ä»½æéå ¬å¸
Noise reduction convolution auto-encoding device and noise reduction convolution self-encoding method
CN116206116A
( en )
*
2021-11-29
2023-06-02
å®ç¢è¡ä»½æéå ¬å¸
Noise reduction convolution self-coding device and noise reduction convolution self-coding method
US12575726B2
( en )
*
2021-12-20
2026-03-17
Carl Zeiss Meditec, Inc.
Method and system for choroid-scleral segmentation using deep learning with a choroid-scleral layer model
US12073539B2
( en )
*
2021-12-29
2024-08-27
Shanghai United Imaging Intelligence Co., Ltd.
Systems and methods for denoising medical images
CN114565946B
( en )
*
2022-01-25
2024-07-16
西å®çµåç§æ大å¦
A fingerprint liveness detection method based on lightweight network with self-attention mechanism
CN114820341B
( en )
*
2022-03-17
2025-09-23
西åå·¥ä¸å¤§å¦
A method and system for blind image denoising based on enhanced Transformer
FI4252628T3
( en )
*
2022-03-28
2024-09-11
Optos Plc
Optical coherence tomography angiography data processing to reduce projection artifacts
CN114781445B
( en )
*
2022-04-11
2022-11-18
å±±ä¸ç人工æºè½ç 究é¢
An ECG Signal Denoising Method Based on Interpretable Deep Neural Networks
WO2023200846A1
( en )
*
2022-04-15
2023-10-19
United States Government As Represented By The Department Of Veterans Affairs
Retinal distance-screening using machine learning
AU2023255422A1
( en )
*
2022-04-18
2024-09-12
Alcon Inc.
Ophthalmic image registration using interpretable artificial intelligence based on deep learning
CN114820535B
( en )
*
2022-05-05
2023-09-12
æ·±å³å¸é±ç¡å»çç§ææéå ¬å¸
Image detection method and device for aneurysm, computer equipment and storage medium
CN114886442A
( en )
*
2022-05-06
2022-08-12
æµæ±å·¥ä¸å¤§å¦
Uterine electromyographic signal (EHG) analysis system based on graph theory
DE23812411T1
( en )
*
2022-05-23
2025-06-05
Topcon Corporation
AUTOMATED OCT ACQUISITION
CN114782443A
( en )
*
2022-06-22
2022-07-22
æ·±å³ç§äºå»çç§ææéå ¬å¸
Device and storage medium for data-based enhanced aneurysm risk assessment
CN115375628B
( en )
*
2022-07-25
2026-04-07
ä¸å±±å¤§å¦ä¸å±±ç¼ç§ä¸å¿
An image noise removal method and system based on AS-OCTA
CN115482198B
( en )
*
2022-08-16
2025-07-04
ä¸çµå¸®æ¡¥å»çå¨æ¢°(广å·)æéè´£ä»»å ¬å¸
Method, device, computer equipment and storage medium for detecting vision parameters
KR102926903B1
( en )
*
2022-09-27
2026-02-11
ì¬íë³µì§ë²ì¸ ì¼ì±ìëª ê³µìµì¬ë¨
A multimodal deep learning model for predicting future visual field in glaucoma patients
CN115587949B
( en )
*
2022-10-27
2024-07-12
è´µå·å¤§å¦
Agricultural multispectral vision reconstruction method based on visible light image
WO2024126868A1
( en )
*
2022-12-16
2024-06-20
Danmarks Tekniske Universitet
A method of using optical coherence tomography for training a machine learning model, a method of determining an unknown condition using the trained model, a method of determining segmentation in a sample, and an optical coherence tomography system
LU503281B1
( en )
*
2022-12-30
2024-07-01
Luxembourg Inst Science & Tech List
Digital image denoising method
CN115984406B
( en )
*
2023-03-20
2023-06-20
å§ç»(æ é¡)å»çç§ææéå ¬å¸
A SS-OCT Compressed Imaging Method Based on Deep Learning and Joint Subsampling in Spectral and Spatial Domains
WO2024206542A1
( en )
*
2023-03-29
2024-10-03
Arizona Board Of Regents On Behalf Of Arizona State University
Systems and methods for enhancing retinal color fundus images for retinopathy analysis
CN116645265B
( en )
*
2023-05-29
2026-03-13
广ä¸å¯ä»å»çç§ææéå ¬å¸
A method for converting FFA imaging images based on deep learning-based OCT and OCTA imaging.
EP4728467A1
( en )
*
2023-06-19
2026-04-22
Medtronic Vascular Inc.
High resolution synthetic medical imaging
CN116823663B
( en )
*
2023-06-30
2026-03-17
èµçç§æ(æ²³å)éå¢è¡ä»½æéå ¬å¸
Neural Network-Based Vascular Image Processing Method and Device
CN116563116B
( en )
*
2023-07-10
2023-12-05
èå·å¤§å¦
OCT axial super-resolution method and system based on complex-valued neural network
JPWO2025028404A1
( en )
*
2023-07-31
2025-02-06
CN117196991B
( en )
*
2023-09-21
2025-10-24
å京ç工大å¦
Image enhancement method based on multimodal loss function U-type codec network
US20250118062A1
( en )
*
2023-10-06
2025-04-10
GE Precision Healthcare LLC
Explainable visual attention for deep learning
EP4550253A1
( en )
*
2023-11-02
2025-05-07
Optos PLC
Noise reduction in ophthalmic images
EP4575980B1
( en )
*
2023-12-22
2025-10-15
Optos plc
Noise reduction in retinal images
CN117689760B
( en )
*
2024-02-02
2024-05-03
å±±ä¸å¤§å¦
OCT axial super-resolution method and system based on histogram information network
CN118154411B
( en )
*
2024-05-10
2024-07-16
æ¸ å大å¦
Digital Adaptive Optics Architecture and System
CN120318504B
( en )
*
2025-06-17
2025-08-29
ä¸å½éå¡è¡ä»½æéå ¬å¸
Method and device for identifying floats, computer program product and electronic equipment
Citations (24)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US6549801B1
( en )
1998-06-11
2003-04-15
The Regents Of The University Of California
Phase-resolved optical coherence tomography and optical doppler tomography for imaging fluid flow in tissue with fast scanning speed and high velocity sensitivity
US6741359B2
( en )
2002-05-22
2004-05-25
Carl Zeiss Meditec, Inc.
Optical coherence tomography optical scanner
US20050171438A1
( en )
2003-12-09
2005-08-04
Zhongping Chen
High speed spectral domain functional optical coherence tomography and optical doppler tomography for in vivo blood flow dynamics and tissue structure
US7301644B2
( en )
2004-12-02
2007-11-27
University Of Miami
Enhanced optical coherence tomography for anatomical mapping
JP2008528954A
( en )
2005-01-21
2008-07-31
ã«ã¼ã« ãã¡ã¤ã¹ ã¡ãã£ãã㯠ã¢ã¯ãã¨ã³ã²ã¼ã«ã·ã£ãã
Motion correction method in optical coherence tomography imaging
US20100027857A1
( en )
2006-09-26
2010-02-04
Wang Ruikang K
In vivo structural and flow imaging
US20120277579A1
( en )
2011-07-07
2012-11-01
Carl Zeiss Meditec, Inc.
Inter-frame complex oct data analysis techniques
US20120307014A1
( en )
2009-05-04
2012-12-06
Oregon Health & Science University
Method and apparatus for ultrahigh sensitive optical microangiography
US8967806B2
( en )
2010-11-06
2015-03-03
Carl Zeiss Meditec Ag
Fundus camera with strip-shaped pupil division, and method for recording artifact-free, high-resolution fundus images
US8998411B2
( en )
2011-07-08
2015-04-07
Carl Zeiss Meditec, Inc.
Light field camera for fundus photography
US20150208915A1
( en )
*
2014-01-29
2015-07-30
University Of Rochester
System and method for observing an object in a blood vessel
US9332902B2
( en )
2012-01-20
2016-05-10
Carl Zeiss Meditec, Inc.
Line-field holoscopy
US20160317027A1
( en )
*
2015-05-01
2016-11-03
Canon Kabushiki Kaisha
Image generating apparatus, image generating method, and program
US20170023887A1
( en )
2015-07-24
2017-01-26
Kyocera Document Solutions Inc.
Image forming apparatus
US9700206B2
( en )
2015-02-05
2017-07-11
Carl Zeiss Meditec, Inc.
Acquistion and analysis techniques for improved outcomes in optical coherence tomography angiography
US20170256054A1
( en )
*
2016-03-03
2017-09-07
Nidek Co., Ltd.
Ophthalmic image processing apparatus and ophthalmic image processing program
US9759544B2
( en )
2014-08-08
2017-09-12
Carl Zeiss Meditec, Inc.
Methods of reducing motion artifacts for optical coherence tomography angiography
WO2018200493A1
( en )
2017-04-25
2018-11-01
The Board Of Trustees Of The Leland Stanford Junior University
Dose reduction for medical imaging using deep convolutional neural networks
EP3404611A1
( en )
2017-05-19
2018-11-21
RetinAI Medical GmbH
Reducing noise in an image
US20190005603A1
( en )
2017-06-30
2019-01-03
Intel Corporation
Approximating image processing functions using convolutional neural networks
US20190130217A1
( en )
*
2018-11-15
2019-05-02
Chyuan-Tyng Wu
Trainable vision scaler
US10426442B1
( en )
*
2019-06-14
2019-10-01
Cycle Clarity, LLC
Adaptive image processing in assisted reproductive imaging modalities
US20200037872A1
( en )
*
2018-08-02
2020-02-06
Nidek Co., Ltd.
Oct apparatus
US11763549B1
( en )
*
2022-07-29
2023-09-19
Contemporary Amperex Technology Co., Limited
Method and apparatus for training cell defect detection model
Family Cites Families (3)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2017143300A1
( en )
2016-02-19
2017-08-24
Optovue, Inc.
Methods and apparatus for reducing artifacts in oct angiography using machine learning techniques
CN106408562B
( en )
*
2016-09-22
2019-04-09
ååç工大å¦
A method and system for retinal blood vessel segmentation in fundus images based on deep learning
CN108764241A
( en )
*
2018-04-20
2018-11-06
å¹³å®ç§æï¼æ·±å³ï¼æéå ¬å¸
Divide method, apparatus, computer equipment and the storage medium of near end of thighbone
2020
2020-02-12
US
US17/428,122
patent/US12444051B2/en
active
Active
2020-02-12
JP
JP2021547257A
patent/JP7518086B2/en
active
Active
2020-02-12
CN
CN202080012077.4A
patent/CN113396440A/en
active
Pending
2020-02-12
EP
EP20704866.1A
patent/EP3924931B1/en
active
Active
2020-02-12
WO
PCT/EP2020/053515
patent/WO2020165196A1/en
not_active
Ceased
Patent Citations (26)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US6549801B1
( en )
1998-06-11
2003-04-15
The Regents Of The University Of California
Phase-resolved optical coherence tomography and optical doppler tomography for imaging fluid flow in tissue with fast scanning speed and high velocity sensitivity
US6741359B2
( en )
2002-05-22
2004-05-25
Carl Zeiss Meditec, Inc.
Optical coherence tomography optical scanner
JP2005525893A
( en )
2002-05-22
2005-09-02
ã«ã¼ã« ãã¢ã¤ã¹ ã¡ãã£ãã㯠ã¢ã¯ãã¨ã³ã²ã¼ã«ã·ã£ãã
Optical coherence tomography scanner using negative image surface curvature
US20050171438A1
( en )
2003-12-09
2005-08-04
Zhongping Chen
High speed spectral domain functional optical coherence tomography and optical doppler tomography for in vivo blood flow dynamics and tissue structure
US7301644B2
( en )
2004-12-02
2007-11-27
University Of Miami
Enhanced optical coherence tomography for anatomical mapping
JP2008528954A
( en )
2005-01-21
2008-07-31
ã«ã¼ã« ãã¡ã¤ã¹ ã¡ãã£ãã㯠ã¢ã¯ãã¨ã³ã²ã¼ã«ã·ã£ãã
Motion correction method in optical coherence tomography imaging
US9706915B2
( en )
2005-01-21
2017-07-18
Carl Zeiss Meditec, Inc.
Method of motion correction in optical coherence tomography imaging
US20100027857A1
( en )
2006-09-26
2010-02-04
Wang Ruikang K
In vivo structural and flow imaging
US20120307014A1
( en )
2009-05-04
2012-12-06
Oregon Health & Science University
Method and apparatus for ultrahigh sensitive optical microangiography
US8967806B2
( en )
2010-11-06
2015-03-03
Carl Zeiss Meditec Ag
Fundus camera with strip-shaped pupil division, and method for recording artifact-free, high-resolution fundus images
US20120277579A1
( en )
2011-07-07
2012-11-01
Carl Zeiss Meditec, Inc.
Inter-frame complex oct data analysis techniques
US8998411B2
( en )
2011-07-08
2015-04-07
Carl Zeiss Meditec, Inc.
Light field camera for fundus photography
US9332902B2
( en )
2012-01-20
2016-05-10
Carl Zeiss Meditec, Inc.
Line-field holoscopy
US20150208915A1
( en )
*
2014-01-29
2015-07-30
University Of Rochester
System and method for observing an object in a blood vessel
US9759544B2
( en )
2014-08-08
2017-09-12
Carl Zeiss Meditec, Inc.
Methods of reducing motion artifacts for optical coherence tomography angiography
US9700206B2
( en )
2015-02-05
2017-07-11
Carl Zeiss Meditec, Inc.
Acquistion and analysis techniques for improved outcomes in optical coherence tomography angiography
<tr i