Neuromorphic single photon avalanche detector (SPAD) array microchip — The Commonwealth Of Australia (US11474215B2)
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
ABSTRACT
Abstract
Described is a Single-Photon Avalanche Diode (SPAD) array microchip comprising: a plurality of SPAD sensors; and a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames. An event based neuromorphic SPAD array microchip is also described. The chip architecture and triggering methodology takes a local group of SPAD sensors connected in a certain way and by using simple digital circuits emulating how neurons behave, patterns within a local receptive field are identified. Only when these unique patterns or features are identified are âeventsâ triggered for each receptive field in the order they occur, or in an asynchronous manner. Each neuromorphic circuit (or collection of silicon neurons) act over overlapping receptive fields, and are tiled across the entire visual spatial field of the SPAD array to a form a convolution layer.
Description
This application is the United States national phase of International Application No. PCT/AU2018/000187 filed Sep. 27, 2018, and claims priority to Australian Provisional Patent Application No. 2017903926 filed Sep. 27, 2017, the disclosures of which are hereby incorporated by reference in their entirety.
TECHNICAL FIELD
The present disclosure relates to sensor circuits comprising an array of Single-Photon Avalanche Diode (SPAD) circuits for three dimensional (3D) camera systems.
BACKGROUND
Avalanche photodiodes (APDs) are solid-state photodetectors in which a high bias voltage is applied to a p-n junction to provide a high first stage gain due to avalanche multiplication. Avalanche multiplication occurs when an incident photon with sufficient energy to liberate an electron arrives at the photodiode. The high electric field rapidly accelerates the photo-generated electron towards the anode, but before it can reach the anode, it collides with the intervening doped material releasing further electrons, all of which are then accelerated towards the anode. This process repeats leading to avalanche multiplication of the photo-generated electron and an output current pulse. APDs are thus semiconductor analogs to photomultiplier tubes. Linear mode APDs are effectively single stage linear amplifiers in which the gain is set by controlling the bias voltage, and with gain factors of several hundred can be achieved in linear mode.
Single-Photon Avalanche Diodes (SPADs) are APDs in which the p-n junction is biased above its breakdown voltage to operate in Geiger mode such that a single incident photon will trigger an ongoing avalanche breakdown and thus easily measureable current pulse. That is a SPAD operates as a trigger device generating a large current pulse compared to linear mode APDs in which the current pulses can be very low at low light intensity. After triggering of the SPAD, a quenching circuit is used to reduce the bias voltage below the breakdown voltage in order to quench the avalanche process. Once quenched the bias voltage is again raised above the breakdown voltage to reset the SPAD for detection of another photon (known as re-biasing the SPAD).
APD and SPADs are solid state devices and can be constructed using a variety of CMOS technologies, and have very small active areas compared with other photon counting devices such as photomultiplier tubes. Through appropriate choice of materials and structure, wavelength sensitivity of a SPAD can be controlled to be in the visible and/or near-infrared range. A SPAD combined with additional circuitry to count pulses and/or measure time of arrival of photons to sub-nano/pico second accuracy can be used to create sensors for ultra-low light imaging or highly sensitive time-resolved imaging applications. For example one potential application of a SPAD arrays are in three dimensional (3D) Flash LIDAR cameras, as they have the potential to provide extremely sensitive devices with high distance resolution and high frame rates. SPAD arrays also have potential for use in other applications that require single photo sensitivity with high frames such as biological/medical imaging applications, adaptive optics applications, and astrophysics applications.
Three dimensional Flash LIDAR systems, also known as 3D Time of Flight (TOF) Cameras, use a laser source to irradiate a target with a short duration laser pulse (i.e. a laser flash). Photons are back scattered off objects and onto the sensor and the time of arrival is used to determine time of flight and thus distance to an object. The first 3D TOF cameras were constructed as scanning laser systems comprising a laser range finder with a rotating or scanning element(s) to progressively scan the field of view. These scanning systems are effectively single pixel devices collecting time of flight information in a single direction which build up a 3D image progressively moving the pointing direction of the sensing element. More recently scanner-less (i.e. staring) 3D Flash LIDAR systems have been developed using a two dimensional array of Geiger mode APDs (pixels) to achieve more rapid scene capture than scanning systems, and to avoid problems with scanning systems such as mechanical wear, vibration, and/or motion blur. By rapidly resetting APDs after triggering, each pixel can be used to receive multiple photons from the laser pulse, each corresponding to different distance. In this way, a 3D point cloud of the target scene can be rapidly generated.
SPAD arrays in 3D Flash LIDAR systems have the potential to achieve even greater light sensitivity with high distance resolution and high frame rates than scanning laser systems or scanner-less APD based systems. In a frame based SPAD chip, each pixel or cell contains 1 or more SPADs and corresponding digital circuitry to time photon events (known as Time of Flight or ToF Mode) or to count photon events (known as photon counting mode. In photon counting mode, counters integrated to each SPAD cell keep a count of the number of arriving photons at each cell and effectively provide a measure of illumination, and the illuminator/laser can be omitted in which case the camera operates as a low light imager (images are intensity only 2D images). In DTOF timing mode, the integrated counters are triggered to start counting by the laser pulse and stop counting at the detection of the first photon. ToF techniques can be grouped into direct ToF or (dToF) and indirect (iToF) categories. Direct ToF methods directly measure the time delay by means of a very accurate timer or a Time-to-Digital (TDC) converter situated in the pixel. Depending on the accuracy or resolution of the timer, this method is typically used for long (kilometres) distance measurements and at very high precision (millimetre) depth resolutions. The Indirect ToF method in contrast reconstructs the time delay (hence distance) from the measurement of the phase delay of the reflected signal when compared to the periodic emitted light signal. This technique is more suited to short or medium distances (tens of meters) and with depth resolutions of some centimetres. For the iToF technique, either a continuous-wave iToF (cw-iToF) can be implemented, whereby a sinusoid modulated light source illuminates the scene and the returned signal is sampled a few times during the modulation period in order to compute the phase delay, or a pulsed-light iToF (p-iToF) method where the illuminator uses square pulses of light. This work will focus on using SPAD technology to implement specifically dToF methods for LADAR imaging.
The data generated by DToF SPAD arrays is a three dimensional time surface corresponding to the relative distance of the visual scene to the camera and illuminator. In DTOF mode a high speed precision counter is used for each SPAD cell to encode the time of flight of the arriving photons. Each frame comprises firing a laser pulse and synchronising the counters for each of the SPADs to the time of firing of the laser pulse. When a SPAD is triggered (i.e. due to an arriving photon), the counter is latched. At the end of the frame (e.g. max counter value) the counter values of SPAD cells must be read off chip for processing (i.e. to build up a 3D spatial picture of the target scene). As the photons are travelling at the speed of light, a single frame can be taken in the space of a few hundred nanoseconds after each laser pulse. This data transfer process creates an information bottleneck as significant bandwidth is required to transfer data from each of the SPAD cells, which is currently a major limiting factor in the speed of operation of such cameras. Further this bottleneck grows with increasing SPAD array size.
There is thus a need to provide an improved SPAD array system for 3D imaging applications at least provide a useful alternative to current DToF SPAD array systems.
SUMMARY
According to a first aspect, there is provided a method for operation of a Photon Avalanche Diode (SPAD) array microchip, the method comprising:
detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval, wherein the timing interval comprises one or more frames; and
reading out an address of each triggered SPAD sensor or an identifier that identifies the location of a group of triggered SPAD sensors in the order of triggering during the timing interval.
In one form, detecting the triggering of one or more SPAD sensors comprises:
detecting, on board the microchip, the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame,
and the step of reading out the address comprises:
reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors, or the addresses of the SPAD sensors in the group of SPAD sensors, only if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.
In a further form:
the plurality of SPAD sensors are each associated with one or more or a plurality of receptive field groups where each receptive field groups comprises a receptive group of SPAD sensors, and different receptive field groups may share some but not all of the same SPAD sensors; and
the plurality of predefined triggering patterns comprises a plurality of neural feature subgroups where each receptive field comprises a two or more neural feature subgroups where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors; and
detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns comprises:
detecting the triggering of a neural feature subgroup of a receptive field, and wherein upon detection the triggering of each other neural feature subgroup in the same receptive field group is suppressed until the end of the current frame;
generating an event trigger for the receptive field group if the triggered neural feature subgroup is the first neural feature subgroup in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature subgroup which previously triggered in the timing interval for this receptive field;
and the step of reading out an identifier is performed upon generation of an event trigger, and comprises reading an address of the neural feature subgroup that triggered.
In a further form, generating an event trigger for the receptive field group further comprises
incrementing a counter if the neural feature subgroup that triggered is the same neural feature subgroup that generated the previous trigger or the counter is zero, otherwise decrementing the counter;
generating an event trigger for the receptive field group only when the counter reaches a predefined counter threshold value wherein the predefined counter threshold value is fixed for at least the timing interval. In a further form, the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.
In a further form, each receptive field group has the same geometry each of the neural feature subgroups in a receptive field group comprises the same number of SPAD sensors. In a further form, the plurality of SPAD sensors are arranged in a RÃC grid, and each receptive field group is an rÃr array of SPAD sensors, and the plurality of receptive field groups overlap, and the plurality of overlapping receptive field groups tile the RÃC grid. In a further form, each neural feature group comprises a contiguous block of the rÃr array of SPAD sensors in the receptive field group. In a further form, r=4, and each neural feature group comprises a block of 8 adjacent SPAD sensors arranged in a 2Ã4 or 4Ã2 grid.
In one form after reading out an identifier the plurality of SPADs are reset and the current frame ends, an in use the start of each frame is synchronised to a laser pulse.
According to a second aspect, there is provided a Single-Photon Avalanche Diode (SPAD) array microchip comprising:
a plurality of SPAD sensors;
a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames.
In one form the triggering circuit comprises:
a plurality of feature detection circuits, each feature detection circuit connected to a group of SPAD sensors and configured to generate a feature trigger when the connected SPADs satisfy a triggering criteria;
an event detection circuit configured to detect the first time a feature trigger is generated by a feature detection circuit during a timing interval wherein the timing interval comprises one or more frames and to read out the address of either the feature detection circuit or the connected SPAD sensors off the microchip.
In a further form, the feature detection circuits are neural feature detection circuits and the triggering circuit further comprises:
a plurality of receptive field circuits wherein each receptive field circuit comprises:
a receptive group of SPAD sensors; two or more neural feature circuits, wherein each neural feature circuit comprises a subgroup of SPAD sensors in the receptive group of SPAD sensors where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors, and each neural feature circuit is configured to generate a neural feature trigger signal and an inhibit signal if all the SPAD sensors in the subgroup trigger, wherein the inhibit signal prevents triggering of the other neural feature circuits in the same receptive field circuit during the current frame; and
the event detection circuit comprises an event detection circuit for each receptive field circuit which is configured to monitor the neural feature circuits for generation of a neural feature trigger signal, and generates an event trigger for the receptive field circuit if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval; and
the triggering circuit further comprises a read out circuit which detects an event trigger from a receptive field circuit, and on detection of an event trigger it reads an address of the neural feature circuit that triggered generation of the event trigger.
In a further form, each receptive field circuit further comprises a memory, and when a neural feature trigger signal is generated an identifier of the neural feature circuit that generated the neural feature trigger signal in stored in the memory, and the event detection circuit is configured to determine if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval by comparing an identifier of the neural feature circuit that generated the neural feature trigger with the identifier stored in the memory.
In a further form, each receptive field circuit further comprises a counter wherein the counter is incremented the ne
This application is the United States national phase of International Application No. PCT/AU2018/000187 filed Sep. 27, 2018, and claims priority to Australian Provisional Patent Application No. 2017903926 filed Sep. 27, 2017, the disclosures of which are hereby incorporated by reference in their entirety.
TECHNICAL FIELD
The present disclosure relates to sensor circuits comprising an array of Single-Photon Avalanche Diode (SPAD) circuits for three dimensional (3D) camera systems.
BACKGROUND
Avalanche photodiodes (APDs) are solid-state photodetectors in which a high bias voltage is applied to a p-n junction to provide a high first stage gain due to avalanche multiplication. Avalanche multiplication occurs when an incident photon with sufficient energy to liberate an electron arrives at the photodiode. The high electric field rapidly accelerates the photo-generated electron towards the anode, but before it can reach the anode, it collides with the intervening doped material releasing further electrons, all of which are then accelerated towards the anode. This process repeats leading to avalanche multiplication of the photo-generated electron and an output current pulse. APDs are thus semiconductor analogs to photomultiplier tubes. Linear mode APDs are effectively single stage linear amplifiers in which the gain is set by controlling the bias voltage, and with gain factors of several hundred can be achieved in linear mode.
Single-Photon Avalanche Diodes (SPADs) are APDs in which the p-n junction is biased above its breakdown voltage to operate in Geiger mode such that a single incident photon will trigger an ongoing avalanche breakdown and thus easily measureable current pulse. That is a SPAD operates as a trigger device generating a large current pulse compared to linear mode APDs in which the current pulses can be very low at low light intensity. After triggering of the SPAD, a quenching circuit is used to reduce the bias voltage below the breakdown voltage in order to quench the avalanche process. Once quenched the bias voltage is again raised above the breakdown voltage to reset the SPAD for detection of another photon (known as re-biasing the SPAD).
APD and SPADs are solid state devices and can be constructed using a variety of CMOS technologies, and have very small active areas compared with other photon counting devices such as photomultiplier tubes. Through appropriate choice of materials and structure, wavelength sensitivity of a SPAD can be controlled to be in the visible and/or near-infrared range. A SPAD combined with additional circuitry to count pulses and/or measure time of arrival of photons to sub-nano/pico second accuracy can be used to create sensors for ultra-low light imaging or highly sensitive time-resolved imaging applications. For example one potential application of a SPAD arrays are in three dimensional (3D) Flash LIDAR cameras, as they have the potential to provide extremely sensitive devices with high distance resolution and high frame rates. SPAD arrays also have potential for use in other applications that require single photo sensitivity with high frames such as biological/medical imaging applications, adaptive optics applications, and astrophysics applications.
Three dimensional Flash LIDAR systems, also known as 3D Time of Flight (TOF) Cameras, use a laser source to irradiate a target with a short duration laser pulse (i.e. a laser flash). Photons are back scattered off objects and onto the sensor and the time of arrival is used to determine time of flight and thus distance to an object. The first 3D TOF cameras were constructed as scanning laser systems comprising a laser range finder with a rotating or scanning element(s) to progressively scan the field of view. These scanning systems are effectively single pixel devices collecting time of flight information in a single direction which build up a 3D image progressively moving the pointing direction of the sensing element. More recently scanner-less (i.e. staring) 3D Flash LIDAR systems have been developed using a two dimensional array of Geiger mode APDs (pixels) to achieve more rapid scene capture than scanning systems, and to avoid problems with scanning systems such as mechanical wear, vibration, and/or motion blur. By rapidly resetting APDs after triggering, each pixel can be used to receive multiple photons from the laser pulse, each corresponding to different distance. In this way, a 3D point cloud of the target scene can be rapidly generated.
SPAD arrays in 3D Flash LIDAR systems have the potential to achieve even greater light sensitivity with high distance resolution and high frame rates than scanning laser systems or scanner-less APD based systems. In a frame based SPAD chip, each pixel or cell contains 1 or more SPADs and corresponding digital circuitry to time photon events (known as Time of Flight or ToF Mode) or to count photon events (known as photon counting mode. In photon counting mode, counters integrated to each SPAD cell keep a count of the number of arriving photons at each cell and effectively provide a measure of illumination, and the illuminator/laser can be omitted in which case the camera operates as a low light imager (images are intensity only 2D images). In DTOF timing mode, the integrated counters are triggered to start counting by the laser pulse and stop counting at the detection of the first photon. ToF techniques can be grouped into direct ToF or (dToF) and indirect (iToF) categories. Direct ToF methods directly measure the time delay by means of a very accurate timer or a Time-to-Digital (TDC) converter situated in the pixel. Depending on the accuracy or resolution of the timer, this method is typically used for long (kilometres) distance measurements and at very high precision (millimetre) depth resolutions. The Indirect ToF method in contrast reconstructs the time delay (hence distance) from the measurement of the phase delay of the reflected signal when compared to the periodic emitted light signal. This technique is more suited to short or medium distances (tens of meters) and with depth resolutions of some centimetres. For the iToF technique, either a continuous-wave iToF (cw-iToF) can be implemented, whereby a sinusoid modulated light source illuminates the scene and the returned signal is sampled a few times during the modulation period in order to compute the phase delay, or a pulsed-light iToF (p-iToF) method where the illuminator uses square pulses of light. This work will focus on using SPAD technology to implement specifically dToF methods for LADAR imaging.
The data generated by DToF SPAD arrays is a three dimensional time surface corresponding to the relative distance of the visual scene to the camera and illuminator. In DTOF mode a high speed precision counter is used for each SPAD cell to encode the time of flight of the arriving photons. Each frame comprises firing a laser pulse and synchronising the counters for each of the SPADs to the time of firing of the laser pulse. When a SPAD is triggered (i.e. due to an arriving photon), the counter is latched. At the end of the frame (e.g. max counter value) the counter values of SPAD cells must be read off chip for processing (i.e. to build up a 3D spatial picture of the target scene). As the photons are travelling at the speed of light, a single frame can be taken in the space of a few hundred nanoseconds after each laser pulse. This data transfer process creates an information bottleneck as significant bandwidth is required to transfer data from each of the SPAD cells, which is currently a major limiting factor in the speed of operation of such cameras. Further this bottleneck grows with increasing SPAD array size.
There is thus a need to provide an improved SPAD array system for 3D imaging applications at least provide a useful alternative to current DToF SPAD array systems.
SUMMARY
According to a first aspect, there is provided a method for operation of a Photon Avalanche Diode (SPAD) array microchip, the method comprising:
detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval, wherein the timing interval comprises one or more frames; and
reading out an address of each triggered SPAD sensor or an identifier that identifies the location of a group of triggered SPAD sensors in the order of triggering during the timing interval.
In one form, detecting the triggering of one or more SPAD sensors comprises:
detecting, on board the microchip, the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame,
and the step of reading out the address comprises:
reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors, or the addresses of the SPAD sensors in the group of SPAD sensors, only if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.
In a further form:
the plurality of SPAD sensors are each associated with one or more or a plurality of receptive field groups where each receptive field groups comprises a receptive group of SPAD sensors, and different receptive field groups may share some but not all of the same SPAD sensors; and
the plurality of predefined triggering patterns comprises a plurality of neural feature subgroups where each receptive field comprises a two or more neural feature subgroups where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors; and
detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns comprises:
detecting the triggering of a neural feature subgroup of a receptive field, and wherein upon detection the triggering of each other neural feature subgroup in the same receptive field group is suppressed until the end of the current frame;
generating an event trigger for the receptive field group if the triggered neural feature subgroup is the first neural feature subgroup in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature subgroup which previously triggered in the timing interval for this receptive field;
and the step of reading out an identifier is performed upon generation of an event trigger, and comprises reading an address of the neural feature subgroup that triggered.
In a further form, generating an event trigger for the receptive field group further comprises
incrementing a counter if the neural feature subgroup that triggered is the same neural feature subgroup that generated the previous trigger or the counter is zero, otherwise decrementing the counter;
generating an event trigger for the receptive field group only when the counter reaches a predefined counter threshold value wherein the predefined counter threshold value is fixed for at least the timing interval. In a further form, the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.
In a further form, each receptive field group has the same geometry each of the neural feature subgroups in a receptive field group comprises the same number of SPAD sensors. In a further form, the plurality of SPAD sensors are arranged in a RÃC grid, and each receptive field group is an rÃr array of SPAD sensors, and the plurality of receptive field groups overlap, and the plurality of overlapping receptive field groups tile the RÃC grid. In a further form, each neural feature group comprises a contiguous block of the rÃr array of SPAD sensors in the receptive field group. In a further form, r=4, and each neural feature group comprises a block of 8 adjacent SPAD sensors arranged in a 2Ã4 or 4Ã2 grid.
In one form after reading out an identifier the plurality of SPADs are reset and the current frame ends, an in use the start of each frame is synchronised to a laser pulse.
According to a second aspect, there is provided a Single-Photon Avalanche Diode (SPAD) array microchip comprising:
a plurality of SPAD sensors;
a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames.
In one form the triggering circuit comprises:
a plurality of feature detection circuits, each feature detection circuit connected to a group of SPAD sensors and configured to generate a feature trigger when the connected SPADs satisfy a triggering criteria;
an event detection circuit configured to detect the first time a feature trigger is generated by a feature detection circuit during a timing interval wherein the timing interval comprises one or more frames and to read out the address of either the feature detection circuit or the connected SPAD sensors off the microchip.
In a further form, the feature detection circuits are neural feature detection circuits and the triggering circuit further comprises:
a plurality of receptive field circuits wherein each receptive field circuit comprises:
a receptive group of SPAD sensors; two or more neural feature circuits, wherein each neural feature circuit comprises a subgroup of SPAD sensors in the receptive group of SPAD sensors where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors, and each neural feature circuit is configured to generate a neural feature trigger signal and an inhibit signal if all the SPAD sensors in the subgroup trigger, wherein the inhibit signal prevents triggering of the other neural feature circuits in the same receptive field circuit during the current frame; and
the event detection circuit comprises an event detection circuit for each receptive field circuit which is configured to monitor the neural feature circuits for generation of a neural feature trigger signal, and generates an event trigger for the receptive field circuit if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval; and
the triggering circuit further comprises a read out circuit which detects an event trigger from a receptive field circuit, and on detection of an event trigger it reads an address of the neural feature circuit that triggered generation of the event trigger.
In a further form, each receptive field circuit further comprises a memory, and when a neural feature trigger signal is generated an identifier of the neural feature circuit that generated the neural feature trigger signal in stored in the memory, and the event detection circuit is configured to determine if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval by comparing an identifier of the neural feature circuit that generated the neural feature trigger with the identifier stored in the memory.
In a further form, each receptive field circuit further comprises a counter wherein the counter is incremented the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the identifier of the neural feature circuit that generated the neural feature trigger is the same as the identifier stored in the memory, and an event trigger is generated when the counter is equal to a predetermined threshold value fixed for at least the timing interval. In a further form, the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.
In a further form, each of the neural feature circuits in a receptive field comprises the same number of SPAD sensors. In a further form, the plurality of SPAD sensors are arranged in a RÃC grid, each receptive field circuits is an rÃr array of SPAD sensors, and the plurality of receptive field circuits overlap, and the plurality of overlapping receptive field circuit tile the RÃC grid. In a further form, each neural feature circuit comprises a contiguous block of the rÃr array of SPAD sensors in the receptive field circuit. In a further form, r=4, and each neural feature circuit comprises a block of 8 adjacent SPAD sensors arranged in a 2Ã4 or 4Ã2 grid.
In one form, each neural feature circuit comprises a logical circuit which ANDs each of the subgroup of SPAD sensors.
In one form, the predetermined threshold value is adjustable between timing intervals.
In one form, the chip further comprises a data bus and an Address Event Representation protocol is used to read data off the microchip
In one form, after reading out the address the plurality of SPAD sensors are reset and the current frame ends, an in use the start of each frame is synchronised to a laser pulse.
BRIEF DESCRIPTION OF DRAWINGS
Embodiments of the present disclosure will be discussed with reference to the accompanying drawings wherein:
FIG. 1A is a schematic diagram of a SPAD array sensor according to an embodiment;
FIG. 1B is a schematic diagram of adjacent SPAD cells according to an embodiment;
FIG. 1C is a flowchart of a method of operation of a SPAD array microchip according to an embodiment;
FIG. 2A is an illustrative example of a 5Ã5 SPAD array microchip with four 4Ã4 receptive fields according to an embodiment;
FIG. 2B is a schematic representation of a 4Ã4 receptive field illustrating the connection of the SPAD sensors to four AND gates to create four neural feature circuits according to an embodiment;
FIG. 2C is a representation of a 3D visual scene and the resultant SPAD timing pattern for the 5Ã5 SPAD array microchip of FIG. 2A according to an embodiment;
FIG. 3A is a schematic wiring diagram of the a 5Ã5 imager with four 4Ã4 receptive fields shown in FIG. 2A where each receptive field is connected to four AND gates a according to an embodiment;
FIG. 3B is a close up of the wiring diagram of the receptive field circuit and event generation circuit in FIG. 3A according to an embodiment;
FIG. 4A is a detailed block diagram of a first part of the feature detection circuit for each receptive field according to an embodiment;
FIG. 4B is a detailed block diagram of a second part of the feature detection circuit for each receptive field according to an embodiment
FIG. 5A is an experimental setup with a SPAD field of view marked by black dotted line a according to an embodiment;
FIG. 5B shows 500 averaged frames showing a stationary B-747 model viewed using the experimental setup of FIG. 5A ;
FIG. 5C shows the result of averaging 5 frames showing an F-14 model plane as it drops at high speed close to the camera lens in the experimental setup of FIG. 5A ;
FIG. 5D is a representation of a raw SPAD input showing common sources of noise according to an embodiment;
FIG. 5E is a representation of SPAD activation across pixels and time according to an embodiment;
FIG. 6A is a histogram over frames 1 to 270 showing stray photons vs. missed detection when the airplane was not in front of the pixel where a cross indicates false positives;
FIG. 6B is a histogram over frames 271 to 481 showing stray photons vs. missed detection when the airplane was in front of the pixel where a cross indicates false negatives;
FIG. 7 shows examples of a persistent background pattern in maxed out counters where panels (a) (b) and (c) show in high contrast the timing patterns in consecutive frames for the pixels whose counters have maxed out and panel (d) highlights the persistence of the pattern by comparing a small section from each frame next to each other according to an embodiment;
FIG. 8 is a comparison between noisy pixels and normal pixels showing SPAD timings averaged over 3600 frames according to an embodiment;
FIG. 9 is a representation showing the number of receptive fields impaired by a single photon at different receptive field sizes where X marks the dead pixel, the black grid indicated the pixel region affected by the dead pixel and the coloured squares indicate receptive fields affected by the dead pixel according to an embodiment; and
FIG. 10 is a schematic diagram of a Flash LIDAR system using a SPAD array according to an embodiment.
In the following description, like reference characters designate like or corresponding parts throughout the figures.
DESCRIPTION OF EMBODIMENTS
Embodiments of a single photon avalanche detector (SPAD) array microchip, and methods of operation are described. With conventional (Frame based) SPAD microchip designs as used in 3D ToF cameras, each pixel or cell contains 1 or more SPADs and corresponding digital circuitry to time when photon events occur. For every pixel the ToF data is collectively communicated off chip synchronous with a laser pulse. With more pixels (higher resolution arrays) come the problems of how to reduce circuit complexity without compromising functionality and how to move large amounts of ToF data off chip. Embodiments of a single photon avalanche detector (SPAD) array microchip, and methods of operation are described in which the inter pixel photon arrival order (not time) is recorded and transmitted off chip.
Referring now to FIG. 1A , there is shown a schematic diagram of a SPAD array sensor 1 according to an embodiment. The SPAD array sensor comprises a SPAD array 2 and a SPAD Array control circuit 3 . The SPAD array 2 comprises an array of SPAD cells 110 , arranged in n rows and m columns (i.e. a nÃm array). This is illustrated in FIG. 1 , in which the first row comprises a first SPAD cell 10 in the first column, a second SPAD cell 111 in the second column, etc. to the mth SPAD cell 112 in the mth column. This structure is repeated for n rows, and thus the nth row comprises the ((nâ1)Ãm)+1 th SPAD cell 114 in the first column of the nth row to the nÃm th SPAD cell 116 in the mth column of the nth row. The SPAD array control circuit performs a range of functions such as providing power to the array, and controlling trigger enablement and resetting of SPAD sensors or SPAD cells, and reading of data from cells in the array.
FIG. 1B is a schematic diagram of adjacent SPAD cells
110 and 111 according to an embodiment. In this embodiment the first SPAD cell 110 comprises a SPAD structure 120 , and a quenching circuit 130 . The second SPAD cell 111 is of identical construction to the first SPAD cell, and comprises a second SPAD structure 121 and a second quenching circuit 131 . As will be discussed below a triggering circuit detects triggering of the SPAD sensors. As will be discussed below, in some embodiments the triggering circuit comprises neural feature circuits. In one embodiment the quenching circuits
130 , 131 are analog circuits and the triggering circuit is a digital circuit.
The SPAD array microchip may be constructed using a range of CMOS technologies, for example from 0.8 μm down to 65 nm (and potentially lower) incorporating multiple metal layers and voltages, and may be designed using VLSI and related IC chip design techniques. In one embodiment the SPAD array sensor is constructed as a monolithic CMOS wafer comprised of the SPAD array and a SPAD control circuit. In other embodiments, multi-wafer layering techniques can be used, in which the top wafer layer is a comprised of an array of SPAD structures, and the lower wafer layer(s) comprise associated SPAD structure quenching circuit and triggering circuits connected to SPAD structures in the top wafer layer, and the SPAD array control circuit.
The SPAD sensor comprises a SPAD structure which comprises an active region, within which an incident photon will generate an output current pulse, which may also be referred to as a trigger pulse as well as anode and cathode connections for application of the bias voltage and detection of the trigger pulse. Various SPAD structures may be used with different SPAD structures having different efficiencies and characteristics.
A quenching circuit is used to control the bias voltage applied to the SPAD structure and quench (i.e. stop) the avalanche process by reducing the bias voltage below the breakdown voltage. The quenching circuit may also be referred to as a quenching and recharge circuit, as once the avalanche process is quenched, the bias voltage is again raised to the operational voltage above the breakdown voltage to allow the SPAD to be triggered again. Quenching may be performed using active or passive quenching. Passive quenching is performed by providing a resistance in series to the SPAD. The avalanche current self-quenches simply because it develops a voltage drop across the resistance (a high-value ballast load), reducing the voltage across the SPAD to below its breakdown voltage. After the quenching of the avalanche current, the SPAD's bias slowly recovers to at or above the breakdown voltage. An alternative to passive quenching is active quenching. An active quenching circuit detects avalanche breakdown by a digital logic circuit or component, for example a transistor acting as a resistive element or a transistor based circuit connected to the SPAD output. Upon detection of a trigger pulse, the active quenching circuit actively pulls the bias voltage to a voltage below breakdown to quench the avalanche, before switching the voltage back to the operational voltage. Compared to passive quenching, active quenching circuits occupies less space on the chip and have reduced dead time, as well as providing options such as time gating of the SPAD to synchronise with firing of a laser pulse. In one embodiment the quenching circuits are analog circuits and the triggering circuits are digital circuits.
FIG. 1C is a flowchart of a method of operation of a SPAD array microchip 100 according to an embodiment. In contrast to traditional SPAD DTOF systems, the inter pixel photon arrival order (not time) is recorded and transmitted off chip using this method. The method broadly comprises step 102 of detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval, wherein the timing interval comprises one or more frames. In some embodiments, only one event is read per frame (where the start of the frame is defined by the laser firing), and in other embodiments, multiple events can be read per frame, each from different receptive fields (described below). In one embodiment the detection step comprises detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame. At step 104 , the method continues with reading out an address of each triggered SPAD sensor or an identifier that identifies the location of a group of triggered SPAD sensors in the order of triggering during the timing interval. In some embodiment this comprises reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors (i.e. indirectly reading out the addresses of the triggered SPAD sensors), or the addresses of the SPAD sensors in the group of SPAD sensors. This is only performed if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.
In one embodiment the design is a âsynchronousâ logic circuit with an external clock that drives all the internal logic circuits. SPADs avalanching are synchronised to this clock (although in other embodiments this restriction can be relaxed) so that âeventsâ triggered by the receptive fields occur synchronous to this clock. Exactly when these events actually occur is unknown, and thus the system can be described as asynchronous in nature between frames (the frame itself is synchronised to the start of the laser pulse). The time when the event occurs is not time stamped (or recorded). However in one embodiment a global counter could be used to time stamp events: In this embodiment the chip operates in timing mode, and is synchronized to the firing of a laser pulse where each laser pulse presents the start of a new frame. A counter is latched at the time the event is read off the chip and based on the clock frequency (period of each clock tick) a time can be determined which can be converted to a distance by multiplying by the speed of light. This allows the use of a single counter for the chip, with this counter synchronised to the laser pulse (i.e. start of the frame). This counter could be located on the chip or off the chip as part of a control circuit 3 associated with reading events off the chip and resetting the chip. In one embodiment the control circuit comprises a processor such as a FPGA which reads the addresses of events off the chip and which comprises a counter which is read each time an event is read off the chip. A timing interval can be defined which is a plurality of frames. Over successive frames new events corresponding to newly detected features are read out and the time latched. In some embodiments only one event is read out per frame, and over a consecutive series of frames (the timing interval) a 3D representation of the scene can be created. In other embodiments, multiple events per frame may be read out and the times latched in which case several counters may be required, or the counter value must be latched and stored with the event without stopping the counter.
This method and embodiments of a microchip and associated circuits for implementing this method will now be discussed in greater detail to more clearly highlight the various features of the method and implementation.
To assist in understanding the invention we will first consider an embodiment where the SPAD array microchip is a regular array with SPAD sensors arranged in an RÃC grid. In this event based triggering approach we introduce the concept of a receptive field. Each receptive field 210 is group of SPAD sensorsâequivalently a macro cell of neighbouring SPADs. Preferably each receptive field is a rÃr array of SPAD sensors or a contiguous block of SPAD sensors. The receptive fields are tiled across the array in an overlapping pattern so that adjacent receptive fields share some but not all of the same SPAD sensors. For example in a 128Ã128 SPAD array chip, with receptive fields with a 4Ã4 shape, the entire chip contains 125Ã125 receptive field block and each receptive field shares overlapping SPADs with adjacent receptive fields.
FIG. 2A is an illustrative example of a 5Ã5 SPAD array microchip 200 with four 4Ã4
receptive fields
212 , 214 , 216 , and 218 . That is each receptive field connects 16 neighbouring SPADs in a 4Ã4 arrangement. Each receptive field is assigned an address based on the (row, column) location of the top corner. For example in FIG. 2A , the first receptive field 212 is indicated with lightly shaded boxes (coloured yellow in the provisional) and has an address of rf(1, 1), the second receptive field 214 is shifted one column to the right and has slightly heavier shading (coloured green in the provisional) has an address of rf(1, 2). The third receptive field 216 is shifted one row down and has medium shading (coloured red in the provisional) and has an address of rf(2, 1), and the fourth receptive field 218 is shifted one column right and one column down and is more darkly shaded (coloured blue in the provisional) and has an address of rf(2, 2). The overlapping receptive fields are shown on 5Ã5 SPAD array microchip 200 . The interior 3Ã3 cells have the darkest shading, each corner has the shading of their respective source ( 212 , 214 , 216 , 218 ) and the 3 border have intermediate shading based on the two bordering end cells.
Each receptive field comprises its own circuitry that digitally processes its own group of SPAD cells to identify the presence of one or more features with a predefined triggering pattern. These will be referred to as neural feature subgroups or neural feature circuits. Each neural feature circuit comprises a subgroup of the SPAD sensors in the receptive field, and the neural feature circuit is configured to detect a triggering pattern within this subgroup. The neural feature groups are thus feature shapes, and are analogous to silicon neurons. Each receptive circuit comprises two or more neural feature subgroups or circuits and where each neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors. In the present example 4 features are used, but it could be higher or lower as required.
In most embodiments, the neural feature circuits are created using an AND gate which takes as input a local group of SPAD sensors and which outputs a neural feature trigger signal when all the input SPADs are triggered (i.e. the AND gate is use to combine all of the SPADs in the subgroup so the feature is only triggered when all SPADs in the feature have triggered). However other more complex digital circuits can be used to capture more complex triggering patterns, including non-triggering of some SPADs in the neural feature group. In this case a logic circuit generates a neural feature trigger signal when the predetermined trigger pattern of SPAD is satisfied.
Preferably the number of input SPAD sensors per AND gate is equal for each neural feature circuit so as to result in an equal probability of AND gate activation. That is each of the neural features in a receptive field comprises the same number of SPAD sensors. However the pattern of connectivity and its correspondence to the observed data also determines AND gate activation. For example an edge bar is more likely to be activated in a natural environment in comparison to a checker board pattern since the latter is not typically observed in the environment. In this way each AND gate encodes a local feature and it's activation indicates that all its input signals have been activated. Thus in most embodiments each of the neural feature circuits in a receptive field are contiguous block with the same shape, for example a rectangular 2Ã4 or 4Ã2 group of SPADs. In other embodiments the neural features could comprise solid squares, lines, rectangles, L shaped blocks, regular polygons (hexagon, octagon), crosses, etc. In the example shown in FIG. 2A each receptive field comprises 16 SPAD sensors, with four neural features.
FIG. 2B is a schematic representation of a 4Ã4 receptive field 210 illustrating the connection of the 16 SPAD sensors to four AND
gates
221 , 223 , 225 , and 227 to create four
neural feature circuits
222 , 224 , 226 , and 228 . In this embodiment each neural feature group is a block of 8 adjacent SPAD sensors arranged in a 2Ã4 or 4Ã2 grid (i.e. edge bar patterns). Each neural feature circuit has an associated address for example based on the (row, column) location of the receptive field, and an identifier of the neural feature circuit. For example in this embodiment the
circuits
222 , 224 , 226 , and 228 are in north, east, south and west locations and are assigned
identifiers
1, 2, 3, 4. A unique address for the SPAD arrays in a neural feature circuit can thus be provided by the (row, column) address of the receptive field, and the identifier (n=1 . . . 4) of the neural circuit within the receptive field. In other embodiments the address of the neural feature could be specified by the (row, column) of a first or reference cell in the neural feature shape (e.g. top left corner of the shape). In one embodiment, the unique address of the receptive field is known and only the âwinningâ neuron feature (or feature event) within that receptive field is known (or detected/stored). One neural circuit is provided for every receptive field and is configured to record/detect/store the neuron feature that triggered an event. In this embodiment the address of âevery neural featureâ in a receptive field is not known or recorded.
FIG. 2C is a representation of a 3D visual scene and the resultant SPAD timing pattern for the 5Ã5 SPAD array microchip of FIG. 2A according to an embodiment. In this embodiment a box 230 is in the foreground and returns a signal 232 over a first time t 1 =4 ns. The background 240 is far away (i.e. no objects) at the maximum range (time t 2 =8 ns). The box occupies pixels (row=3:5 and col=2:5), then the lower/south AND of rf(1,2) (green) and rf(2,2) (blue) will latch which can be expressed as AND(1,2,3) and AND(2,2,3) latching at t=4 ns. The resultant image of time 250 on the 5Ã5 SPAD imager is colour coded according to the photon arrival time or equivalently the SPAD latch time (nsec) for each detected event. In this figure the image comprises a 3Ã4 medium shaded (orange in the provisional application) block of pixels corresponding to a time of 4 ns, and the remaining pixels are dark shaded (red in the provisional) at time t 2 =8 ns.
Another feature of the receptive field circuit, is that as a neural feature trigger signal is generated, an inhibit signal is also generated which suppresses (i.e. prevents) triggering of the other neural feature circuits in the same receptive field circuit during the current frame.
That is the going high or latching of the first AND gate (first neural feature circuit) prevents subsequent latching of any later neurons (other neural feature circuit) via a recurrent enable connection that gates all the ANDs. Thus, at the beginning of the pulse cycle when the laser pulse is sent, the enable to all ANDS for all the receptive fields are high allowing any AND gate to latch but then for each receptive field as soon as the first AND gate latches the enable to the other ANDs of this receptive field is set low. Note that the enable flag going low does not affect the first AND gate that caused the lowering in the first place. This is achieved by the AND gate blocking its own blocking signal. This is realised through the following logic: an AND gate can only be high if and only if all its input pixels are high and (the local receptive fields enable flag is high or the AND was already high (In the previous time step in a synchronous system or an asynchronous memory in an asynchronous implementation)).
This is illustrated in FIG. 3A is a schematic wiring diagram of the 5Ã5 imager with four 4Ã4 receptive fields shown in FIG. 2A where each receptive field is connected to four AND gates a according to an embodiment. FIG. 3B is a close up of the wiring diagram of the receptive field circuit and event generation circuit in FIG. 3A according to an embodiment. A first receptive field 320 rf(1,1) comprises a neural feature circuit 320 with a 2Ã4 geometry of highlighted SPADS (in red), labelled 301 , 302 , 303 , and 304 (row 3); and 311 , 312 , 313 , and 314 (row 4). Column 5 SPAD 304 in row 3 and SPAD 315 in row 5 are not part of this receptive field. The wires carry binary on/off values from the SPAD sensors (trigger output) to four AND gates labelled AND_south, AND_north, AND east and AND_west as shown in FIG. 3B . The AND_south gate receives inputs from neural feature circuit 320 (
SPADs
301 , 302 , 303 , 304 , 311 , 312 , 313 , and 314 ). The output of AND_south is provides as input In1 to event circuit 340 . Similarly the three other AND gates for the three other neural feature circuits are provided to inputs In2 In3 and In4 of the event circuit 340 . Note that if only the highlighted SPADS (in red, labelled 301 , 302 , 303 , 304 , 311 , 312 , 313 , and 314 ) latch at t=t 1 then the AND_south (or AND(1,1,3)) will latch at t=t 1 .
The event circuit 340 is configured to generate an event trigger for the receptive field if one of the input neural feature subgroup triggers. As will be discussed below, the trigger is only generated if the neural feature circuit which triggered is the first in the receptive field to trigger in the timing interval or if the triggered neural feature circuit is different to the neural feature circuit which previously triggered in the timing interval for this receptive field.
If a trigger (EventFlag) is generated by the event trigger then the address of the neural feature is read out (e.g. using neuronAddress0 for the row, and neuronAddress2 for the column) on a data bus. This may be the (row, column) address of the receptive field, e.g. rc(1, 1) and the identifier, e.g. In1, is read out. Alternatively the addresses of the SPADs could be read out.
FIGS. 4A and 4B are detailed block diagram of a first and second part of the <figure-callout id="340" label="feature detection circuit" filenames="US11474215-20221018-D00005.png,US11474215-20221018-D00006.png" state="{{stat
CLAIMS
Claims ( 22 )
The invention claimed is:
1. A method for operation of a Single Photon Avalanche Diode (SPAD) array microchip, the method comprising:
detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval comprising detecting, on board the SPAD array microchip, the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame, wherein the timing interval comprises one or more frames wherein a start of each frame is defined by a synchronisation signal and the triggering time of each SPAD sensor is not time stamped or recorded; and
reading out an address of each triggered SPAD sensor, or an identifier that identifies the location of a group of triggered SPAD sensors, in the order of triggering during the timing interval comprising reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors, or the addresses of the SPAD sensors in the group of SPAD sensors, only if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.
2. The method as claimed in claim 1 , wherein:
the plurality of SPAD sensors are each associated with one or more or a plurality of receptive field groups where each receptive field groups comprises a receptive group of SPAD sensors, and different receptive field groups may share some but not all of the same SPAD sensors; and
the plurality of predefined triggering patterns comprises a plurality of neural feature subgroups where each receptive field comprises two or more neural feature subgroups where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors; and
detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns comprises:
detecting the triggering of a neural feature subgroup of a receptive field, and wherein upon detection the triggering of each other neural feature subgroup in the same receptive field group is suppressed until the end of the current frame;
generating an event trigger for the receptive field group if the triggered neural feature subgroup is the first neural feature subgroup in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature subgroup which previously triggered in the timing interval for this receptive field;
and the step of reading out an identifier is performed upon generation of an event trigger, and comprises reading an address of the neural feature subgroup that triggered.
3. The method as claimed in claim 2 , wherein generating an event trigger for the receptive field group further comprises
incrementing a counter if the neural feature subgroup that triggered is the same neural feature subgroup that generated the previous trigger or the counter is zero;
generating an event trigger for the receptive field group only when the counter reaches a predefined counter threshold value wherein the predefined counter threshold value is fixed for at least the timing interval.
4. The method as claimed in claim 3 , wherein the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.
5. The method as claimed in claim 2 , wherein each receptive field group has the same geometry each of the neural feature subgroups in a receptive field group comprises the same number of SPAD sensors.
6. The method as claimed in claim 5 , wherein the plurality of SPAD sensors are arranged in a RÃC grid, and each receptive field group is an rÃr array of SPAD sensors, and the plurality of receptive field groups overlap, and the plurality of overlapping receptive field groups tile the RÃC grid.
7. The method as claimed in claim 5 , wherein each neural feature group comprises a contiguous block of the rÃr array of SPAD sensors in the receptive field group.
8. The method as claimed in claim 5 , wherein r=4, and each neural feature group comprises a block of 8 adjacent SPAD sensors arranged in a 2Ã4 or 4Ã2 grid.
9. The method as claimed in claim 2 , wherein after reading out an identifier the plurality of SPADs are reset and the current frame ends, an in use the synchronisation signal is synchronised to the firing of a laser pulse.
10. A Single-Photon Avalanche Diode (SPAD) array microchip comprising:
a plurality of SPAD sensors;
a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames wherein a start of each frame is defined by a synchronisation signal and the triggering time of each SPAD sensor is not time stamped or recorded and the triggering circuit comprises:
a plurality of feature detection circuits, each feature detection circuit connected to a group of SPAD sensors and configured to generate a feature trigger when the connected SPADs satisfy a triggering criteria; and
an event detection circuit configured to detect the first time a feature trigger is generated by a feature detection circuit during a timing interval wherein the timing interval comprises one or more frames and to read out the address of either the feature detection circuit or the connected SPAD sensors off the microchip.
11. The SPAD array microchip as claimed in claim 10 , wherein the feature detection circuits are neural feature detection circuits and the triggering circuit further comprises:
a plurality of receptive field circuits wherein each receptive field circuit comprises:
a receptive group of SPAD sensors;
two or more neural feature circuits, wherein each neural feature circuit comprises a subgroup of SPAD sensors in the receptive group of SPAD sensors where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors, and each neural feature circuit is configured to generate a neural feature trigger signal and an inhibit signal if all the SPAD sensors in the subgroup trigger, wherein the inhibit signal prevents triggering of the other neural feature circuits in the same receptive field circuit during the current frame;
the event detection circuit comprises an event detection circuit for each receptive field circuit which is configured to monitor the neural feature circuits for generation of a neural feature trigger signal, and generates an event trigger for the receptive field circuit if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval; and
the triggering circuit further comprises a read out circuit which detects an event trigger from a receptive field circuit, and on detection of an event trigger it reads an address of the neural feature circuit that triggered generation of the event trigger.
12. The SPAD array microchip as claimed in claim 11 , each receptive field circuit further comprises a memory, and when a neural feature trigger signal is generated an identifier of the neural feature circuit that generated the neural feature trigger signal in stored in the memory, and the event detection circuit is configured to determine if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval by comparing an identifier of the neural feature circuit that generated the neural feature trigger with the identifier stored in the memory.
13. The SPAD array microchip as claimed in claim 12 , wherein each receptive field circuit further comprises a counter wherein the counter is incremented the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the identifier of the neural feature circuit that generated the neural feature trigger is the same as the identifier stored in the memory, and an event trigger is generated when the counter is equal to a predetermined threshold value fixed for at least the timing interval.
14. The SPAD array microchip as claimed in claim 13 , wherein the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.
15. The SPAD array microchip as claimed in claim 13 , wherein the predetermined threshold value is adjustable between timing intervals.
16. The SPAD array microchip as claimed in claim 12 , wherein each of the neural feature circuits in a receptive field comprises the same number of SPAD sensors.
17. The SPAD array microchip as claimed in claim 16 , wherein the plurality of SPAD sensors are arranged in a RÃC grid, each receptive field circuits is an rÃr array of SPAD sensors, and the plurality of receptive field circuits overlap, and the plurality of overlapping receptive field circuit tile the RÃC grid.
18. The SPAD array microchip as claimed in claim 17 , wherein each neural feature circuit comprises a contiguous block of the rÃr array of SPAD sensors in the receptive field circuit.
19. The SPAD array microchip as claimed in claim 18 , wherein r=4, and each neural feature circuit comprises a block of 8 adjacent SPAD sensors arranged in a 2Ã4 or 4Ã2 grid.
20. The SPAD array microchip as claimed in claim 12 , wherein each neural feature circuit comprises a logical circuit which ANDs each of the subgroup of SPAD sensors.
21. The SPAD array microchip as claimed in claim 10 wherein the chip further comprises a data bus and an Address Event Representation protocol is used to read data off the microchip.
22. The SPAD array microchip as claimed in claim 10 wherein after reading out the address the plurality of SPAD sensors are reset and the current frame ends, an in use the synchronisation signal is synchronised to the firing of a laser pulse.
US16/650,561
2017-09-27
2018-09-27
Neuromorphic single photon avalanche detector (SPAD) array microchip
Active
2039-08-07
US11474215B2
( en )
Applications Claiming Priority (3)
Application Number
Priority Date
Filing Date
Title
AU2017903926A
AU2017903926A0
( en )
2017-09-27
A neuromorphic single photon avalanche diode (spad) array microchip
AU2017903926
2017-09-27
PCT/AU2018/000187
WO2019060942A1
( en )
2017-09-27
2018-09-27
A neuromorphic single photon avalanche detector (spad) array microchip
Publications (2)
Publication Number
Publication Date
US20200326414A1
US20200326414A1 ( en )
2020-10-15
US11474215B2
true
US11474215B2 ( en )
2022-10-18
Family
ID=65900174
Family Applications (1)
Application Number
Title
Priority Date
Filing Date
US16/650,561
Active
2039-08-07
US11474215B2
( en )
2017-09-27
2018-09-27
Neuromorphic single photon avalanche detector (SPAD) array microchip
Country Status (4)
Country
Link
US
( 1 )
US11474215B2
( en )
EP
( 1 )
EP3688809B1
( en )
AU
( 1 )
AU2018341551B9
( en )
WO
( 1 )
WO2019060942A1
( en )
Families Citing this family (20)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
US11272112B2
( en )
*
2018-08-08
2022-03-08
Canon Kabushiki Kaisha
Image capturing apparatus
US11579263B2
( en )
*
2018-11-29
2023-02-14
Taiwan Semiconductor Manufacturing Co., Ltd.
Method and apparatus for a hybrid time-of-flight sensor with high dynamic range
US12212871B2
( en )
2020-05-29
2025-01-28
Voxelsensors Srl
Pixel sensor system
EP3989535B1
( en )
*
2020-10-20
2025-09-24
Canon Kabushiki Kaisha
Photoelectric conversion device, method of controlling photoelectric conversion device, and information processing apparatus
US11726187B2
( en )
*
2020-10-30
2023-08-15
Taiwan Semiconductor Manufacturing Co., Ltd.
High resolution low power inter-correlation SPAD assisted time-of-flight sensor
EP4256778B1
( en )
*
2020-12-23
2024-10-23
VoxelSensors SRL
Neighborhood-gated switching pixel sensor
BE1028950B9
( en )
*
2020-12-23
2023-10-09
Voxelsensors Srl
NEIGHBORHOOD SWITCHED PIXEL SENSOR
CN116648924A
( en )
*
2020-12-23
2023-08-25
ä½ç´ ä¼ æå¨æéè´£ä»»å ¬å¸
Neighborhood Gated Switching Pixel Sensors
US11580713B2
( en )
*
2021-01-12
2023-02-14
Microsoft Technology Licensing, Llc
Motion compensation for a SPAD array camera
CN115128572B
( en )
*
2021-03-24
2025-07-08
æ·±å³å¼ææºè½ææ¯æéå ¬å¸
Signal receiving device, detecting device, signal processing method and device thereof
BE1029775B9
( en )
*
2021-09-21
2023-10-09
Voxelsensors Srl
PERSISTENCE FILTERING AT SPD ARRAYS
JP7757107B2
( en )
*
2021-09-24
2025-10-21
ãã¤ãã³æ ªå¼ä¼ç¤¾
Photoelectric conversion device, imaging device, control method, and computer program
CN114638349B
( en )
*
2022-01-27
2024-05-28
æ¸ å大å¦
Photosensitive neurons, sensing, storage and computing integrated intelligent sensing device, method and medium
CN116609769B
( en )
*
2022-02-09
2026-03-20
宿³¢é£è¯çµåç§ææéå ¬å¸
Pixel array
US12063040B2
( en )
2022-09-28
2024-08-13
Stmicroelectronics (Research & Development) Limited
Reconfigurable high voltage generation circuit for SPAD sensors
US12684258B2
( en )
*
2023-02-21
2026-07-14
Canon Kabushiki Kaisha
Image capturing apparatus, image capturing system, and equipment
DE112024003185T5
( en )
2023-08-01
2026-07-02
Ams-Osram Ag
2D TDC arrangement
CN122477390A
( en )
*
2024-04-12
2026-07-28
å¾è¾¾éæºè½ç¾å½æéå ¬å¸
SPAD side photodetector
US12470846B1
( en )
*
2024-05-10
2025-11-11
Rockwell Collins, Inc.
Noise reduction method for very low light level night vision
CN121691956B
( en )
*
2026-02-11
2026-05-12
西å®çµåç§æå¤§å¦æå·ç ç©¶é¢
Light-weight reading circuit for large-array SPAD real-time data compression
Citations (4)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2013018006A1
( en )
2011-08-03
2013-02-07
Koninklijke Philips Electronics N.V.
Position-sensitive readout modes for digital silicon photomultiplier arrays
US20150285625A1
( en )
2014-04-07
2015-10-08
Samsung Electronics Co., Ltd.
High resolution, high frame rate, low power image sensor
WO2017004663A1
( en )
2015-07-08
2017-01-12
The Commonwealth Of Australia
Spad array structures and methods of operation
US20170131143A1
( en )
*
2014-07-02
2017-05-11
Andreas G. Andreou
Photodetection circuit and operating method thereof
2018
2018-09-27
EP
EP18862953.9A
patent/EP3688809B1/en
active
Active
2018-09-27
WO
PCT/AU2018/000187
patent/WO2019060942A1/en
not_active
Ceased
2018-09-27
US
US16/650,561
patent/US11474215B2/en
active
Active
2018-09-27
AU
AU2018341551A
patent/AU2018341551B9/en
active
Active
Patent Citations (7)
* Cited by examiner, â Cited by third party
Publication number
Priority date
Publication date
Assignee
Title
WO2013018006A1
( en )
2011-08-03
2013-02-07
Koninklijke Philips Electronics N.V.
Position-sensitive readout modes for digital silicon photomultiplier arrays
US20140175294A1
( en )
2011-08-03
2014-06-26
Koninklijke Philips N.V.
Position-sensitive readout modes for digital silicon photomultiplier arrays
US9176241B2
( en )
2011-08-03
2015-11-03
Koninklijke Philips N.V.
Position-sensitive readout modes for digital silicon photomultiplier arrays
US20150285625A1
( en )
2014-04-07
2015-10-08
Samsung Electronics Co., Ltd.
High resolution, high frame rate, low power image sensor
US20170131143A1
( en )
*
2014-07-02
2017-05-11
Andreas G. Andreou
Photodetection circuit and operating method thereof
WO2017004663A1
( en )
2015-07-08
2017-01-12
The Commonwealth Of Australia
Spad array structures and methods of operation
US20180195900A1
( en )
2015-07-08
2018-07-12
The Commonwealth Of Australia
SPAD Array Structures and Methods of Operation
Non-Patent Citations (2)
* Cited by examiner, â Cited by third party
Title
Belbachir et al., " High-precision shape representation using a neuromorphic vision sensor with synchronous address-event communication interface ", Measurement Science and Technology, 2009, pp. 1-9, vol. 20.
Berkovich et al., " A Scalable 20x20 Fully Asynchronous SPAD-Based Imaging Sensor with AER Readout ", IEEE International Symposium on Circuits and Systems (ISCAS), 2015, pp. 1110-1113.
Also Published As
Publication number
Publication date
AU2018341551B9
( en )
2023-03-09
US20200326414A1
( en )
2020-10-15
AU2018341551B2
( en )
2022-11-03
WO2019060942A1
( en )
2019-04-04
EP3688809B1
( en )
2025-08-20
EP3688809A1
( en )
2020-08-05
EP3688809A4
( en )
2021-07-28
AU2018341551A1
( en )
2020-05-07
Similar Documents
Publication
Publication Date
Title
EP3688809B1
( en )
2025-08-20
A neuromorphic single photon avalanche detector (spad) array microchip
US12189038B2
( en )
2025-01-07
Processing system for LIDAR measurements
CN110389358B
( en )
2023-05-19
Related time-of-flight sensor
JP6761000B2
( en )
2020-09-23
Photoelectric sensors and methods for measuring distance
EP3320559B1
( en )
2021-03-03
Spad array structures and methods of operation
Niclass et al.
2012
A 100-m Range 10-Frame/s 340$\,\times\, $96-Pixel Time-of-Flight Depth Sensor in 0.18-$\mu\hbox {m} $ CMOS
Gyongy et al.
2023
A direct time-of-flight image sensor with in-pixel surface detection and dynamic vision
KR102409952B1
( en )
2022-06-17
High resolution, high frame rate, low power image sensor
TWI743471B
( en )
2021-10-21
First photon correlated time-of-flight sensor
US11626446B2
( en )
2023-04-11
Pixel circuit and method of operating the same in an always-on mode
KR20210150765A
( en )
2021-12-13
Image Sensing Device and Image photographing apparatus including the same
US11442153B2
( en )
2022-09-13
Systolic processor system for a light ranging system
US20250341619A1
( en )
2025-11-06
High photon rate spad device
Kumar et al.
2012
Low power time-of-flight 3D imager system in standard CMOS
US20240385325A1
( en )
2024-11-21
Direct-time-of-flight device, system, and method
US20250012902A1
( en )
2025-01-09
Time-of-flight circuitry and time-of-flight method
NZ738738B2
( en )
2021-11-30
Spad array structures and methods of operation
Legal Events
Date
Code
Title
Description
2020-03-25
FEPP
Fee payment procedure
Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY
2020-07-27
STPP
Information on status: patent application and granting procedure in general
Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION
2020-08-20
AS
Assignment
Owner name : THE COMMONWEALTH OF AUSTRALIA, AUSTRALIA
Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DELIC, DENNIS VICTOR;AFSHAR, SAEED;SIGNING DATES FROM 20200526 TO 20200603;REEL/FRAME:053549/0064
2022-04-04
STPP
Information on status: patent application and granting procedure in general
Free format text : NON FINAL ACTION MAILED
2022-06-08
STPP
Information on status: patent application and granting procedure in general
Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER
2022-06-27
STPP
Information on status: patent application and granting procedure in general
Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS
2022-08-25
STPP
Information on status: patent application and granting procedure in general
Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED
2022-08-26
STPP
Information on status: patent application and granting procedure in general
Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED
2022-09-28
STCF
Information on status: patent grant
Free format text : PATENTED CASE
2026-03-10
MAFP
Maintenance fee payment
Free format text : PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY
Year of fee payment : 4
Related documents
- Memory- and Bandwidth-Efficient SPAD-LiDAR Ranging via Coarse-to-Fine Spline Sketching#1129727
- Characterization of a low-power 3D photon-to-digital converter readout improved for system integration in meter-scale applications#822264
- Optoelectronic Reservoir Computing with an On-Chip True-Time-Delay Element#973526
- In situ characterization of a photon-subtraction device via heralding counts and homodyne detection#997015
- Developing a photon-number-resolving detection chain for quantum communication protocols involving mesoscopic states of light#971806
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive