Conceptio › Archive › Google Patents
Google Patents open access

Neuromorphic single photon avalanche detector (SPAD) array microchip — The Commonwealth Of Australia (US11474215B2)

The Commonwealth Of Australia · Google Patents
Google Patents · Patents
Open Source ↗
engoogle-patentsintellectual-propertypatent
1. A method for operation of a Single Photon Avalanche Diode (SPAD) array microchip, the method comprising: detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval comprising detecting, on board the SPAD … Neuromorphic single photon avalanche detector (SPAD) array microchip — The Commonwealth Of Australia (US11474215B2)
Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

ABSTRACT

Abstract

Described is a Single-Photon Avalanche Diode (SPAD) array microchip comprising: a plurality of SPAD sensors; and a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames. An event based neuromorphic SPAD array microchip is also described. The chip architecture and triggering methodology takes a local group of SPAD sensors connected in a certain way and by using simple digital circuits emulating how neurons behave, patterns within a local receptive field are identified. Only when these unique patterns or features are identified are “events” triggered for each receptive field in the order they occur, or in an asynchronous manner. Each neuromorphic circuit (or collection of silicon neurons) act over overlapping receptive fields, and are tiled across the entire visual spatial field of the SPAD array to a form a convolution layer.

Description

This application is the United States national phase of International Application No. PCT/AU2018/000187 filed Sep. 27, 2018, and claims priority to Australian Provisional Patent Application No. 2017903926 filed Sep. 27, 2017, the disclosures of which are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to sensor circuits comprising an array of Single-Photon Avalanche Diode (SPAD) circuits for three dimensional (3D) camera systems.

BACKGROUND

Avalanche photodiodes (APDs) are solid-state photodetectors in which a high bias voltage is applied to a p-n junction to provide a high first stage gain due to avalanche multiplication. Avalanche multiplication occurs when an incident photon with sufficient energy to liberate an electron arrives at the photodiode. The high electric field rapidly accelerates the photo-generated electron towards the anode, but before it can reach the anode, it collides with the intervening doped material releasing further electrons, all of which are then accelerated towards the anode. This process repeats leading to avalanche multiplication of the photo-generated electron and an output current pulse. APDs are thus semiconductor analogs to photomultiplier tubes. Linear mode APDs are effectively single stage linear amplifiers in which the gain is set by controlling the bias voltage, and with gain factors of several hundred can be achieved in linear mode.

Single-Photon Avalanche Diodes (SPADs) are APDs in which the p-n junction is biased above its breakdown voltage to operate in Geiger mode such that a single incident photon will trigger an ongoing avalanche breakdown and thus easily measureable current pulse. That is a SPAD operates as a trigger device generating a large current pulse compared to linear mode APDs in which the current pulses can be very low at low light intensity. After triggering of the SPAD, a quenching circuit is used to reduce the bias voltage below the breakdown voltage in order to quench the avalanche process. Once quenched the bias voltage is again raised above the breakdown voltage to reset the SPAD for detection of another photon (known as re-biasing the SPAD).

APD and SPADs are solid state devices and can be constructed using a variety of CMOS technologies, and have very small active areas compared with other photon counting devices such as photomultiplier tubes. Through appropriate choice of materials and structure, wavelength sensitivity of a SPAD can be controlled to be in the visible and/or near-infrared range. A SPAD combined with additional circuitry to count pulses and/or measure time of arrival of photons to sub-nano/pico second accuracy can be used to create sensors for ultra-low light imaging or highly sensitive time-resolved imaging applications. For example one potential application of a SPAD arrays are in three dimensional (3D) Flash LIDAR cameras, as they have the potential to provide extremely sensitive devices with high distance resolution and high frame rates. SPAD arrays also have potential for use in other applications that require single photo sensitivity with high frames such as biological/medical imaging applications, adaptive optics applications, and astrophysics applications.

Three dimensional Flash LIDAR systems, also known as 3D Time of Flight (TOF) Cameras, use a laser source to irradiate a target with a short duration laser pulse (i.e. a laser flash). Photons are back scattered off objects and onto the sensor and the time of arrival is used to determine time of flight and thus distance to an object. The first 3D TOF cameras were constructed as scanning laser systems comprising a laser range finder with a rotating or scanning element(s) to progressively scan the field of view. These scanning systems are effectively single pixel devices collecting time of flight information in a single direction which build up a 3D image progressively moving the pointing direction of the sensing element. More recently scanner-less (i.e. staring) 3D Flash LIDAR systems have been developed using a two dimensional array of Geiger mode APDs (pixels) to achieve more rapid scene capture than scanning systems, and to avoid problems with scanning systems such as mechanical wear, vibration, and/or motion blur. By rapidly resetting APDs after triggering, each pixel can be used to receive multiple photons from the laser pulse, each corresponding to different distance. In this way, a 3D point cloud of the target scene can be rapidly generated.

SPAD arrays in 3D Flash LIDAR systems have the potential to achieve even greater light sensitivity with high distance resolution and high frame rates than scanning laser systems or scanner-less APD based systems. In a frame based SPAD chip, each pixel or cell contains 1 or more SPADs and corresponding digital circuitry to time photon events (known as Time of Flight or ToF Mode) or to count photon events (known as photon counting mode. In photon counting mode, counters integrated to each SPAD cell keep a count of the number of arriving photons at each cell and effectively provide a measure of illumination, and the illuminator/laser can be omitted in which case the camera operates as a low light imager (images are intensity only 2D images). In DTOF timing mode, the integrated counters are triggered to start counting by the laser pulse and stop counting at the detection of the first photon. ToF techniques can be grouped into direct ToF or (dToF) and indirect (iToF) categories. Direct ToF methods directly measure the time delay by means of a very accurate timer or a Time-to-Digital (TDC) converter situated in the pixel. Depending on the accuracy or resolution of the timer, this method is typically used for long (kilometres) distance measurements and at very high precision (millimetre) depth resolutions. The Indirect ToF method in contrast reconstructs the time delay (hence distance) from the measurement of the phase delay of the reflected signal when compared to the periodic emitted light signal. This technique is more suited to short or medium distances (tens of meters) and with depth resolutions of some centimetres. For the iToF technique, either a continuous-wave iToF (cw-iToF) can be implemented, whereby a sinusoid modulated light source illuminates the scene and the returned signal is sampled a few times during the modulation period in order to compute the phase delay, or a pulsed-light iToF (p-iToF) method where the illuminator uses square pulses of light. This work will focus on using SPAD technology to implement specifically dToF methods for LADAR imaging.

The data generated by DToF SPAD arrays is a three dimensional time surface corresponding to the relative distance of the visual scene to the camera and illuminator. In DTOF mode a high speed precision counter is used for each SPAD cell to encode the time of flight of the arriving photons. Each frame comprises firing a laser pulse and synchronising the counters for each of the SPADs to the time of firing of the laser pulse. When a SPAD is triggered (i.e. due to an arriving photon), the counter is latched. At the end of the frame (e.g. max counter value) the counter values of SPAD cells must be read off chip for processing (i.e. to build up a 3D spatial picture of the target scene). As the photons are travelling at the speed of light, a single frame can be taken in the space of a few hundred nanoseconds after each laser pulse. This data transfer process creates an information bottleneck as significant bandwidth is required to transfer data from each of the SPAD cells, which is currently a major limiting factor in the speed of operation of such cameras. Further this bottleneck grows with increasing SPAD array size.

There is thus a need to provide an improved SPAD array system for 3D imaging applications at least provide a useful alternative to current DToF SPAD array systems.

SUMMARY

According to a first aspect, there is provided a method for operation of a Photon Avalanche Diode (SPAD) array microchip, the method comprising:

detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval, wherein the timing interval comprises one or more frames; and

reading out an address of each triggered SPAD sensor or an identifier that identifies the location of a group of triggered SPAD sensors in the order of triggering during the timing interval.

In one form, detecting the triggering of one or more SPAD sensors comprises:

detecting, on board the microchip, the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame,

and the step of reading out the address comprises:

reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors, or the addresses of the SPAD sensors in the group of SPAD sensors, only if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.

In a further form:

the plurality of SPAD sensors are each associated with one or more or a plurality of receptive field groups where each receptive field groups comprises a receptive group of SPAD sensors, and different receptive field groups may share some but not all of the same SPAD sensors; and

the plurality of predefined triggering patterns comprises a plurality of neural feature subgroups where each receptive field comprises a two or more neural feature subgroups where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors; and

detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns comprises:

detecting the triggering of a neural feature subgroup of a receptive field, and wherein upon detection the triggering of each other neural feature subgroup in the same receptive field group is suppressed until the end of the current frame;

generating an event trigger for the receptive field group if the triggered neural feature subgroup is the first neural feature subgroup in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature subgroup which previously triggered in the timing interval for this receptive field;

and the step of reading out an identifier is performed upon generation of an event trigger, and comprises reading an address of the neural feature subgroup that triggered.

In a further form, generating an event trigger for the receptive field group further comprises

incrementing a counter if the neural feature subgroup that triggered is the same neural feature subgroup that generated the previous trigger or the counter is zero, otherwise decrementing the counter;

generating an event trigger for the receptive field group only when the counter reaches a predefined counter threshold value wherein the predefined counter threshold value is fixed for at least the timing interval. In a further form, the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.

In a further form, each receptive field group has the same geometry each of the neural feature subgroups in a receptive field group comprises the same number of SPAD sensors. In a further form, the plurality of SPAD sensors are arranged in a R×C grid, and each receptive field group is an r×r array of SPAD sensors, and the plurality of receptive field groups overlap, and the plurality of overlapping receptive field groups tile the R×C grid. In a further form, each neural feature group comprises a contiguous block of the r×r array of SPAD sensors in the receptive field group. In a further form, r=4, and each neural feature group comprises a block of 8 adjacent SPAD sensors arranged in a 2×4 or 4×2 grid.

In one form after reading out an identifier the plurality of SPADs are reset and the current frame ends, an in use the start of each frame is synchronised to a laser pulse.

According to a second aspect, there is provided a Single-Photon Avalanche Diode (SPAD) array microchip comprising:

a plurality of SPAD sensors;

a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames.

In one form the triggering circuit comprises:

a plurality of feature detection circuits, each feature detection circuit connected to a group of SPAD sensors and configured to generate a feature trigger when the connected SPADs satisfy a triggering criteria;

an event detection circuit configured to detect the first time a feature trigger is generated by a feature detection circuit during a timing interval wherein the timing interval comprises one or more frames and to read out the address of either the feature detection circuit or the connected SPAD sensors off the microchip.

In a further form, the feature detection circuits are neural feature detection circuits and the triggering circuit further comprises:

a plurality of receptive field circuits wherein each receptive field circuit comprises:

a receptive group of SPAD sensors; two or more neural feature circuits, wherein each neural feature circuit comprises a subgroup of SPAD sensors in the receptive group of SPAD sensors where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors, and each neural feature circuit is configured to generate a neural feature trigger signal and an inhibit signal if all the SPAD sensors in the subgroup trigger, wherein the inhibit signal prevents triggering of the other neural feature circuits in the same receptive field circuit during the current frame; and

the event detection circuit comprises an event detection circuit for each receptive field circuit which is configured to monitor the neural feature circuits for generation of a neural feature trigger signal, and generates an event trigger for the receptive field circuit if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval; and

the triggering circuit further comprises a read out circuit which detects an event trigger from a receptive field circuit, and on detection of an event trigger it reads an address of the neural feature circuit that triggered generation of the event trigger.

In a further form, each receptive field circuit further comprises a memory, and when a neural feature trigger signal is generated an identifier of the neural feature circuit that generated the neural feature trigger signal in stored in the memory, and the event detection circuit is configured to determine if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval by comparing an identifier of the neural feature circuit that generated the neural feature trigger with the identifier stored in the memory.

In a further form, each receptive field circuit further comprises a counter wherein the counter is incremented the ne

This application is the United States national phase of International Application No. PCT/AU2018/000187 filed Sep. 27, 2018, and claims priority to Australian Provisional Patent Application No. 2017903926 filed Sep. 27, 2017, the disclosures of which are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to sensor circuits comprising an array of Single-Photon Avalanche Diode (SPAD) circuits for three dimensional (3D) camera systems.

BACKGROUND

Avalanche photodiodes (APDs) are solid-state photodetectors in which a high bias voltage is applied to a p-n junction to provide a high first stage gain due to avalanche multiplication. Avalanche multiplication occurs when an incident photon with sufficient energy to liberate an electron arrives at the photodiode. The high electric field rapidly accelerates the photo-generated electron towards the anode, but before it can reach the anode, it collides with the intervening doped material releasing further electrons, all of which are then accelerated towards the anode. This process repeats leading to avalanche multiplication of the photo-generated electron and an output current pulse. APDs are thus semiconductor analogs to photomultiplier tubes. Linear mode APDs are effectively single stage linear amplifiers in which the gain is set by controlling the bias voltage, and with gain factors of several hundred can be achieved in linear mode.

Single-Photon Avalanche Diodes (SPADs) are APDs in which the p-n junction is biased above its breakdown voltage to operate in Geiger mode such that a single incident photon will trigger an ongoing avalanche breakdown and thus easily measureable current pulse. That is a SPAD operates as a trigger device generating a large current pulse compared to linear mode APDs in which the current pulses can be very low at low light intensity. After triggering of the SPAD, a quenching circuit is used to reduce the bias voltage below the breakdown voltage in order to quench the avalanche process. Once quenched the bias voltage is again raised above the breakdown voltage to reset the SPAD for detection of another photon (known as re-biasing the SPAD).

APD and SPADs are solid state devices and can be constructed using a variety of CMOS technologies, and have very small active areas compared with other photon counting devices such as photomultiplier tubes. Through appropriate choice of materials and structure, wavelength sensitivity of a SPAD can be controlled to be in the visible and/or near-infrared range. A SPAD combined with additional circuitry to count pulses and/or measure time of arrival of photons to sub-nano/pico second accuracy can be used to create sensors for ultra-low light imaging or highly sensitive time-resolved imaging applications. For example one potential application of a SPAD arrays are in three dimensional (3D) Flash LIDAR cameras, as they have the potential to provide extremely sensitive devices with high distance resolution and high frame rates. SPAD arrays also have potential for use in other applications that require single photo sensitivity with high frames such as biological/medical imaging applications, adaptive optics applications, and astrophysics applications.

Three dimensional Flash LIDAR systems, also known as 3D Time of Flight (TOF) Cameras, use a laser source to irradiate a target with a short duration laser pulse (i.e. a laser flash). Photons are back scattered off objects and onto the sensor and the time of arrival is used to determine time of flight and thus distance to an object. The first 3D TOF cameras were constructed as scanning laser systems comprising a laser range finder with a rotating or scanning element(s) to progressively scan the field of view. These scanning systems are effectively single pixel devices collecting time of flight information in a single direction which build up a 3D image progressively moving the pointing direction of the sensing element. More recently scanner-less (i.e. staring) 3D Flash LIDAR systems have been developed using a two dimensional array of Geiger mode APDs (pixels) to achieve more rapid scene capture than scanning systems, and to avoid problems with scanning systems such as mechanical wear, vibration, and/or motion blur. By rapidly resetting APDs after triggering, each pixel can be used to receive multiple photons from the laser pulse, each corresponding to different distance. In this way, a 3D point cloud of the target scene can be rapidly generated.

SPAD arrays in 3D Flash LIDAR systems have the potential to achieve even greater light sensitivity with high distance resolution and high frame rates than scanning laser systems or scanner-less APD based systems. In a frame based SPAD chip, each pixel or cell contains 1 or more SPADs and corresponding digital circuitry to time photon events (known as Time of Flight or ToF Mode) or to count photon events (known as photon counting mode. In photon counting mode, counters integrated to each SPAD cell keep a count of the number of arriving photons at each cell and effectively provide a measure of illumination, and the illuminator/laser can be omitted in which case the camera operates as a low light imager (images are intensity only 2D images). In DTOF timing mode, the integrated counters are triggered to start counting by the laser pulse and stop counting at the detection of the first photon. ToF techniques can be grouped into direct ToF or (dToF) and indirect (iToF) categories. Direct ToF methods directly measure the time delay by means of a very accurate timer or a Time-to-Digital (TDC) converter situated in the pixel. Depending on the accuracy or resolution of the timer, this method is typically used for long (kilometres) distance measurements and at very high precision (millimetre) depth resolutions. The Indirect ToF method in contrast reconstructs the time delay (hence distance) from the measurement of the phase delay of the reflected signal when compared to the periodic emitted light signal. This technique is more suited to short or medium distances (tens of meters) and with depth resolutions of some centimetres. For the iToF technique, either a continuous-wave iToF (cw-iToF) can be implemented, whereby a sinusoid modulated light source illuminates the scene and the returned signal is sampled a few times during the modulation period in order to compute the phase delay, or a pulsed-light iToF (p-iToF) method where the illuminator uses square pulses of light. This work will focus on using SPAD technology to implement specifically dToF methods for LADAR imaging.

The data generated by DToF SPAD arrays is a three dimensional time surface corresponding to the relative distance of the visual scene to the camera and illuminator. In DTOF mode a high speed precision counter is used for each SPAD cell to encode the time of flight of the arriving photons. Each frame comprises firing a laser pulse and synchronising the counters for each of the SPADs to the time of firing of the laser pulse. When a SPAD is triggered (i.e. due to an arriving photon), the counter is latched. At the end of the frame (e.g. max counter value) the counter values of SPAD cells must be read off chip for processing (i.e. to build up a 3D spatial picture of the target scene). As the photons are travelling at the speed of light, a single frame can be taken in the space of a few hundred nanoseconds after each laser pulse. This data transfer process creates an information bottleneck as significant bandwidth is required to transfer data from each of the SPAD cells, which is currently a major limiting factor in the speed of operation of such cameras. Further this bottleneck grows with increasing SPAD array size.

There is thus a need to provide an improved SPAD array system for 3D imaging applications at least provide a useful alternative to current DToF SPAD array systems.

SUMMARY

According to a first aspect, there is provided a method for operation of a Photon Avalanche Diode (SPAD) array microchip, the method comprising:

detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval, wherein the timing interval comprises one or more frames; and

reading out an address of each triggered SPAD sensor or an identifier that identifies the location of a group of triggered SPAD sensors in the order of triggering during the timing interval.

In one form, detecting the triggering of one or more SPAD sensors comprises:

detecting, on board the microchip, the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame,

and the step of reading out the address comprises:

reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors, or the addresses of the SPAD sensors in the group of SPAD sensors, only if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.

In a further form:

the plurality of SPAD sensors are each associated with one or more or a plurality of receptive field groups where each receptive field groups comprises a receptive group of SPAD sensors, and different receptive field groups may share some but not all of the same SPAD sensors; and

the plurality of predefined triggering patterns comprises a plurality of neural feature subgroups where each receptive field comprises a two or more neural feature subgroups where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors; and

detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns comprises:

detecting the triggering of a neural feature subgroup of a receptive field, and wherein upon detection the triggering of each other neural feature subgroup in the same receptive field group is suppressed until the end of the current frame;

generating an event trigger for the receptive field group if the triggered neural feature subgroup is the first neural feature subgroup in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature subgroup which previously triggered in the timing interval for this receptive field;

and the step of reading out an identifier is performed upon generation of an event trigger, and comprises reading an address of the neural feature subgroup that triggered.

In a further form, generating an event trigger for the receptive field group further comprises

incrementing a counter if the neural feature subgroup that triggered is the same neural feature subgroup that generated the previous trigger or the counter is zero, otherwise decrementing the counter;

generating an event trigger for the receptive field group only when the counter reaches a predefined counter threshold value wherein the predefined counter threshold value is fixed for at least the timing interval. In a further form, the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.

In a further form, each receptive field group has the same geometry each of the neural feature subgroups in a receptive field group comprises the same number of SPAD sensors. In a further form, the plurality of SPAD sensors are arranged in a R×C grid, and each receptive field group is an r×r array of SPAD sensors, and the plurality of receptive field groups overlap, and the plurality of overlapping receptive field groups tile the R×C grid. In a further form, each neural feature group comprises a contiguous block of the r×r array of SPAD sensors in the receptive field group. In a further form, r=4, and each neural feature group comprises a block of 8 adjacent SPAD sensors arranged in a 2×4 or 4×2 grid.

In one form after reading out an identifier the plurality of SPADs are reset and the current frame ends, an in use the start of each frame is synchronised to a laser pulse.

According to a second aspect, there is provided a Single-Photon Avalanche Diode (SPAD) array microchip comprising:

a plurality of SPAD sensors;

a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames.

In one form the triggering circuit comprises:

a plurality of feature detection circuits, each feature detection circuit connected to a group of SPAD sensors and configured to generate a feature trigger when the connected SPADs satisfy a triggering criteria;

an event detection circuit configured to detect the first time a feature trigger is generated by a feature detection circuit during a timing interval wherein the timing interval comprises one or more frames and to read out the address of either the feature detection circuit or the connected SPAD sensors off the microchip.

In a further form, the feature detection circuits are neural feature detection circuits and the triggering circuit further comprises:

a plurality of receptive field circuits wherein each receptive field circuit comprises:

a receptive group of SPAD sensors; two or more neural feature circuits, wherein each neural feature circuit comprises a subgroup of SPAD sensors in the receptive group of SPAD sensors where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors, and each neural feature circuit is configured to generate a neural feature trigger signal and an inhibit signal if all the SPAD sensors in the subgroup trigger, wherein the inhibit signal prevents triggering of the other neural feature circuits in the same receptive field circuit during the current frame; and

the event detection circuit comprises an event detection circuit for each receptive field circuit which is configured to monitor the neural feature circuits for generation of a neural feature trigger signal, and generates an event trigger for the receptive field circuit if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval; and

the triggering circuit further comprises a read out circuit which detects an event trigger from a receptive field circuit, and on detection of an event trigger it reads an address of the neural feature circuit that triggered generation of the event trigger.

In a further form, each receptive field circuit further comprises a memory, and when a neural feature trigger signal is generated an identifier of the neural feature circuit that generated the neural feature trigger signal in stored in the memory, and the event detection circuit is configured to determine if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval by comparing an identifier of the neural feature circuit that generated the neural feature trigger with the identifier stored in the memory.

In a further form, each receptive field circuit further comprises a counter wherein the counter is incremented the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the identifier of the neural feature circuit that generated the neural feature trigger is the same as the identifier stored in the memory, and an event trigger is generated when the counter is equal to a predetermined threshold value fixed for at least the timing interval. In a further form, the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.

In a further form, each of the neural feature circuits in a receptive field comprises the same number of SPAD sensors. In a further form, the plurality of SPAD sensors are arranged in a R×C grid, each receptive field circuits is an r×r array of SPAD sensors, and the plurality of receptive field circuits overlap, and the plurality of overlapping receptive field circuit tile the R×C grid. In a further form, each neural feature circuit comprises a contiguous block of the r×r array of SPAD sensors in the receptive field circuit. In a further form, r=4, and each neural feature circuit comprises a block of 8 adjacent SPAD sensors arranged in a 2×4 or 4×2 grid.

In one form, each neural feature circuit comprises a logical circuit which ANDs each of the subgroup of SPAD sensors.

In one form, the predetermined threshold value is adjustable between timing intervals.

In one form, the chip further comprises a data bus and an Address Event Representation protocol is used to read data off the microchip

In one form, after reading out the address the plurality of SPAD sensors are reset and the current frame ends, an in use the start of each frame is synchronised to a laser pulse.

BRIEF DESCRIPTION OF DRAWINGS

Embodiments of the present disclosure will be discussed with reference to the accompanying drawings wherein:

FIG. 1A is a schematic diagram of a SPAD array sensor according to an embodiment;

FIG. 1B is a schematic diagram of adjacent SPAD cells according to an embodiment;

FIG. 1C is a flowchart of a method of operation of a SPAD array microchip according to an embodiment;

FIG. 2A is an illustrative example of a 5×5 SPAD array microchip with four 4×4 receptive fields according to an embodiment;

FIG. 2B is a schematic representation of a 4×4 receptive field illustrating the connection of the SPAD sensors to four AND gates to create four neural feature circuits according to an embodiment;

FIG. 2C is a representation of a 3D visual scene and the resultant SPAD timing pattern for the 5×5 SPAD array microchip of FIG. 2A according to an embodiment;

FIG. 3A is a schematic wiring diagram of the a 5×5 imager with four 4×4 receptive fields shown in FIG. 2A where each receptive field is connected to four AND gates a according to an embodiment;

FIG. 3B is a close up of the wiring diagram of the receptive field circuit and event generation circuit in FIG. 3A according to an embodiment;

FIG. 4A is a detailed block diagram of a first part of the feature detection circuit for each receptive field according to an embodiment;

FIG. 4B is a detailed block diagram of a second part of the feature detection circuit for each receptive field according to an embodiment

FIG. 5A is an experimental setup with a SPAD field of view marked by black dotted line a according to an embodiment;

FIG. 5B shows 500 averaged frames showing a stationary B-747 model viewed using the experimental setup of FIG. 5A ;

FIG. 5C shows the result of averaging 5 frames showing an F-14 model plane as it drops at high speed close to the camera lens in the experimental setup of FIG. 5A ;

FIG. 5D is a representation of a raw SPAD input showing common sources of noise according to an embodiment;

FIG. 5E is a representation of SPAD activation across pixels and time according to an embodiment;

FIG. 6A is a histogram over frames 1 to 270 showing stray photons vs. missed detection when the airplane was not in front of the pixel where a cross indicates false positives;

FIG. 6B is a histogram over frames 271 to 481 showing stray photons vs. missed detection when the airplane was in front of the pixel where a cross indicates false negatives;

FIG. 7 shows examples of a persistent background pattern in maxed out counters where panels (a) (b) and (c) show in high contrast the timing patterns in consecutive frames for the pixels whose counters have maxed out and panel (d) highlights the persistence of the pattern by comparing a small section from each frame next to each other according to an embodiment;

FIG. 8 is a comparison between noisy pixels and normal pixels showing SPAD timings averaged over 3600 frames according to an embodiment;

FIG. 9 is a representation showing the number of receptive fields impaired by a single photon at different receptive field sizes where X marks the dead pixel, the black grid indicated the pixel region affected by the dead pixel and the coloured squares indicate receptive fields affected by the dead pixel according to an embodiment; and

FIG. 10 is a schematic diagram of a Flash LIDAR system using a SPAD array according to an embodiment.

In the following description, like reference characters designate like or corresponding parts throughout the figures.

DESCRIPTION OF EMBODIMENTS

Embodiments of a single photon avalanche detector (SPAD) array microchip, and methods of operation are described. With conventional (Frame based) SPAD microchip designs as used in 3D ToF cameras, each pixel or cell contains 1 or more SPADs and corresponding digital circuitry to time when photon events occur. For every pixel the ToF data is collectively communicated off chip synchronous with a laser pulse. With more pixels (higher resolution arrays) come the problems of how to reduce circuit complexity without compromising functionality and how to move large amounts of ToF data off chip. Embodiments of a single photon avalanche detector (SPAD) array microchip, and methods of operation are described in which the inter pixel photon arrival order (not time) is recorded and transmitted off chip.

Referring now to FIG. 1A , there is shown a schematic diagram of a SPAD array sensor 1 according to an embodiment. The SPAD array sensor comprises a SPAD array 2 and a SPAD Array control circuit 3 . The SPAD array 2 comprises an array of SPAD cells 110 , arranged in n rows and m columns (i.e. a n×m array). This is illustrated in FIG. 1 , in which the first row comprises a first SPAD cell 10 in the first column, a second SPAD cell 111 in the second column, etc. to the mth SPAD cell 112 in the mth column. This structure is repeated for n rows, and thus the nth row comprises the ((n−1)×m)+1 th SPAD cell 114 in the first column of the nth row to the n×m th SPAD cell 116 in the mth column of the nth row. The SPAD array control circuit performs a range of functions such as providing power to the array, and controlling trigger enablement and resetting of SPAD sensors or SPAD cells, and reading of data from cells in the array.

FIG. 1B is a schematic diagram of adjacent SPAD cells

110 and 111 according to an embodiment. In this embodiment the first SPAD cell 110 comprises a SPAD structure 120 , and a quenching circuit 130 . The second SPAD cell 111 is of identical construction to the first SPAD cell, and comprises a second SPAD structure 121 and a second quenching circuit 131 . As will be discussed below a triggering circuit detects triggering of the SPAD sensors. As will be discussed below, in some embodiments the triggering circuit comprises neural feature circuits. In one embodiment the quenching circuits

130 , 131 are analog circuits and the triggering circuit is a digital circuit.

The SPAD array microchip may be constructed using a range of CMOS technologies, for example from 0.8 μm down to 65 nm (and potentially lower) incorporating multiple metal layers and voltages, and may be designed using VLSI and related IC chip design techniques. In one embodiment the SPAD array sensor is constructed as a monolithic CMOS wafer comprised of the SPAD array and a SPAD control circuit. In other embodiments, multi-wafer layering techniques can be used, in which the top wafer layer is a comprised of an array of SPAD structures, and the lower wafer layer(s) comprise associated SPAD structure quenching circuit and triggering circuits connected to SPAD structures in the top wafer layer, and the SPAD array control circuit.

The SPAD sensor comprises a SPAD structure which comprises an active region, within which an incident photon will generate an output current pulse, which may also be referred to as a trigger pulse as well as anode and cathode connections for application of the bias voltage and detection of the trigger pulse. Various SPAD structures may be used with different SPAD structures having different efficiencies and characteristics.

A quenching circuit is used to control the bias voltage applied to the SPAD structure and quench (i.e. stop) the avalanche process by reducing the bias voltage below the breakdown voltage. The quenching circuit may also be referred to as a quenching and recharge circuit, as once the avalanche process is quenched, the bias voltage is again raised to the operational voltage above the breakdown voltage to allow the SPAD to be triggered again. Quenching may be performed using active or passive quenching. Passive quenching is performed by providing a resistance in series to the SPAD. The avalanche current self-quenches simply because it develops a voltage drop across the resistance (a high-value ballast load), reducing the voltage across the SPAD to below its breakdown voltage. After the quenching of the avalanche current, the SPAD's bias slowly recovers to at or above the breakdown voltage. An alternative to passive quenching is active quenching. An active quenching circuit detects avalanche breakdown by a digital logic circuit or component, for example a transistor acting as a resistive element or a transistor based circuit connected to the SPAD output. Upon detection of a trigger pulse, the active quenching circuit actively pulls the bias voltage to a voltage below breakdown to quench the avalanche, before switching the voltage back to the operational voltage. Compared to passive quenching, active quenching circuits occupies less space on the chip and have reduced dead time, as well as providing options such as time gating of the SPAD to synchronise with firing of a laser pulse. In one embodiment the quenching circuits are analog circuits and the triggering circuits are digital circuits.

FIG. 1C is a flowchart of a method of operation of a SPAD array microchip 100 according to an embodiment. In contrast to traditional SPAD DTOF systems, the inter pixel photon arrival order (not time) is recorded and transmitted off chip using this method. The method broadly comprises step 102 of detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval, wherein the timing interval comprises one or more frames. In some embodiments, only one event is read per frame (where the start of the frame is defined by the laser firing), and in other embodiments, multiple events can be read per frame, each from different receptive fields (described below). In one embodiment the detection step comprises detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame. At step 104 , the method continues with reading out an address of each triggered SPAD sensor or an identifier that identifies the location of a group of triggered SPAD sensors in the order of triggering during the timing interval. In some embodiment this comprises reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors (i.e. indirectly reading out the addresses of the triggered SPAD sensors), or the addresses of the SPAD sensors in the group of SPAD sensors. This is only performed if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.

In one embodiment the design is a “synchronous” logic circuit with an external clock that drives all the internal logic circuits. SPADs avalanching are synchronised to this clock (although in other embodiments this restriction can be relaxed) so that “events” triggered by the receptive fields occur synchronous to this clock. Exactly when these events actually occur is unknown, and thus the system can be described as asynchronous in nature between frames (the frame itself is synchronised to the start of the laser pulse). The time when the event occurs is not time stamped (or recorded). However in one embodiment a global counter could be used to time stamp events: In this embodiment the chip operates in timing mode, and is synchronized to the firing of a laser pulse where each laser pulse presents the start of a new frame. A counter is latched at the time the event is read off the chip and based on the clock frequency (period of each clock tick) a time can be determined which can be converted to a distance by multiplying by the speed of light. This allows the use of a single counter for the chip, with this counter synchronised to the laser pulse (i.e. start of the frame). This counter could be located on the chip or off the chip as part of a control circuit 3 associated with reading events off the chip and resetting the chip. In one embodiment the control circuit comprises a processor such as a FPGA which reads the addresses of events off the chip and which comprises a counter which is read each time an event is read off the chip. A timing interval can be defined which is a plurality of frames. Over successive frames new events corresponding to newly detected features are read out and the time latched. In some embodiments only one event is read out per frame, and over a consecutive series of frames (the timing interval) a 3D representation of the scene can be created. In other embodiments, multiple events per frame may be read out and the times latched in which case several counters may be required, or the counter value must be latched and stored with the event without stopping the counter.

This method and embodiments of a microchip and associated circuits for implementing this method will now be discussed in greater detail to more clearly highlight the various features of the method and implementation.

To assist in understanding the invention we will first consider an embodiment where the SPAD array microchip is a regular array with SPAD sensors arranged in an R×C grid. In this event based triggering approach we introduce the concept of a receptive field. Each receptive field 210 is group of SPAD sensors—equivalently a macro cell of neighbouring SPADs. Preferably each receptive field is a r×r array of SPAD sensors or a contiguous block of SPAD sensors. The receptive fields are tiled across the array in an overlapping pattern so that adjacent receptive fields share some but not all of the same SPAD sensors. For example in a 128×128 SPAD array chip, with receptive fields with a 4×4 shape, the entire chip contains 125×125 receptive field block and each receptive field shares overlapping SPADs with adjacent receptive fields.

FIG. 2A is an illustrative example of a 5×5 SPAD array microchip 200 with four 4×4

receptive fields

212 , 214 , 216 , and 218 . That is each receptive field connects 16 neighbouring SPADs in a 4×4 arrangement. Each receptive field is assigned an address based on the (row, column) location of the top corner. For example in FIG. 2A , the first receptive field 212 is indicated with lightly shaded boxes (coloured yellow in the provisional) and has an address of rf(1, 1), the second receptive field 214 is shifted one column to the right and has slightly heavier shading (coloured green in the provisional) has an address of rf(1, 2). The third receptive field 216 is shifted one row down and has medium shading (coloured red in the provisional) and has an address of rf(2, 1), and the fourth receptive field 218 is shifted one column right and one column down and is more darkly shaded (coloured blue in the provisional) and has an address of rf(2, 2). The overlapping receptive fields are shown on 5×5 SPAD array microchip 200 . The interior 3×3 cells have the darkest shading, each corner has the shading of their respective source ( 212 , 214 , 216 , 218 ) and the 3 border have intermediate shading based on the two bordering end cells.

Each receptive field comprises its own circuitry that digitally processes its own group of SPAD cells to identify the presence of one or more features with a predefined triggering pattern. These will be referred to as neural feature subgroups or neural feature circuits. Each neural feature circuit comprises a subgroup of the SPAD sensors in the receptive field, and the neural feature circuit is configured to detect a triggering pattern within this subgroup. The neural feature groups are thus feature shapes, and are analogous to silicon neurons. Each receptive circuit comprises two or more neural feature subgroups or circuits and where each neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors. In the present example 4 features are used, but it could be higher or lower as required.

In most embodiments, the neural feature circuits are created using an AND gate which takes as input a local group of SPAD sensors and which outputs a neural feature trigger signal when all the input SPADs are triggered (i.e. the AND gate is use to combine all of the SPADs in the subgroup so the feature is only triggered when all SPADs in the feature have triggered). However other more complex digital circuits can be used to capture more complex triggering patterns, including non-triggering of some SPADs in the neural feature group. In this case a logic circuit generates a neural feature trigger signal when the predetermined trigger pattern of SPAD is satisfied.

Preferably the number of input SPAD sensors per AND gate is equal for each neural feature circuit so as to result in an equal probability of AND gate activation. That is each of the neural features in a receptive field comprises the same number of SPAD sensors. However the pattern of connectivity and its correspondence to the observed data also determines AND gate activation. For example an edge bar is more likely to be activated in a natural environment in comparison to a checker board pattern since the latter is not typically observed in the environment. In this way each AND gate encodes a local feature and it's activation indicates that all its input signals have been activated. Thus in most embodiments each of the neural feature circuits in a receptive field are contiguous block with the same shape, for example a rectangular 2×4 or 4×2 group of SPADs. In other embodiments the neural features could comprise solid squares, lines, rectangles, L shaped blocks, regular polygons (hexagon, octagon), crosses, etc. In the example shown in FIG. 2A each receptive field comprises 16 SPAD sensors, with four neural features.

FIG. 2B is a schematic representation of a 4×4 receptive field 210 illustrating the connection of the 16 SPAD sensors to four AND

gates

221 , 223 , 225 , and 227 to create four

neural feature circuits

222 , 224 , 226 , and 228 . In this embodiment each neural feature group is a block of 8 adjacent SPAD sensors arranged in a 2×4 or 4×2 grid (i.e. edge bar patterns). Each neural feature circuit has an associated address for example based on the (row, column) location of the receptive field, and an identifier of the neural feature circuit. For example in this embodiment the

circuits

222 , 224 , 226 , and 228 are in north, east, south and west locations and are assigned

identifiers

1, 2, 3, 4. A unique address for the SPAD arrays in a neural feature circuit can thus be provided by the (row, column) address of the receptive field, and the identifier (n=1 . . . 4) of the neural circuit within the receptive field. In other embodiments the address of the neural feature could be specified by the (row, column) of a first or reference cell in the neural feature shape (e.g. top left corner of the shape). In one embodiment, the unique address of the receptive field is known and only the “winning” neuron feature (or feature event) within that receptive field is known (or detected/stored). One neural circuit is provided for every receptive field and is configured to record/detect/store the neuron feature that triggered an event. In this embodiment the address of “every neural feature” in a receptive field is not known or recorded.

FIG. 2C is a representation of a 3D visual scene and the resultant SPAD timing pattern for the 5×5 SPAD array microchip of FIG. 2A according to an embodiment. In this embodiment a box 230 is in the foreground and returns a signal 232 over a first time t 1 =4 ns. The background 240 is far away (i.e. no objects) at the maximum range (time t 2 =8 ns). The box occupies pixels (row=3:5 and col=2:5), then the lower/south AND of rf(1,2) (green) and rf(2,2) (blue) will latch which can be expressed as AND(1,2,3) and AND(2,2,3) latching at t=4 ns. The resultant image of time 250 on the 5×5 SPAD imager is colour coded according to the photon arrival time or equivalently the SPAD latch time (nsec) for each detected event. In this figure the image comprises a 3×4 medium shaded (orange in the provisional application) block of pixels corresponding to a time of 4 ns, and the remaining pixels are dark shaded (red in the provisional) at time t 2 =8 ns.

Another feature of the receptive field circuit, is that as a neural feature trigger signal is generated, an inhibit signal is also generated which suppresses (i.e. prevents) triggering of the other neural feature circuits in the same receptive field circuit during the current frame.

That is the going high or latching of the first AND gate (first neural feature circuit) prevents subsequent latching of any later neurons (other neural feature circuit) via a recurrent enable connection that gates all the ANDs. Thus, at the beginning of the pulse cycle when the laser pulse is sent, the enable to all ANDS for all the receptive fields are high allowing any AND gate to latch but then for each receptive field as soon as the first AND gate latches the enable to the other ANDs of this receptive field is set low. Note that the enable flag going low does not affect the first AND gate that caused the lowering in the first place. This is achieved by the AND gate blocking its own blocking signal. This is realised through the following logic: an AND gate can only be high if and only if all its input pixels are high and (the local receptive fields enable flag is high or the AND was already high (In the previous time step in a synchronous system or an asynchronous memory in an asynchronous implementation)).

This is illustrated in FIG. 3A is a schematic wiring diagram of the 5×5 imager with four 4×4 receptive fields shown in FIG. 2A where each receptive field is connected to four AND gates a according to an embodiment. FIG. 3B is a close up of the wiring diagram of the receptive field circuit and event generation circuit in FIG. 3A according to an embodiment. A first receptive field 320 rf(1,1) comprises a neural feature circuit 320 with a 2×4 geometry of highlighted SPADS (in red), labelled 301 , 302 , 303 , and 304 (row 3); and 311 , 312 , 313 , and 314 (row 4). Column 5 SPAD 304 in row 3 and SPAD 315 in row 5 are not part of this receptive field. The wires carry binary on/off values from the SPAD sensors (trigger output) to four AND gates labelled AND_south, AND_north, AND east and AND_west as shown in FIG. 3B . The AND_south gate receives inputs from neural feature circuit 320 (

SPADs

301 , 302 , 303 , 304 , 311 , 312 , 313 , and 314 ). The output of AND_south is provides as input In1 to event circuit 340 . Similarly the three other AND gates for the three other neural feature circuits are provided to inputs In2 In3 and In4 of the event circuit 340 . Note that if only the highlighted SPADS (in red, labelled 301 , 302 , 303 , 304 , 311 , 312 , 313 , and 314 ) latch at t=t 1 then the AND_south (or AND(1,1,3)) will latch at t=t 1 .

The event circuit 340 is configured to generate an event trigger for the receptive field if one of the input neural feature subgroup triggers. As will be discussed below, the trigger is only generated if the neural feature circuit which triggered is the first in the receptive field to trigger in the timing interval or if the triggered neural feature circuit is different to the neural feature circuit which previously triggered in the timing interval for this receptive field.

If a trigger (EventFlag) is generated by the event trigger then the address of the neural feature is read out (e.g. using neuronAddress0 for the row, and neuronAddress2 for the column) on a data bus. This may be the (row, column) address of the receptive field, e.g. rc(1, 1) and the identifier, e.g. In1, is read out. Alternatively the addresses of the SPADs could be read out.

FIGS. 4A and 4B are detailed block diagram of a first and second part of the <figure-callout id="340" label="feature detection circuit" filenames="US11474215-20221018-D00005.png,US11474215-20221018-D00006.png" state="{{stat

CLAIMS

Claims ( 22 )

The invention claimed is:

1. A method for operation of a Single Photon Avalanche Diode (SPAD) array microchip, the method comprising:

detecting, on board a SPAD array microchip comprising a plurality of SPAD sensors, the triggering of one or more SPAD sensors during a timing interval comprising detecting, on board the SPAD array microchip, the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns during a frame, wherein the timing interval comprises one or more frames wherein a start of each frame is defined by a synchronisation signal and the triggering time of each SPAD sensor is not time stamped or recorded; and

reading out an address of each triggered SPAD sensor, or an identifier that identifies the location of a group of triggered SPAD sensors, in the order of triggering during the timing interval comprising reading out an identifier of the triggering pattern which identifies the location of the group of SPAD sensors, or the addresses of the SPAD sensors in the group of SPAD sensors, only if the group of SPAD sensors matching the predefined pattern has not previously been detected during the timing interval.

2. The method as claimed in claim 1 , wherein:

the plurality of SPAD sensors are each associated with one or more or a plurality of receptive field groups where each receptive field groups comprises a receptive group of SPAD sensors, and different receptive field groups may share some but not all of the same SPAD sensors; and

the plurality of predefined triggering patterns comprises a plurality of neural feature subgroups where each receptive field comprises two or more neural feature subgroups where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors; and

detecting the triggering of a group of SPAD sensors matching one of a plurality of predefined triggering patterns comprises:

detecting the triggering of a neural feature subgroup of a receptive field, and wherein upon detection the triggering of each other neural feature subgroup in the same receptive field group is suppressed until the end of the current frame;

generating an event trigger for the receptive field group if the triggered neural feature subgroup is the first neural feature subgroup in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature subgroup which previously triggered in the timing interval for this receptive field;

and the step of reading out an identifier is performed upon generation of an event trigger, and comprises reading an address of the neural feature subgroup that triggered.

3. The method as claimed in claim 2 , wherein generating an event trigger for the receptive field group further comprises

incrementing a counter if the neural feature subgroup that triggered is the same neural feature subgroup that generated the previous trigger or the counter is zero;

generating an event trigger for the receptive field group only when the counter reaches a predefined counter threshold value wherein the predefined counter threshold value is fixed for at least the timing interval.

4. The method as claimed in claim 3 , wherein the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.

5. The method as claimed in claim 2 , wherein each receptive field group has the same geometry each of the neural feature subgroups in a receptive field group comprises the same number of SPAD sensors.

6. The method as claimed in claim 5 , wherein the plurality of SPAD sensors are arranged in a R×C grid, and each receptive field group is an r×r array of SPAD sensors, and the plurality of receptive field groups overlap, and the plurality of overlapping receptive field groups tile the R×C grid.

7. The method as claimed in claim 5 , wherein each neural feature group comprises a contiguous block of the r×r array of SPAD sensors in the receptive field group.

8. The method as claimed in claim 5 , wherein r=4, and each neural feature group comprises a block of 8 adjacent SPAD sensors arranged in a 2×4 or 4×2 grid.

9. The method as claimed in claim 2 , wherein after reading out an identifier the plurality of SPADs are reset and the current frame ends, an in use the synchronisation signal is synchronised to the firing of a laser pulse.

10. A Single-Photon Avalanche Diode (SPAD) array microchip comprising:

a plurality of SPAD sensors;

a triggering circuit configured to detect and read out the triggering order of SPAD sensors over a timing interval wherein the timing interval comprises one or more frames wherein a start of each frame is defined by a synchronisation signal and the triggering time of each SPAD sensor is not time stamped or recorded and the triggering circuit comprises:

a plurality of feature detection circuits, each feature detection circuit connected to a group of SPAD sensors and configured to generate a feature trigger when the connected SPADs satisfy a triggering criteria; and

an event detection circuit configured to detect the first time a feature trigger is generated by a feature detection circuit during a timing interval wherein the timing interval comprises one or more frames and to read out the address of either the feature detection circuit or the connected SPAD sensors off the microchip.

11. The SPAD array microchip as claimed in claim 10 , wherein the feature detection circuits are neural feature detection circuits and the triggering circuit further comprises:

a plurality of receptive field circuits wherein each receptive field circuit comprises:

a receptive group of SPAD sensors;

two or more neural feature circuits, wherein each neural feature circuit comprises a subgroup of SPAD sensors in the receptive group of SPAD sensors where the respective neural feature subgroups comprise different subgroups of the SPAD sensors from the receptive group of SPAD sensors, and each neural feature circuit is configured to generate a neural feature trigger signal and an inhibit signal if all the SPAD sensors in the subgroup trigger, wherein the inhibit signal prevents triggering of the other neural feature circuits in the same receptive field circuit during the current frame;

the event detection circuit comprises an event detection circuit for each receptive field circuit which is configured to monitor the neural feature circuits for generation of a neural feature trigger signal, and generates an event trigger for the receptive field circuit if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval; and

the triggering circuit further comprises a read out circuit which detects an event trigger from a receptive field circuit, and on detection of an event trigger it reads an address of the neural feature circuit that triggered generation of the event trigger.

12. The SPAD array microchip as claimed in claim 11 , each receptive field circuit further comprises a memory, and when a neural feature trigger signal is generated an identifier of the neural feature circuit that generated the neural feature trigger signal in stored in the memory, and the event detection circuit is configured to determine if the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the triggered neural feature subgroup is different to the neural feature circuit most previously triggered in the timing interval by comparing an identifier of the neural feature circuit that generated the neural feature trigger with the identifier stored in the memory.

13. The SPAD array microchip as claimed in claim 12 , wherein each receptive field circuit further comprises a counter wherein the counter is incremented the neural feature circuit which generated the neural feature trigger signal is the first neural feature circuit in the receptive field to trigger in the timing interval or if the identifier of the neural feature circuit that generated the neural feature trigger is the same as the identifier stored in the memory, and an event trigger is generated when the counter is equal to a predetermined threshold value fixed for at least the timing interval.

14. The SPAD array microchip as claimed in claim 13 , wherein the counter is decremented if the counter is greater than zero and the neural feature subgroup that triggered is not the same neural feature subgroup that generated the previous trigger.

15. The SPAD array microchip as claimed in claim 13 , wherein the predetermined threshold value is adjustable between timing intervals.

16. The SPAD array microchip as claimed in claim 12 , wherein each of the neural feature circuits in a receptive field comprises the same number of SPAD sensors.

17. The SPAD array microchip as claimed in claim 16 , wherein the plurality of SPAD sensors are arranged in a R×C grid, each receptive field circuits is an r×r array of SPAD sensors, and the plurality of receptive field circuits overlap, and the plurality of overlapping receptive field circuit tile the R×C grid.

18. The SPAD array microchip as claimed in claim 17 , wherein each neural feature circuit comprises a contiguous block of the r×r array of SPAD sensors in the receptive field circuit.

19. The SPAD array microchip as claimed in claim 18 , wherein r=4, and each neural feature circuit comprises a block of 8 adjacent SPAD sensors arranged in a 2×4 or 4×2 grid.

20. The SPAD array microchip as claimed in claim 12 , wherein each neural feature circuit comprises a logical circuit which ANDs each of the subgroup of SPAD sensors.

21. The SPAD array microchip as claimed in claim 10 wherein the chip further comprises a data bus and an Address Event Representation protocol is used to read data off the microchip.

22. The SPAD array microchip as claimed in claim 10 wherein after reading out the address the plurality of SPAD sensors are reset and the current frame ends, an in use the synchronisation signal is synchronised to the firing of a laser pulse.

US16/650,561

2017-09-27

2018-09-27

Neuromorphic single photon avalanche detector (SPAD) array microchip

Active

2039-08-07

US11474215B2

( en )

Applications Claiming Priority (3)

Application Number

Priority Date

Filing Date

Title

AU2017903926A

AU2017903926A0

( en )

2017-09-27

A neuromorphic single photon avalanche diode (spad) array microchip

AU2017903926

2017-09-27

PCT/AU2018/000187

WO2019060942A1

( en )

2017-09-27

2018-09-27

A neuromorphic single photon avalanche detector (spad) array microchip

Publications (2)

Publication Number

Publication Date

US20200326414A1

US20200326414A1 ( en )

2020-10-15

US11474215B2

true

US11474215B2 ( en )

2022-10-18

Family

ID=65900174

Family Applications (1)

Application Number

Title

Priority Date

Filing Date

US16/650,561

Active

2039-08-07

US11474215B2

( en )

2017-09-27

2018-09-27

Neuromorphic single photon avalanche detector (SPAD) array microchip

Country Status (4)

Country

Link

US

( 1 )

US11474215B2

( en )

EP

( 1 )

EP3688809B1

( en )

AU

( 1 )

AU2018341551B9

( en )

WO

( 1 )

WO2019060942A1

( en )

Families Citing this family (20)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

US11272112B2

( en )

*

2018-08-08

2022-03-08

Canon Kabushiki Kaisha

Image capturing apparatus

US11579263B2

( en )

*

2018-11-29

2023-02-14

Taiwan Semiconductor Manufacturing Co., Ltd.

Method and apparatus for a hybrid time-of-flight sensor with high dynamic range

US12212871B2

( en )

2020-05-29

2025-01-28

Voxelsensors Srl

Pixel sensor system

EP3989535B1

( en )

*

2020-10-20

2025-09-24

Canon Kabushiki Kaisha

Photoelectric conversion device, method of controlling photoelectric conversion device, and information processing apparatus

US11726187B2

( en )

*

2020-10-30

2023-08-15

Taiwan Semiconductor Manufacturing Co., Ltd.

High resolution low power inter-correlation SPAD assisted time-of-flight sensor

EP4256778B1

( en )

*

2020-12-23

2024-10-23

VoxelSensors SRL

Neighborhood-gated switching pixel sensor

BE1028950B9

( en )

*

2020-12-23

2023-10-09

Voxelsensors Srl

NEIGHBORHOOD SWITCHED PIXEL SENSOR

CN116648924A

( en )

*

2020-12-23

2023-08-25

体素传感器有限责任公司

Neighborhood Gated Switching Pixel Sensors

US11580713B2

( en )

*

2021-01-12

2023-02-14

Microsoft Technology Licensing, Llc

Motion compensation for a SPAD array camera

CN115128572B

( en )

*

2021-03-24

2025-07-08

深圳引望智能技术有限公司

Signal receiving device, detecting device, signal processing method and device thereof

BE1029775B9

( en )

*

2021-09-21

2023-10-09

Voxelsensors Srl

PERSISTENCE FILTERING AT SPD ARRAYS

JP7757107B2

( en )

*

2021-09-24

2025-10-21

キヤノン株式会社

Photoelectric conversion device, imaging device, control method, and computer program

CN114638349B

( en )

*

2022-01-27

2024-05-28

清华大学

Photosensitive neurons, sensing, storage and computing integrated intelligent sensing device, method and medium

CN116609769B

( en )

*

2022-02-09

2026-03-20

宁波飞芯电子科技有限公司

Pixel array

US12063040B2

( en )

2022-09-28

2024-08-13

Stmicroelectronics (Research &amp; Development) Limited

Reconfigurable high voltage generation circuit for SPAD sensors

US12684258B2

( en )

*

2023-02-21

2026-07-14

Canon Kabushiki Kaisha

Image capturing apparatus, image capturing system, and equipment

DE112024003185T5

( en )

2023-08-01

2026-07-02

Ams-Osram Ag

2D TDC arrangement

CN122477390A

( en )

*

2024-04-12

2026-07-28

图达通智能美国有限公司

SPAD side photodetector

US12470846B1

( en )

*

2024-05-10

2025-11-11

Rockwell Collins, Inc.

Noise reduction method for very low light level night vision

CN121691956B

( en )

*

2026-02-11

2026-05-12

西安电子科技大学杭州研究院

Light-weight reading circuit for large-array SPAD real-time data compression

Citations (4)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

WO2013018006A1

( en )

2011-08-03

2013-02-07

Koninklijke Philips Electronics N.V.

Position-sensitive readout modes for digital silicon photomultiplier arrays

US20150285625A1

( en )

2014-04-07

2015-10-08

Samsung Electronics Co., Ltd.

High resolution, high frame rate, low power image sensor

WO2017004663A1

( en )

2015-07-08

2017-01-12

The Commonwealth Of Australia

Spad array structures and methods of operation

US20170131143A1

( en )

*

2014-07-02

2017-05-11

Andreas G. Andreou

Photodetection circuit and operating method thereof

2018

2018-09-27

EP

EP18862953.9A

patent/EP3688809B1/en

active

Active

2018-09-27

WO

PCT/AU2018/000187

patent/WO2019060942A1/en

not_active

Ceased

2018-09-27

US

US16/650,561

patent/US11474215B2/en

active

Active

2018-09-27

AU

AU2018341551A

patent/AU2018341551B9/en

active

Active

Patent Citations (7)

* Cited by examiner, † Cited by third party

Publication number

Priority date

Publication date

Assignee

Title

WO2013018006A1

( en )

2011-08-03

2013-02-07

Koninklijke Philips Electronics N.V.

Position-sensitive readout modes for digital silicon photomultiplier arrays

US20140175294A1

( en )

2011-08-03

2014-06-26

Koninklijke Philips N.V.

Position-sensitive readout modes for digital silicon photomultiplier arrays

US9176241B2

( en )

2011-08-03

2015-11-03

Koninklijke Philips N.V.

Position-sensitive readout modes for digital silicon photomultiplier arrays

US20150285625A1

( en )

2014-04-07

2015-10-08

Samsung Electronics Co., Ltd.

High resolution, high frame rate, low power image sensor

US20170131143A1

( en )

*

2014-07-02

2017-05-11

Andreas G. Andreou

Photodetection circuit and operating method thereof

WO2017004663A1

( en )

2015-07-08

2017-01-12

The Commonwealth Of Australia

Spad array structures and methods of operation

US20180195900A1

( en )

2015-07-08

2018-07-12

The Commonwealth Of Australia

SPAD Array Structures and Methods of Operation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party

Title

Belbachir et al., " High-precision shape representation using a neuromorphic vision sensor with synchronous address-event communication interface ", Measurement Science and Technology, 2009, pp. 1-9, vol. 20.

Berkovich et al., " A Scalable 20x20 Fully Asynchronous SPAD-Based Imaging Sensor with AER Readout ", IEEE International Symposium on Circuits and Systems (ISCAS), 2015, pp. 1110-1113.

Also Published As

Publication number

Publication date

AU2018341551B9

( en )

2023-03-09

US20200326414A1

( en )

2020-10-15

AU2018341551B2

( en )

2022-11-03

WO2019060942A1

( en )

2019-04-04

EP3688809B1

( en )

2025-08-20

EP3688809A1

( en )

2020-08-05

EP3688809A4

( en )

2021-07-28

AU2018341551A1

( en )

2020-05-07

Similar Documents

Publication

Publication Date

Title

EP3688809B1

( en )

2025-08-20

A neuromorphic single photon avalanche detector (spad) array microchip

US12189038B2

( en )

2025-01-07

Processing system for LIDAR measurements

CN110389358B

( en )

2023-05-19

Related time-of-flight sensor

JP6761000B2

( en )

2020-09-23

Photoelectric sensors and methods for measuring distance

EP3320559B1

( en )

2021-03-03

Spad array structures and methods of operation

Niclass et al.

2012

A 100-m Range 10-Frame/s 340$\,\times\, $96-Pixel Time-of-Flight Depth Sensor in 0.18-$\mu\hbox {m} $ CMOS

Gyongy et al.

2023

A direct time-of-flight image sensor with in-pixel surface detection and dynamic vision

KR102409952B1

( en )

2022-06-17

High resolution, high frame rate, low power image sensor

TWI743471B

( en )

2021-10-21

First photon correlated time-of-flight sensor

US11626446B2

( en )

2023-04-11

Pixel circuit and method of operating the same in an always-on mode

KR20210150765A

( en )

2021-12-13

Image Sensing Device and Image photographing apparatus including the same

US11442153B2

( en )

2022-09-13

Systolic processor system for a light ranging system

US20250341619A1

( en )

2025-11-06

High photon rate spad device

Kumar et al.

2012

Low power time-of-flight 3D imager system in standard CMOS

US20240385325A1

( en )

2024-11-21

Direct-time-of-flight device, system, and method

US20250012902A1

( en )

2025-01-09

Time-of-flight circuitry and time-of-flight method

NZ738738B2

( en )

2021-11-30

Spad array structures and methods of operation

Legal Events

Date

Code

Title

Description

2020-03-25

FEPP

Fee payment procedure

Free format text : ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

2020-07-27

STPP

Information on status: patent application and granting procedure in general

Free format text : DOCKETED NEW CASE - READY FOR EXAMINATION

2020-08-20

AS

Assignment

Owner name : THE COMMONWEALTH OF AUSTRALIA, AUSTRALIA

Free format text : ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DELIC, DENNIS VICTOR;AFSHAR, SAEED;SIGNING DATES FROM 20200526 TO 20200603;REEL/FRAME:053549/0064

2022-04-04

STPP

Information on status: patent application and granting procedure in general

Free format text : NON FINAL ACTION MAILED

2022-06-08

STPP

Information on status: patent application and granting procedure in general

Free format text : RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

2022-06-27

STPP

Information on status: patent application and granting procedure in general

Free format text : NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

2022-08-25

STPP

Information on status: patent application and granting procedure in general

Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

2022-08-26

STPP

Information on status: patent application and granting procedure in general

Free format text : PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

2022-09-28

STCF

Information on status: patent grant

Free format text : PATENTED CASE

2026-03-10

MAFP

Maintenance fee payment

Free format text : PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment : 4

Related documents

Record · ID 1329540
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive