Detailed CCD and CMOS image sensor shutter

1. Some basic knowledge
Before introducing the principle of image sensor, this article first introduces some keywords related to image sensor performance, as follows:

1.1 Noise correlation
Image sensor noise is divided into fixed pattern noise, transient noise, smear and highlight:
(1) Fixed pattern noise: The English abbreviation is FPN (fixed-pattern noise), which is the noise that appears at a fixed position in the image. The image sensor mainly includes white spot defects and shadows;
(2) Transient noise: It is the noise that changes with time. The three basic transient noises in electricity and optics are thermal noise, shot noise, and 1/f noise. The image sensor mainly includes reset (kTC) Noise, read noise (noise floor), dark current shot noise, photon shot noise, input reference noise and output reference noise, etc.;
(3) Smearing and highlight overflow: Smearing appears as white vertical stripes, which usually occurs when diffused light enters the V-CDD register or when the charge generated in the depth of the bulk silicon diffuses into the V-CCD. Highlight overflow occurs when the photo-generated charge exceeds the full well capacity of the pixel, and the overflow charge will enter the adjacent pixel or V-CCD.

Dark current refers to the current observed by the target object under no light conditions. The dark current will be integrated into dark charge and stored in the charge storage node in the pixel. The above-mentioned fixed mode noise and transient noise both have a part caused by dark current.

2. Introduction

In a digital camera, the light of the scene is converted into an electric signal to form an image through the photodiode in the CCD or CMOS image sensor. The electric signal generated by the sensor's photosensitive element depends on the light intensity and the illumination time (integration time). Therefore, the digital camera The shutter is required to control the exposure time, which can usually be achieved using external mechanical shutters and on-chip electronic shutters.

For digital cameras equipped with a mechanical shutter, the integration time of the photosensitive element depends on the mechanical shutter. Some image sensors have electronic shutters. These sensors allow the elimination of mechanical shutters. A combination of electronic shutters and mechanical shutters is usually used to control exposure.

There are three electronic shutter mechanisms:

(1) CCD image sensor with electronic shutter

(2) CMOS image sensor with rolling shutter

(3) CMOS image sensor with global shutter

3. CCD with electronic shutter

The CCD image sensor has two scanning methods: progressive and interlaced. These two scanning methods affect the use of mechanical shutters.

(1) For progressive scan sensors, the entire sensor is reset to eliminate the residual signal of the photodiode before integration. The photodiode is charged during the exposure period. After the charging is completed, all the charged charges are transferred to the dark area of the sensor, and then the charge Being removed from the light-proof area and read out, this mode does not require a mechanical shutter to control exposure.

(2) For interlaced scanning sensors, at the beginning of exposure, the electronic shutter circuit will also reset all photodiodes, but at the end of exposure, not all the charged charges are removed from the photodiodes by colleagues. The odd and even lines are in different The time is shifted out, so the sensor cannot properly end the exposure: odd and even lines will have different integration times. In this case, a mechanical shutter is required to terminate the exposure correctly: a mechanical shutter is closed, and the odd and even field can be It is read out at the same time and the exposure time is the same. Most consumer cameras operate in this way.

4. Rolling shutter CMOS image sensor

The rolling shutter generally means that the pixel rows in the image sensor are reset sequentially, starting from the top row, and processing row by row until the bottom row. When the reset process has performed some rows, the readout process starts, and the pixel rows are sequentially read out to the bottom row at the same speed as the reset.

The delay between row reset and row readout is the integration time. The integration time (exposure time) can be controlled by changing this delay time. For rolling shutters, the integration time can range from one line time to a whole frame time.

Because the start time of each line of integration is different, or the exposure is one line by line, the rolling shutter will have flaws on the moving image. We call this the rolling shutter effect.

5. CMOS sensor with global shutter

Global exposure means that the entire sensor is reset at the same time before integration and then starts integration at the same time. After the integration time is reached, integration stops at the same time and transfers the accumulated charge to the dark storage area, and then reads out from the dark storage area, so there is no rolling shutter. Blemishes in sports scenes

6. The use of flash in CMOS camera

The flash is usually used in low illumination to make the image suitable for exposure. The flash time of the flash is very short, usually only hundreds of microseconds to milliseconds. The readout time of the CMOS image sensor is much longer, so the use of the flash for the CMOS image sensor Special considerations, the following three situations are discussed

(1) CMOS image sensor with mechanical shutter (rolling shutter or global)

(2) Global shutter CMOS camera

(3) Rolling shutter CMOS camera

7. How to use flash for CMOS cameras with both rolling shutter and mechanical shutter

If the mechanical shutter is used with a CMOS sensor, the flash operation is as follows:

(1) The integration time of the electronic shutter of CMOS is adjusted to allow all pixels to integrate during flash, which will require a very long integration time. Because the flash is generally used when the light is very low, all pixels must be integrated when the flash fires.

(2) The mechanical shutter opens

(3) The flash fires

(4) Mechanical shutter closed

(5) Data read

Because the flash lighting time is very short relative to the time for the entire frame to be read out, the flash can only be used when all pixels are integrated at the same time. Consider this situation: the delay (integration time) of reset and readout is less than the entire frame time, such as 1/4 of the entire frame line number. Obviously, only part of the line exposure can use the flash.

8. How to use flash in CMOS camera with rolling shutter but no mechanical shutter

In this case, use the flash as follows:

(1) The integration time is adjusted so that all pixels can integrate at the same time during the flash (this will require a very long integration time)

(2) The reset process is performed line by line until the entire sensor is reset.

(3) The flash fires.

(4) Read the image line by line

Because all pixels of the sensor must be integrated at the same time, in order to use the flash, the minimum exposure time should be set to the total time to reset the entire sensor plus the flash lighting time.

The net exposure of this mode is the integral of the ambient light and the flash light. In order to obtain the best image quality, the ambient light should be significantly lower than the minimum illumination brightness of the flash, so that the flash light contributes absolutely to the exposure Component, due to the slow reset and read-out processing speed, the minimum exposure time is sufficient to cause blur due to the movement of the camera or target. In order to minimize this blur, the light of the flash should be increased.

9. How to use flash for CMOS camera with global shutter

In this case, the flash should be used as follows:

(1) The mechanical shutter is open (if any)

(2) All pixels are reset at the same time

(3) The flash fires

(4) The charged charge of each pixel is simultaneously transferred to the dark storage area

(5) The mechanical shutter is closed (if any)

(6) Read the data.

The time required for steps (2) to (4) is only the lighting time of the flash. We know that the lighting time is very short. As a result, the contribution of exposure mainly comes from the flash, and the impact of ambient light is small.

For more information about our products, please visit our product website: http://www.cmr-cctv.com