1. What is image noise

Wikipedia's definition: Image noise is a random variation of brightness or color information in images (which is not present in the object being photographed), and is usually an aspect of electronic noise.

It is generally produced by the sensor and circuitry of a scanner or digital camera, and can also be caused by film grain or the unavoidable shot noise in an ideal photodetector.

Image noise is an undesirable by-product of image capture, adding erroneous and extraneous information to the image.

The ISO definition: unwanted variations in the response of an imaging system.

To put it simply: it is erroneous, extraneous information that is not present in the photographed object itself but is introduced during the imaging process.

2. Classifications of noise

2-1. Classification by cause of noise generation

1) Internal noise

Noise caused by internal factors of the imaging system, such as noise arising from the fundamental properties of light and electricity, noise caused by the mechanical movement of electrical components, noise caused by the equipment materials themselves, and noise caused by the circuits of internal system devices.

Taking the 3T-APS (3-Transistor Active Pixel Sensor) CMOS structure as an example:

• The Micro-Lens is responsible for focusing light (here, noise is introduced due to differences in the forms of the QCFA, PD, etc., and pixel noise can also result from crosstalk caused by design issues);
• The Color Filter (the red region; the manufacturing process also introduces noise here, most of which is eliminated through calibration);
• The Reset Transistor (reset noise, or kTC noise, is introduced here, appearing at the moment the MOS switch turns off);
• The Amplifier Transistor (the amplifier also introduces noise, mainly analog noise here);
• The Column Bus Transistor (mainly transmission noise or readout noise here).

The largest source of noise is the photodiode (mainly involving Poisson distribution noise and thermal noise, with dark current noise as a secondary contributor). All of these are internal noise.

There is a classic image of pixel noise caused by photosensitivity:

2) External noise

Noise caused by external factors of the imaging system, such as external electrical equipment, celestial discharge phenomena, etc. These affect the interior of the imaging system in the form of electromagnetic waves or electric currents, thereby generating noise.

Taking the figure below as an example: after a pixel generates a voltage through the photoelectric effect, noise is introduced at every step during signal conversion and transmission. The higher the frequency and the lower the voltage, the more sensitive and prominent the noise becomes. These include power supply noise, analog amplifier noise, ADC noise, digital amplifier noise, and transmission noise.

a. Power supply noise — more often manifested as column noise?

b. Analog amplification noise — analog amplification introduces noise, but it is also easier to mitigate through circuit design. 

c. ADC noise — quantization noise is an unavoidable hurdle, but this problem gradually weakens as sampling precision increases.

SNR = 6.02N + 1.76dB is the commonly used consensus formula for quantization noise, where N is the number of ADC bits. This formula also shows that the larger the bit width, the higher the signal-to-noise ratio. d. Digital amplification noise — as digital gain is used in more and more scenarios, the noise problems it introduces are becoming increasingly apparent. e. Transmission noise — noise introduced during transmission, mostly related to high-frequency electromagnetic fields around the signal lines, which can be mitigated through methods such as grounding guards (copper pour shielding) and ferrite beads.

2-2. Classification by statistical properties

Stationary noise refers to noise whose statistical properties do not change over time; white noise, for example, is a typical stationary noise.

Non-stationary noise refers to noise whose statistical properties change over time; for example, when multi-frame noise reduction is applied to video, the noise in motion areas is typical non-stationary noise.

2-3. Classification by the relationship between noise and the image signal

1) Additive noise

Additive noise is independent of the intensity of the image signal, such as channel noise introduced during image transmission.

An image containing additive noise can be expressed as: f = g + n, where g is the assumed ideal noise-free image, n represents the additive noise, and f is the final image containing noise.

2) Multiplicative noise

Multiplicative noise is related to the intensity of the image signal — it changes as the intensity of the image signal changes. Examples include the noise of flying-spot scanners when scanning images, the noise generated by TV scanning rasters, and film grain noise.

An image containing multiplicative noise can be expressed as: f = g + n × g.

2-4. Classification by probability density distribution of the noise

1. Gaussian noise
2. Rayleigh noise
3. Gamma noise
4. Exponential noise
5. Uniform noise
6. Impulse (salt-and-pepper) noise — named after the colors of salt and pepper: "salt" refers to bright spots in dark areas, and "pepper" refers to dark spots in bright areas. Hot pixels in CMOS sensors appear as salt-and-pepper noise in images.

2-5. Classification from the perspective of image post-processing

a. From the frequency domain: noise can be divided into high-frequency, mid-frequency, and low-frequency noise. b. From the color space: noise can be divided into luma noise and chroma noise. c. From the spatial-temporal domain: spatial noise and temporal noise.

Spatial noise refers to unwanted signal variations formed in space, such as the high-, mid-, and low-frequency noise mentioned above; luma and chroma noise are also spatial noise.

Temporal noise refers to unwanted signal variations formed over time between consecutive frames, appearing as flickering noise, which is more noticeable under low illumination.