Objective

2

Low-illumination images are easily produced when taking pictures because of weak lighting conditions or devices with poor filling flash. Low-illumination images are difficult to recognize. Thus

the quality of low-illumination images needs to be improved. In the past

low-illumination image enhancement was dominated by histogram equalization (HE) and Retinex

but these methods cannot easily generate the desired results. Their results often entail problems

such as color distortion and blurred edges. A conditional generative adversarial network (CGAN)-based method is proposed to solve this poor visual perception problem. CGAN is an extension of the generative adversarial network (GAN). At present

it is widely used in data generation

including image de-raining

image resolution enhancement

and speech denoising. Unlike traditional low-illumination image enhancement methods that work on single image adjustment

this method achieves data-driven enhancement.

Method

2

This study proposes an encode-decode convolutional neural network (CNN) model as the generative model and a CNN model with a classification function as the discriminative model. The two models constitute a GAN. The model processes input images end to end and without adjusting the parameters manually. Instead of using synthetic image datasets

real-shot low-illumination images from the multi-exposure image dataset are used for training and testing. This image dataset contains multi-exposure sequential images

including under-and over-exposure images. The exposure of images is shifted by the exposure value (EV) of cameras or phones. Moreover

this dataset offers high-quality reference light images. During training

by offering reference light images from datasets as conditions to GAN

both models optimize their parameters according to the light images. As a result

the entire model is transformed into CGAN. The coding path of the generative model samples low-illumination images and processes the images at different scales. After coding

the encoding path restores the image size and shortens the distance between the outputs and conditional light images. The low-illumination images are denoised and restored by a different convolution processing of the generative model

and enhanced images are obtained. The discriminative model distinguishes the enhanced and reference light images by comparing their differences. The enhanced images are regarded as false

and the reference light images are regarded as true. Then

the discriminative model provides the result to the generative model. According to the feedback

the generative model optimizes the parameters to obtain an improved enhancement capability

and the discriminative model obtains an improved distinguishing capability by optimizing its own parameters. After training thousands of pairs of images

the parameters of both models are optimized. By using the discriminative model to supervise the generative model and by combining the interrelation between the two models

an improved image enhancement effect is achieved. When the proposed model is used to enhance low-illumination images

the discriminative model is no longer involved in the work

and the result is obtained directly from the generative model. Furthermore

skip connection and batch normalization are integrated into the proposed model. Skip connection transmits the gradient from shallow to deep layers. It has a transitional effect on the shallow and deep features. Batch normalization can effectively avoid gradient vanishing and explosion. Both approaches enhance the processing capability of the model.

Result

2

In this study

the entire network model and the single generative model are compared; the two sets of models represent CGAN and CNN methods

respectively.

Results

2

show that the entire network model has a better processing effect than the single generative model. This finding proves that the discriminative model improves the effect of the generative model during training. Afterward

eight existing methods are applied for comparison with the proposed method. By subjectively comparing the results of these methods

we find that our method achieves a better effect in terms of brightness

clarity

and color restoration. By using the peak signal-to-noise ratio (PSNR)

histogram similarity (HS)

and structural similarity (SSIM) as the objectives of comparison

our method exhibits improvements of 0.7 dB

3.9%

and 8.2%

respectively. Meanwhile

the processing time of each method is compared. By using a graphics processing unit (GPU) for acceleration

the proposed method becomes much faster than the other methods

especially traditional central processing unit (CPU)-based methods. The proposed method can meet the requirement of real-time applications. Furthermore

for several low-illumination images with bright parts

our method does not enhance these parts

whereas other existing methods always over-enhance the bright parts.

Conclusion

2

A conditional generative adversarial network-based method for low-illumination image enhancement is proposed. Experimental results show that the method proposed is more effective than existing methods not only in perception but also in speed.