At present

3D videos have become extensively integrated into the daily lives of people due to the immersive visual experience that they provide to users. However

viewers can experience visual discomfort when watching 3D videos

and even suffer from eye fatigue

headache

nausea

and other symptoms due to defects in 3D imaging technology. Therefore

the study of visual comfort enhancement methods for stereoscopic images or videos is highly significant to improve stereoscopic display technology and provide users with higher-quality 3D vision service. The factors that can cause visual discomfort when people watch stereoscopic images or videos include the followings: vergence-accommodation conflict

excessive cross and non-cross disparities

disparity distribution

spatial frequency

mismatch between left and right images

and object movement. Vergence-accommodation conflict is the fundamental cause of visual discomfort. Binocular vergence-accommodation conflict is characterized by a large disparity that occurs in 3D space. If the disparity is outside the fusion range

then the viewer cannot fuse the left and right images into a stereoscopic image

and instead

will see an unclear crosstalk image

thereby resulting in severe visual fatigue. Disparity distribution is also one of the main factors that affect visual comfort. Excessive cross disparity is more likely to cause visual discomfort than excessive non-cross disparity. When the entire image is located in front of the screen

visual comfort will be lower compared with when the entire image is positioned behind the screen. The disparity distribution of an image is more concentrated on the zero-disparity plane

thereby making the image more comfortable to view. As dispersion decreases

viewing the image becomes more comfortable. Spatial frequency also influences visual comfort by affecting binocular fusion limit. An image with high spatial frequency causes a higher degree of visual discomfort than an image with low spatial frequency. Disparity adjustment is the main method that can enhance the visual comfort of stereoscopic images because the vergence-accommodation conflict caused by the increased disparity is the main factor that leads to visual discomfort. Disparity adjustment methods can be divided into two categories: disparity shifting and disparity scaling. A disparity shifting method adjusts disparity by shifting the zero-disparity plane of the original image

thereby keeping the disparity range unchanged. Although this method has low computational complexity

simultaneously ensuring maximum cross disparity and non-cross disparity within the comfort zone is difficult regardless of how disparity is moved when the original disparity range exceeds a certain range of comfortable viewing area. Thus

visual discomfort remains unavoidable in this case. By contrast

the disparity range of the original scene can be linearly or nonlinearly adjusted into the comfort area by using a disparity scaling method. In general

excessive vergence-accommodation conflict can be avoided effectively by reducing the disparity range of the scene. However

when a large-scale disparity reduction is performed

the overall perceived depth of the stereoscopic image is significantly decreased

and an unnatural visual effect occurs due to the limited range of the comfortable viewing area. A new visual comfort enhancement method for stereoscopic images is proposed by combining global linear and local nonlinear disparity remapping based on the effect of disparity on visual comfort. This method can prevent visual discomfort when viewing stereoscopic images; it also balances the improvement of visual comfort of stereoscopic images and the weakening of the 3D sense of scenes. First

an objective visual comfort assessment model is constructed to automatically predict the visual comfort of stereoscopic images and to judge the improvement of the visual comfort of stereoscopic images during disparity adjustment. On the one hand

when binocular fusion limitation is considered

the global visual comfort features of stereoscopic images are extracted by combining spatial frequency and disparity. On the other hand

we perform a disparity statistical analysis on stereoscopic significant regions and obtain local visual comfort features based on the hypothesis that the human eye tends to pay excessive attention to perceived salient regions. Support vector regression is adopted in this study to construct the objective visual comfort prediction model for stereoscopic images by establishing the mapping relationship between features and subjective scores. Then

the visual comfort of the input stereoscopic image is analyzed using the constructed prediction model. A two-stage disparity remapping strategy is designed for less-comfortable stereoscopic images. This strategy consists of the global linear adjustment of the disparity range and the local nonlinear adjustment of the disparity in the extracted potentially less-comfortable regions. The global disparity remapping of the input disparity map is performed during the first stage to adjust the uncomfortable stereoscopic images to a relatively comfortable degree. The global disparity linear iterative adjustment process is performed if the predicted visual comfort objective score is less than the preset threshold. Only the global features are applied at this point to construct the visual comfort prediction function. Local nonlinear disparity remapping is then performed during the second stage to further enhance the viewing comfort of the stereoscopic image and maintain the 3D sense of the scene. The disparity of the potentially less-comfortable regions extracted from the disparity map after global linear remapping is adjusted via nonlinear iteration until the predicted visual comfort objective score is higher than the preset target threshold. The visual comfort of the adjusted stereoscopic image is predicted in conjunction with global and local features at this point. Lastly

an updated comfortable stereoscopic image is reconstructed via a rendering technique according to the remapped disparity map. A subjective evaluation experiment is designed on the IVY Lab stereoscopic image database to verify the effectiveness of the proposed method in improving the visual comfort and maintaining the 3D sense of stereoscopic images. Experimental results show that the proposed method can more effectively enhance the visual comfort of less-comfortable stereoscopic images while maintaining the 3D sense of scenes compared with state-of-the-art stereoscopic image visual comfort enhancement methods. The proposed method can automatically implement global linear and local nonlinear disparity remapping processes based on the visual comfort prediction model constructed with different features of stereoscopic images. The proposed method can realize the purpose of improving the visual comfort of stereoscopic images under the premise of ensuring 3D sense

which enhances the overall 3D experience of stereoscopic images.