3.1 Parallax distribution and visual comfort of stereoscopic images
One of the issues that arises when making stereoscopic images widely available to large numbers of viewers is the visual discomfort experienced by viewers of some scenes. This discomfort is thought to be caused by discrepancies and crosstalk between the characteristics of the images presented to the left and right eyes, such as differences in their geometrical characteristics or video properties, and is thought to be more of a problem in scenes where these characteristics are prominent. Another factor is parallax itself, which plays a key role in conveying the sense of depth.
It has often been noted that excessive parallax can cause visual fatigue. There are many productions where surprisingly large amounts of parallax are used intentionally for dramatic impact. If we can find out about the distribution of parallax within the same frame and how this distribution affects the viewer's visual discomfort, then this should provide us with very useful clues for the production of visually comfortable stereoscopic images in each scene. It can sometimes be difficult to analyse the parallax values included in a frame. In what follows, we describe such an analysis and apply it to a number of stereoscopic images. We then compare its results with those of subjective evaluation tests.
3.1.2 Parallax measurements
Here we summarize the parallax measurement method used in this study.
For many applications, we ideally need to be able to determine the depth (i.e. the amount of parallax) for each pixel of an image. However, algorithms for analysing parallax are generally prone to errors. Also, in practice there are cases where corresponding pixels do not exist in the images presented to the left and right eyes due to occlusion. In this study, we do not need to measure parallax on a strict per-pixel basis as long as we are able to extract the characteristics of a parallax distribution from the images. To achieve this aim, the method we propose combines phase correlation with a number of threshold processing methods. A detailed description of this algorithm can be found in the Reference below. The algorithm was used in the parallax analysis of stereoscopic images discussed below.
3.1.3 Subjective evaluation tests of parallax distributions and visual comfort
We performed subjective evaluation tests to investigate the relationship between visual comfort of stereoscopic images and their parallax distributions11. The subjective evaluation test conditions are shown in Table 7. The images used for the evaluation consisted of 48 different still images. These images were presented as stereoscopic images with 2D images as a standard reference, and their relative visual comfort was evaluated on a seven-grade scale. The 2D images were produced by presenting the left-eye image to both eyes and were evaluated by test subjects wearing the same polarizing glasses used for the stereoscopic images. The parallax in the stereoscopic images was measured by using the phase correlation method discussed in § 2. Here, the amount of parallax was measured on a per-pixel basis with the screen corresponding to a value of zero, and positions behind and in front of the screen correspond to positive and negative parallax values respectively. In the viewing conditions shown in Table 7, the amount of parallax of a single pixel corresponds to a separation of approximately 1 mm between the left and right images on the screen. Figure 10 shows the results of the visual comfort subjective evaluation tests and the results of measuring the amount of parallax in the images. In the graphs of this figure, the numbers on the horizontal axis designate the images to be evaluated. The graph at the top shows the results of the subjective evaluation tests. The vertical axis shows the mean value of the evaluation scores from 24 evaluators, and the vertical bars represent the standard deviations of these scores. The lower graph shows the results of measuring the amount of parallax. The vertical axis shows the amount of parallax measured in pixel units, the plotted points show the average values of the parallax in the images, and the vertical bars show the range of the parallax distributions. The upper and lower ends of the vertical bars represent the maximum and minimum parallax values. A comparison of the two graphs shows that the images with an evaluation score of 3 or less have a very large parallax distribution range.
Conditions of subjective evaluation tests of parallax distributions
Images used in test
48 still images (including a standard pattern)
24 adult males and females (not expert)
10 s viewing of 2D image (for reference), following by 10 s viewing of stereoscopic image (for evaluation)
Evaluation test results and parallax measurement results
On analysing the correlation between the results of the subjective evaluation tests relating to visual comfort and the statistical quantities (mean, range, minimum, maximum, variance) of the parallax distributions, it can be seen that the parallax range exhibits a strong correlation with a correlation coefficient of −0.86 (99% confidence).
Figure 11 shows the relationship between the parallax distributions and visual comfort of the images used in the evaluation. The vertical axis shows the amount of parallax in pixel units with a value of zero corresponding to the position of the screen, and the horizontal axis shows the visual comfort derived by using the method of successive categories to combine the psychometric values. The plotted points represent the mean values of the parallax distributions in the images, and the vertical bars represent the range of the parallax distributions. From Fig. 11, we can see that in each image evaluated as being visually comfortable, the range of the parallax distribution is approximately 60 pixels or less. This translates to a value of 0.3 diopters. Images are evaluated as being comfortable when the parallax distributions are in the range from approximately 30 pixels in front of the screen to approximately 65 pixels behind it.
Next, we investigated the relationship between the average values of the parallax distributions and the visual comfort of the images. With seven of the images, we performed visual comfort evaluation tests in which the average value of the parallax distribution was shifted to different positions. These tests were performed with 20 test subjects. The other test conditions were the same as in
Table 7. Figure 12 shows the experimental results. In this figure, the points plotted with outlined symbols represent data obtained without horizontal shifting. As Fig. 12 shows, as the average value of the parallax distribution became closer to the screen position, the images were evaluated as being more visually comfortable.
Parallax distribution vs. visual comfort
Average values of the parallax distributions vs. visual comfort
3.1.4 Subjective evaluation of the sense of presence
When scenes are limited to small values of parallax there might be a reduction in the positive effects of the stereoscopic images, such as the sense of presence. In the tests reported in § 3, the images were evaluated in terms of their sense of presence as well as their visual comfort. Specifically, the stereoscopic images were presented with 2D images as a standard reference, and their sense of presence was evaluated on a seven-grade scale.
In the analysis of the test results, we found no significant correlation between the sense of presence scores and the statistical quantities of the parallax distributions. We extracted the images for which the stereoscopic image was evaluated as more visually comfortable than the 2D image (35 images in total), and as a result of analysing these images, we showed that there is a strong correlation between the range of the parallax distribution and the sense of presence (correlation coefficient 0.65).
On the other hand, we observed no factorial effect of the average value of the parallax distribution on the sense of presence evaluation scores.
3.2 Visual comfort and discomfort in viewing stereoscopic images
3.2.1 Discrepancies between left and right images
Here we present the results of evaluating visual comfort/discomfort in viewing stereoscopic images from the perspective of discrepancies between left and right images in the performance or characteristics of image capture and display equipment. A series of experiments was conducted on discrepancies in size, verticality, inclination, and brightness, as well as on cross-talk regarding discomfort. Natural images of HDTV were mainly used for the evaluation. The major findings of the studies are summarized in Table 8. The values represent the results of a subjective evaluation test on the five-grade impairment scale.
When shooting and displaying stereoscopic 3DTV programmes, there can be geometrical distortions, such as size inconsistency, vertical shift, and rotation error, between left and right images . Results of experiments where three kinds of distortions occurred independently from each other are shown in Table 8.
As differences between left and right images in amplitude and offset can be corrected but clipped white or black levels cannot, the degree of interference when the white or black level of one of the left and right images is clipped was evaluated. The evaluation results on detection and tolerance limits were reported to depend on image content . Some of the earlier studies on differences in brightness and contrast are given by References  and .
Stereoscopic image cross-talk, in which the left and right images “leak” and can be partially seen by the opposite eye, can also result in discomfort for the viewer, and experiments evaluating it have also been done. These values on detection and tolerance limits were reported to be highly dependent on image content and display contrast, and cross talk must be reduced further still on high contrast displays . One of earlier studies on cross-talk is given by Reference .
3.2.2 Depth range, distribution and change in parallax
Cases of extreme parallax or sudden changes in parallax cause visual discomfort, so it is important to manage parallax with special care when producing programs with stereoscopic images. Results of experiments subjectively evaluating the relationship between visual comfort and the distribution of parallax in images, and the change in parallax before and after scene cuts are shown in Table 9. The positioning of subtitles has also been evaluated. All tests were done under standard viewing conditions for HDTV. The range of parallax for visual comfort has already been reported in § 3.1 of Annex 4.
Research on impairment caused by discrepancies and cross-talk between L/R stereo images
L/R image discrepancy
Taking size of one image as 100%
Taking image height
Angle of rotation about image centre
White clip level
Taking 100 IRE white level as 100%
Black clip level
Taking 100 IRE white level as 100%
Contrast ratio of 100:1
Research on parallax distribution, parallax range and parallax change over time
Comfortable to view if parallax range is within 60 min.
Preferable if subtitle positions 10-15 min. in front of background image
Uncomfortable to view if temporal change of parallax exceeds 60 min.
 H. Yamanoue, M. Nagayama, M. Bitou, J. Tanada, T. Motoki, T. Mitsuhashi, and M. Hatori, “Tolerance for Geometrical Distortions between L/R images in 3D-HDTV,” Systems and Computers in Japan, Vo1. 29, No. 5, 1998.
 A. Hanazato, M. Okui, and F. Okano, “Evaluation of Stereoscopic Images Impaired by the Differences in Black/White Levels,” ITE Annual Convention 2001, 4-5, pp. 49-50 (2001) (in Japanese).
 J. Fournier and T. Alpert, “Human factor requirements for a stereoscopic television service - Admissible contrast differences between the two channels of a stereoscopic camera,” Proc. SPIE, Vol. 2177, pp. 45-54, 1994.
 P. Beldie and B. Kost, “Luminance Asymmetry in Stereo TV Images, ” SPIE, Vol. 1457, pp. 242‑247, 1991.
 A. Hanazato, M. Okui, and I. Yuyama, “Subjective Evaluation of Cross Talk Disturbance in Stereoscopic Displays,” The 20th International Display Research Conference, pp. 288-291, Sep. 2000.
 S. Patoor, “Human Factors of 3D imaging: Results of Recent Research at Heinrich‑Hertz-Institut Berlin,” Proc. of the 2nd International Display Workshops (IDW’95), Vol. 3, 3D-7, pp. 69-72, 1995.
 S. Ide, H. Yamanoue, M. Okui, F. Okano, M. Bitou, and N. Terashima, “Parallax distribution for ease of viewing in stereoscopic HDTV,” Proc. SPIE, Vol. 4660, pp. 38‑45, 2002.
 Y. Nojiri, H. Yamanoue, A. Hanazato, and F. Okano, “Measurement of parallax distribution, and its application to the analysis of visual comfort for stereoscopic HDTV,” Proc. SPIE, Vol. 5006, pp. 195-205, 2003.
 Y. Nojiri, H. Yamanoue, A. Hanazato, M. Emoto, and F. Okano, “Visual comfort/discomfort and visual fatigue caused by stereoscopic HDTV viewing,” Proc. SPIE, Vol. 5291, pp. 303-313, 2004.
It has been confirmed through subjective evaluation that viewing stereoscopic images can result in a great degree of visual fatigue compared to viewing 2D ones. On this issue, it has been shown that changes in visual performance can be observed before and after viewing stereoscopic images, and that the fusional amplitude (the parallax range over which viewers can fuse left and right images) in particular decreases. The experimental result suggested the possibility that a fusional amplitude can be one of indices for evaluating visual fatigue12.
Here, conditions in which vergence and accommodation are not consistent were reproduced using specialized equipment, and an objective evaluation of visual fatigue was obtained by measuring fusional amplitude before and after using these glasses for one hour. Results of these trials are shown in Figs 13 and 14. In both experiments, a common one-hour HDTV programme was used. The figures indicate mean values of ratios of the relative range of convergence (a relative value using the fusional amplitude before viewing as the basis), with small values indicating a narrower fusional amplitude.
The results from viewing a flat image in 3D for one hour with a fixed amount of parallax (such that the image is displayed in front or behind the screen) are shown in Fig. 13. The results of changing the parallax over time with the vergence and accommodation being consistent in one case, and not consistent in the other are shown in Fig. 14. Parallax was changed 16 times over a period of two minutes, and this was repeated 29 times. In both experiments, the same one-hour HDTV programme was used.
A large change in ratio value was observed when viewing images with parallax than when viewing 2D ones, and even more fatigue was caused by the time-varying images with inconsistencies between vergence and accommodation. These results indicate that inconsistencies between vergence and accommodation as well as fluctuations in time due to parallax, can be factors causing visual fatigue13.
4.2 Experimental results on parallax amount and lateral/depth motion
A series of experiments have also been done to examine the relationship between this issue of inconsistency between vergence and accommodation, and depth-of-focus of the eyes.
First, the accommodation responses were measured14. Subjects viewed image content for approximately one hour on a 120-inch screen at a viewing distance of 4.5 m. The 3D image contents were two motion video sequences, and parallax for the video displayed under the conditions of the experiment was within the depth of field in almost all cases, 2D video was also used for reference.
The result was assumed significant for visual fatigue when the change of amplitude of the accommodation response in the before and after viewing was bigger than 0.5 diopters. From this aspect the results of comparing accommodation response before and after viewing showed no significant difference for the 2D images, whereas for the 3D images, three of five subjects for the first image sequence and two of five subjects for the second sequence showed visual fatigue. On the basis of these results and the fact that the video used was within the depth of focus, it is presumed that causes of visual fatigue other than depth of focus must also be considered.
Next, experiments were done to compare images in different amount of parallax, as well as 3D images with and without motion15. In the experiments, Japanese text with added parallax was displayed on a field-sequential 3D HD monitor (28-inch diagonal). From a viewing distance of 3H, subjects read for approximately one hour while turning pages using a mouse. This was repeated several times, changing the amount of parallax. Experiments were also done with moving text. Two types of text motion were tested: forward and backward in the depth direction, and horizontal motion. Here, the amount of parallax was limited within the depth of focus. The timing of motion was generated from an existing 3D programme.
The degree of fatigue was estimated on the basis of subjective evaluation and accommodation response. The subjective evaluation results are presented in Fig. 15. The figure shows there was a significant difference at the parallax of 1.36 degrees whereas at +1.36 degrees there was not, i.e. there was a large variance. Fatigue was also inferred when there was motion in the depth direction. These results suggest that changes in the depth direction can cause visual fatigue even when the amount of parallax remains within the depth of focus.
Subjective evaluation of visual fatigue while changing amount of parallax
and stationary vs. moving objects
Figure shows mean values with standard deviation.
Asterisk indicates significant difference from still image on screen