Spatial distortion of reproduced stereoscopic images is determined by a combination of factors including program production techniques, display devices, 3D glasses, viewing conditions, and viewer characteristics. It is highly desirable to predict beforehand the degree and type of spatial distortion of the reproduced stereoscopic images so that more natural and more comfortable stereoscopic images can be presented to viewers.
This document describes a spatial distortion prediction system for a 3DTV (see Ref. (1)). This system calculates the spatial distortion of a reproduced stereoscopic image and predicts the extent of the puppet-theatre and cardboard effects, excessive binocular parallax, and excessive parallax distribution on the basis of the shooting, display, and viewing conditions.
5.2 Spatial distortion in 3DTV
Conditions under which images are captured, displayed, and viewed can contribute to the introduction of spatial distortions, that is, the differences between the real and the reproduced 3D spaces. Some spatial distortions might cause unnatural effects, such as the puppet-theatre effect and the cardboard effect. The puppet-theatre effect is an undesirable miniaturization effect that makes people look like animated puppets; the cardboard effect is a stereoscopic distortion causing an unnatural depth perception, where objects appear flat as if the scene is divided into discrete depth planes. When some objects are close to the camera, the entire stereoscopic image might appear to pop out from the screen and excessive binocular parallax and excessive parallax distribution may occur. Excessive parallax might also arise for background objects, which is known to cause visual discomfort or to prevent binocular fusion.
5.3 Spatial distortion prediction system for 3DTV
5.3.1 Use cases
A system capable of predicting spatial distortion and excessive parallax would be of great benefit to the industry, helping to provide more natural and more comfortable stereoscopic images.
Because a stereoscopic image can only be viewed properly under a particular viewing condition for most stereoscopic displays, the system can be used to select appropriate shooting parameters for a particular “standard” display/viewing environment. For a programme directed at children, the system might also be used to tailor the shooting parameters to their small interpupillary distance.
When a director intends to emphasize the reproduced depth to make objects jump out from the screen to have an impact on the viewers, the director must manage the shooting conditions to avoid spatial distortion that causes the puppet-theatre effect and excessive parallax distribution. It is particularly difficult to produce the intended stereoscopic images for large displays when shooting on location. This is because a small stereoscopic display or even a 2D display is often used to monitor the stereoscopicity at a close distance or to merely measure the horizontal disparities, resulting in the director choosing the shooting conditions more by trial and error than careful selection. The system would make it possible for the director to control the stereoscopicity accurately and easily.
In a 3DTV broadcast, a broadcaster might have control of the shooting conditions but has little control over the display and viewing conditions. On the other hand, the opposite is true for the viewer. Even so, the system might help identify suitable viewing conditions to recommend to viewers.
5.3.2 System outline
The system calculates the spatial distortion of a reproduced stereoscopic image and predicts the extent of the puppet-theatre and cardboard effects, excessive binocular parallax, and excessive parallax distribution on the basis of the shooting, display, and viewing conditions listed in Table 10. The relationship between the space to be shot (real space) and the space of the reproduced stereoscopic image (reproduced space) is calculated geometrically in terms of their depth and size. The shooting conditions and the right and left images can be obtained from a stereoscopic camera system.
Parameters for shooting, display, and viewing conditions
Figure 16 shows a ground plan of real and reproduced space grids without spatial distortion; the real space grid (shown by red dots) and the reproduced space grid (shown by blue dots and texture) coincide. It should be noted that the real space grid is always displayed as a square. The shooting, display, and viewing conditions are shown in the left pane of the window. The camera field of view and the display viewing angle are equalized, as are the camera separation, interpupillary distance, and horizontal offset. The system can measure the parallaxes of up to three objects, namely the object of interest, background, and foreground. Each object’s depth in real space is calculated on the basis of the measured parallax and the shooting conditions. In order to determine the depth range of the space grid for calculating the spatial distortion, the user selects two portions, one at the maximum depth (a desk lamp in this example) and another at the minimum depth (a stuffed animal). A portion including the object of interest (a woman) should be selected to calculate the extent of the perceived puppet-theatre effect. The depth of the object of interest may be determined as the focus plane.
Screen shots of the spatial distortion prediction system
Figure 17 shows the simulation results obtained under four conditions for which some parameters were changed while other parameters remained the same as in Fig. 16. The extent of the perceived puppet-theatre and cardboard effects is expressed by changing the hue of the texture: magenta is increased in proportion to the extent of the perceived puppet-theatre effect and green is increased in proportion to the extent of the perceived cardboard effect. This system also produces an alert when excessive binocular parallax or parallax distribution is predicted.
Case 1: Large camera separation of 200 mm
The reproduced space shrinks, and the objects appear to be small.
Case 2: Narrow camera field of view of 15°
The overall depth in the reproduced space is reduced, and a large cardboard effect is expected.
Case 3: Short interpupillary distance of 45 mm
A slight puppet-theatre effect is expected, and the object of interest (woman) appears small.
Case 4: Short convergence distance of 2.3 m
The puppet-theatre effect is expected. Objects at a depth of approximately 7.5 m are reproduced at infinity.
NOTE – Magenta colour is increased in proportion to the extent of the perceived puppet-theatre effect. Green colour is increased in proportion to the extent of the perceived cardboard effect.
(1) K. Masaoka, A. Hanazato, M. Emoto, H. Yamanoue, Y. Nojiri, F. Okano, “Spatial distortion prediction system for stereoscopic images,” Journal of Electronic Imaging 15(01), 013002, 2006.