Output - Draft New Recommendation P.3D-sam: “Subjective assessment methods for 3D video quality”
TSB Note: This document represents the output from Q12/9 session during SG 9 meeting, 14-18, January 2013.
Draft P.3D-sam: Subjective Assessment Methods for 3D Video Quality AAP Summary
This Recommendation describes non-interactive subjective assessment methods for evaluating the one-way overall video quality for three dimensional (3D) video applications such as 3D videoconferencing, and 3D cable television. These methods can be used for several different purposes including, but not limited to, selection of algorithms, ranking of system performance and evaluation of the quality level during a video connection. This Recommendation also outlines the characteristics of the source sequences to be used, like duration, kind of content, number of sequences, etc.
Details within this document are expected to change, based on experiments into how best to conduct 3DTV subjective tests.
Stereoscopic three-dimensional (3D) television attempts to emulate the response of the human binocular visual system to the relative depth perception of objects. This Recommendation applies to stereoscopic imaging that directs a different view of the same scene to each eye. The images of the objects depicted in the scene will have different relative positions in the left- and right-view. 3D television viewing does not perfectly recreate the real viewing experience, because the normal formulae for accommodation (i.e., focus) and vergence (i.e., eye angle) do not apply. A variety of displays produce this effect, including stereoscopic and autostereoscopic displays, using glasses with polarized lenses or shutters; and 2D televisions using complementary color anaglyphs and glasses with colored filters.
Assessment factors generally applied to monoscopic (two-dimensional, or 2D) television pictures can be applied to stereoscopic television systems. In addition, there are many factors unique to stereoscopic television systems. These include factors such as depth resolution, which is the spatial resolution in depth direction, depth motion (that is, whether motion or movements along depth direction), and visual comfort.
This Recommendation describes subjective assessment methods for 3D video quality in terms of light condition, viewing distance, comfort level, maximum crosstalk, display size, etc.
[ITU-T P.910] ITU-T Recommendation P.910 (2008), Subjective video quality assessment methods or multimedia applications.
[ITU-T P.911] ITU-T Recommendation P.911 (1998), Subjective audiovisual quality assessment methods for multimedia applications.
[ITU-R BT.1788] ITU-R Recommendation BT.1788 (2007), Methodology for the subjective assessment of video quality in multimedia applications .
[ITU-R BT.500] ITU-R Recommendation BT.500-13 (01/2012), Methodology for the subjective assessment of the quality of television pictures. [ITU-R BT.2021] ITU-R Recommendation BT.2021 (08/2012), Subjective methods for the assessment of stereoscopic 3DTV systems.
This recommendation uses the following terms defined elsewhere.
3.1 Terms defined elsewhere:
This Recommendation uses the following terms defined elsewhere:
3.2 Terms defined in this Recommendation
This Recommendation defines the following terms:
3.2.1 Depth quality refers to the ability of the system to deliver a sensation of depth. The presence of monocular cues (such as linear perspective, blur, gradients) convey some sensation of depth even in standard 2D images. Stereoscopic 3D images contain also disparity information which provides additional depth information and thus an enhanced sense of depth as compared to 2D.
3.2.2 Double Stimulus: a quality rating method where the subject is presented with two stimuli and then rates both stimuli in the context of the joint presentation (e.g., a rating that compares the quality of one sequence to the quality of the other sequence).
3.2.3ImagePicture Quality is related to the quality of rendering of texture and motion, the level of visibility of visual artifacts and rendering details. It refers to the perceived quality of the video provided by the system. This is a main determinant of the performance of any video system. Picture quality is mainly affected by technical parameters and errors introduced by, for example, encoding and/or transmission processes and/or 3D video formats conversions e.g. 2D plus depth to stereoscopic images. [editor’s note: definition and term will be discussed further and perhaps modified.]
3.2.4Naturalness refers to the perception of the stereoscopic image as being a truthful representation of reality (i.e. perceptual realism). The stereoscopic image may present different types of distortion which make it less natural. For example, stereoscopic objects are sometimes perceived as unnaturally large or small (puppet theatre effect), or they appear unnaturally thin (cardboard effect).
3.2.5 Reference: the source sequence. This is the highest quality version available of the audio, video, or audiovisual sequence.
3.2.6Sense of presence refers to “the subjective experience of being in one place or environment even when one is situated in another.”
3.2.7 Single Stimulus: a quality rating method where the subject is presented with one stimulus and then rates each stimulus in isolation (e.g., a viewer watches one video sequence, and then rates it).
3.2.8 Visual Comfort can be related to multi-symptoms, e.g. eye strain, dry eyes, double vision. Moreover, variation of Visual Comfort can be perceived as the sensation of visual impairment as well as the sense of vision difficulties when moving the fixation point from one area of the image to another area (due to the decoupling of accommodation and convergence). [editor’s note: definition and term will be discussed further and perhaps modified.]
the perceived quality of the video provided by the system. This is a main determinant of the performance of any video system. 3.2.10 Visual discomfort refers to the subjective sensation of discomfort that can be associated with the viewing of stereoscopic images. This issue is usually referred to as visual comfort. [editor’s note: this definition may need to be changed or deleted. Compare with ‘visual comfort’.]
3.2.11Visual Experience is defined as the overall quality of experience of the images in terms of immersion, perceived image quality as well as depth rendering considering the shape and the dimension of objects. [editor’s note: definition and term will be discussed further and perhaps modified.]