| This paper explores the challenges associated with | | | | algorithms. Full reference algorithms perform a detailed |
| assessing video quality. Assessing video quality | | | | comparison of the input and output video sequence. |
| ultimately depends on the customer's reaction on their | | | | This is a computationally intensive process, which |
| new TV. Many factors affect the video before it gets | | | | involves per-pixel processing, and temporal, spatial |
| to the TV: compression, image processing, scaling, | | | | alignment of the input and output streams. Full |
| decoding, transmission, etc. To isolate their video | | | | reference algorithms can achieve good levels of |
| algorithm, companies perform verification using: | | | | correlation with subjective test data. Having the |
| subjective and objective video assessment techniques. | | | | reference video sequence available is only possible for |
| While it would be far cheaper, to perform only | | | | certain applications: for example in lab testing, |
| objective test, nothing beats the human eye. Thus, | | | | pre-deployment test or troubleshooting. |
| Objective video quality measurements and Subjective | | | | One of the earliest full reference algorithms is PSNR |
| video quality assessment are complementary rather | | | | (Peak Signal to Noise Ratio), which is literally a |
| than interchangeable. Subjective assessment is | | | | measurement of the mean error between input and |
| appropriate for research related purposes; objective | | | | output as a ratio of the peak signal level, expressed in |
| measurements are required for equipment | | | | dB. A typical "good" PSNR is around 30dB and it is |
| specifications and day-to-day system performance | | | | generally accepted that PSNR values of less than |
| measurement and monitoring. | | | | 18dB are unacceptable. PSNR is the most widely used |
| Problem Statement | | | | technique for image and video quality measurement. |
| When dealing with equipment to process TV & video | | | | A wide range of full reference algorithms have been |
| transmissions, every design depends on accurate, | | | | developed, including MPQM (Moving Pictures Quality |
| repeatable measurements. A complex relationship | | | | Metric - 1996) full reference algorithm from EPFL in |
| between objective parameter measurements and | | | | Switzerland, the US Government NTIA ITS lab's VQM |
| subjective video quality exists. The goal is to achieve | | | | (Video Quality Metric - 1999), and CVQE (Continuous |
| an objective metric; that is an automated | | | | Video Quality Evaluation - 2004) which is more suited |
| measurement. All of the equipment must be tested | | | | for low bit rate video. ITU-T J.144 does not actually |
| from video processors, compression units, transmission | | | | specify a single algorithm but "provides guidelines on |
| gear, set-top boxes, and displays. The evaluation of | | | | the selection of appropriate" techniques. J.144 does |
| the video quality and ultimately, the customer's reaction | | | | contain descriptions and test results for four full |
| to the picture shown on their new HDTV drives the | | | | reference algorithms. The VQM algorithm from the US |
| business. | | | | Government's NTIA ITS lab achieved slightly better |
| Previous Options | | | | performance than the other algorithms listed. |
| For years, traditional techniques that looked at color, | | | | Video Clarity Solution |
| brightness, contrast, etc. were effective. However, the | | | | The ClearView Video Analysis system simplifies the |
| advent of compressed digital video transmission has | | | | work flow, by combining the video server and capture |
| complicated the process of evaluating video | | | | device into one unit. By doing this, the original source |
| sequences, with respect to perceived picture quality. | | | | and processed video sequences can be displayed - |
| During compression, a certain amount of the original | | | | side-by-side, mirrored, or seamless split - on the same |
| content is knowingly discarded. Visible impairments | | | | display. Further the operator can play any video |
| such as "blockiness" and Gaussian noise are | | | | sequence in any order, at any speed, for any duration |
| by-products. Traditional measurement techniques that | | | | with zoom, pan, jog, and shuttle capabilities. |
| look at color, brightness, contrast, etc. are no longer | | | | ClearView applies various objective metrics to the |
| effective. | | | | video sequences, generates graphs, and calculates an |
| Objective Measurement Status | | | | objective score. While development of more |
| After analyzing the subjective results, considerable | | | | advanced algorithms is ongoing, we have built a hybrid |
| work has been done to come up with a quantifiable, | | | | system, which offers subjective viewing modes; while |
| repeatable measurement which is not dependent on | | | | calculating objective measurements. ClearView can |
| the video sequence. To date, objective measurements | | | | easily be programmed to display video sequences for |
| have not proven to estimate the user's opinion. To | | | | the expert viewers; while recording the objective |
| introduce and qualify new algorithms, Video Quality | | | | metric scores along with the MOS. While the MOS |
| Experts Group (VQEG) was formed in 1997, and | | | | cannot be repeated, the objective metric can be easily |
| generally acts in cooperation with ITU. VQEG has | | | | and readily. |
| conducted two phases of testing; in the first phase ten | | | | Benefits |
| algorithms were tested, and the conclusion reached | | | | * Repeatable tests, quantitative results, and a |
| was that all of them were statistically equivalent. A | | | | streamlined setup. |
| second phase of testing, conducted several years | | | | * Capable of analyzing 2, 1080P video sequences in |
| later, involved a smaller set of algorithms, more | | | | real-time |
| controlled video sequences, and a better defined test | | | | * ClearView does not alter the original video |
| environment. The result of the second phases | | | | sequences - video sequences are processed |
| warranted the recent ITU-T Recommendation J.144. | | | | completely uncompressed. |
| Three basic types of objective video assessment | | | | * Multiple viewing modes are presented on 1 display - |
| metrics exist: | | | | no need to calibrate 2 separate Television displays to |
| * Full Reference - A method that conducts a | | | | compare video sequences. |
| comparison of video source to resultant. | | | | * Objective Metrics are included as add-ons as the |
| * Reduced Reference - A method that conducts a | | | | technology advances. |
| comparison of a reduced video source to a full result. | | | | Implementation |
| * No Reference - A method when there is no | | | | ClearView takes advantage of the high-reliability of |
| reference. | | | | today's off-the-shelf computer platforms. This ensures |
| The three methods have different applications, and | | | | that products are made with the latest and greatest |
| they provide different degrees of measurement | | | | hardware available, while at the same time avoiding |
| accuracy, expressed in terms of correlation with | | | | the high cost of custom designs. ClearView is |
| subjective assessment results. | | | | packages in a 4U, 20" deep Industrial Chassis. |
| The work to date has centered on full reference | | | | |