| This paper explores the challenges associated | | | | algorithms perform a detailed comparison of |
| with assessing video quality. Assessing video | | | | the input and output video sequence. This is |
| quality ultimately depends on the customer's | | | | a computationally intensive process, which |
| reaction on their new TV. Many factors affect | | | | involves per-pixel processing, and temporal, |
| the video before it gets to the TV: | | | | spatial alignment of the input and output |
| compression, image processing, scaling, | | | | streams. Full reference algorithms can |
| decoding, transmission, etc. To isolate their | | | | achieve good levels of correlation with |
| video algorithm, companies perform | | | | subjective test data. Having the reference |
| verification using: subjective and objective | | | | video sequence available is only possible for |
| video assessment techniques. While it would | | | | certain applications: for example in lab |
| be far cheaper, to perform only objective | | | | testing, pre-deployment test or |
| test, nothing beats the human eye. Thus, | | | | troubleshooting. |
| Objective video quality measurements and | | | | |
| Subjective video quality assessment are | | | | One of the earliest full reference algorithms |
| complementary rather than interchangeable. | | | | is PSNR (Peak Signal to Noise Ratio), which |
| Subjective assessment is appropriate for | | | | is literally a measurement of the mean error |
| research related purposes; objective | | | | between input and output as a ratio of the |
| measurements are required for equipment | | | | peak signal level, expressed in dB. A typical |
| specifications and day-to-day system | | | | "good" PSNR is around 30dB and it is |
| performance measurement and monitoring. | | | | generally accepted that PSNR values of less |
| | | | than 18dB are unacceptable. PSNR is the most |
| Problem Statement | | | | widely used technique for image and video |
| | | | quality measurement. |
| When dealing with equipment to process TV & | | | | |
| video transmissions, every design depends on | | | | A wide range of full reference algorithms |
| accurate, repeatable measurements. A complex | | | | have been developed, including MPQM (Moving |
| relationship between objective parameter | | | | Pictures Quality Metric - 1996) full |
| measurements and subjective video quality | | | | reference algorithm from EPFL in Switzerland, |
| exists. The goal is to achieve an objective | | | | the US Government NTIA ITS lab's VQM (Video |
| metric; that is an automated measurement. All | | | | Quality Metric - 1999), and CVQE (Continuous |
| of the equipment must be tested from video | | | | Video Quality Evaluation - 2004) which is |
| processors, compression units, transmission | | | | more suited for low bit rate video. ITU-T |
| gear, set-top boxes, and displays. The | | | | J.144 does not actually specify a single |
| evaluation of the video quality and | | | | algorithm but "provides guidelines on the |
| ultimately, the customer's reaction to the | | | | selection of appropriate" techniques. J.144 |
| picture shown on their new HDTV drives the | | | | does contain descriptions and test results |
| business. | | | | for four full reference algorithms. The VQM |
| | | | algorithm from the US Government's NTIA ITS |
| Previous Options | | | | lab achieved slightly better performance than |
| | | | the other algorithms listed. |
| For years, traditional techniques that looked | | | | |
| at color, brightness, contrast, etc. were | | | | Video Clarity Solution |
| effective. However, the advent of compressed | | | | |
| digital video transmission has complicated | | | | The ClearView Video Analysis system |
| the process of evaluating video sequences, | | | | simplifies the work flow, by combining the |
| with respect to perceived picture quality. | | | | video server and capture device into one |
| During compression, a certain amount of the | | | | unit. By doing this, the original source and |
| original content is knowingly discarded. | | | | processed video sequences can be displayed - |
| Visible impairments such as "blockiness" and | | | | side-by-side, mirrored, or seamless split - |
| Gaussian noise are by-products. Traditional | | | | on the same display. Further the operator can |
| measurement techniques that look at color, | | | | play any video sequence in any order, at any |
| brightness, contrast, etc. are no longer | | | | speed, for any duration with zoom, pan, jog, |
| effective. | | | | and shuttle capabilities. |
| | | | |
| Objective Measurement Status | | | | ClearView applies various objective metrics |
| | | | to the video sequences, generates graphs, and |
| After analyzing the subjective results, | | | | calculates an objective score. While |
| considerable work has been done to come up | | | | development of more advanced algorithms is |
| with a quantifiable, repeatable measurement | | | | ongoing, we have built a hybrid system, which |
| which is not dependent on the video sequence. | | | | offers subjective viewing modes; while |
| To date, objective measurements have not | | | | calculating objective measurements. ClearView |
| proven to estimate the user's opinion. To | | | | can easily be programmed to display video |
| introduce and qualify new algorithms, Video | | | | sequences for the expert viewers; while |
| Quality Experts Group (VQEG) was formed in | | | | recording the objective metric scores along |
| 1997, and generally acts in cooperation with | | | | with the MOS. While the MOS cannot be |
| ITU. VQEG has conducted two phases of | | | | repeated, the objective metric can be easily |
| testing; in the first phase ten algorithms | | | | and readily. |
| were tested, and the conclusion reached was | | | | |
| that all of them were statistically | | | | Benefits |
| equivalent. A second phase of testing, | | | | |
| conducted several years later, involved a | | | | * Repeatable tests, quantitative results, and |
| smaller set of algorithms, more controlled | | | | a streamlined setup. |
| video sequences, and a better defined test | | | | |
| environment. The result of the second phases | | | | * Capable of analyzing 2, 1080P video |
| warranted the recent ITU-T Recommendation | | | | sequences in real-time |
| J.144. | | | | |
| | | | * ClearView does not alter the original video |
| Three basic types of objective video | | | | sequences - video sequences are processed |
| assessment metrics exist: | | | | completely uncompressed. |
| | | | |
| * Full Reference - A method that conducts a | | | | * Multiple viewing modes are presented on 1 |
| comparison of video source to resultant. | | | | display - no need to calibrate 2 separate |
| | | | Television displays to compare video |
| * Reduced Reference - A method that conducts | | | | sequences. |
| a comparison of a reduced video source to a | | | | |
| full result. | | | | * Objective Metrics are included as add-ons |
| | | | as the technology advances. |
| * No Reference - A method when there is no | | | | |
| reference. | | | | Implementation |
| | | | |
| The three methods have different | | | | ClearView takes advantage of the |
| applications, and they provide different | | | | high-reliability of today's off-the-shelf |
| degrees of measurement accuracy, expressed in | | | | computer platforms. This ensures that |
| terms of correlation with subjective | | | | products are made with the latest and |
| assessment results. | | | | greatest hardware available, while at the |
| | | | same time avoiding the high cost of custom |
| The work to date has centered on full | | | | designs. ClearView is packages in a 4U, 20" |
| reference algorithms. Full reference | | | | deep Industrial Chassis. |