First off, we need some kind of quality measure. Here we decided to take the widely used PSNR (Peak signal-to-noise ratio) which is easy to compute and is also easy for us to optimize our software for.
On the fly
While developing and testing the software, we work with a hardware quality measurement system. This microcontroller driven system allows real time analysis. However, the problem with this system is that the quality result is not on PSNR. So for the final tests we need to use a public known test environment with an established quality measure.
Environment
We use the well known avisynth and virtual dub to get the quality. Both of them are freely available, so everyone can validate our results!
1. First we need to copy the movie with DVD-Ranger and DVDShrink.
2. Then we merge the VOB files to one big VOB file.
3. In order to make sure that no frames are skipped by the MPEG2 decoder during our quality computations, we index the VOB files with DGIndex (also freely available).
4. If necessary we trim the movie at the end or at the beginning in order to make sure that each frame in our movie corresponds to the same frame in the reference movie.
5. We load our avisynth script with the compare functionality into Virtual Dub and run a Video Analysis Pass to obtain the results.
Please take a look at the pictures below for a visual comparison of the results.
Quality Test Movie No .1 This movie is known for giving recompression engines a hard time generating quality output. The comparison picture shows how classical engines break down on dark scenes because, due to their simplicity they treat dark scenes the same as the lighter ones, just look how the terrible the person and its cloths look with DVDShrink and how much detail our Physxtech engine offers you.
Quality Test Movie No .2 Even those scenes with enough light are coded with more details than usual by DVD-Ranger. Look at the trees and bushes in the distance, with the Physxtech engine we retain much more details than DVDShrink even with its high quality adaptive error compensation enabled, or any other DVD copying solution.
Quality Test Movie No.1
Results DVD-Ranger (with Physxtech)
Results DVDShrink(with high quality adaptive error compensation)
Minimum
Average
Maximum
Mean Absolute Deviation:
0.0000
0.5124
1.0297
Mean Deviation:
-0.1199
+0.0180
+0.2466
PSNR:
42.2329
48.2072
107.3187
Overall PSNR:
47.6376
Minimum
Average
Maximum
Mean Absolute Deviation:
0.0000
0.4864
2.304
Mean Deviation:
-0.1187
+0.0106
+0.1430
PSNR:
35.1349
48.5391
107.3187
Overall PSNR:
47.4292
Quality Test Movie No .2
Results DVD-Ranger (with Physxtech)
Results DVDShrink (with high quality adaptive error compensation)
Minimum
Average
Maximum
Mean Absolute Deviation:
0.0000
0.5632
0.9844
Mean Deviation:
-0.1685
+0.0572
+0.2890
PSNR:
44.2883
48.7991
106.5268
Overall PSNR:
48.2848
Minimum
Average
Maximum
Mean Absolute Deviation:
0.0000
0.5341
2.2226
Mean Deviation:
-0.1508
+0.0401
+0.3318
PSNR:
36.4087
48.7987
106.5268
Overall PSNR:
48.1798
Quality Test Movie No.2 Compare with others
Results AnyDVD & CloneDVD
Minimum
Average
Maximum
PSNR:
34.20
47.01
106.53
Overall PSNR:
46.02
Results AnyDVD & DVD2One
Minimum
Average
Maximum
PSNR:
32.13
46.61
106.53
Overall PSNR:
45.14
Results AnyDVD & DVDFab Platinum
Minimum
Average
Maximum
PSNR:
34.83
47.17
106.53
Overall PSNR:
46.23
Interpretation of the quality results
1) First of all take a look at the overall Peak Signal to Noise Ratio (PSNR). The higher the PSNR the better the quality of the resultant copy. We have used two quite large films filling their whole DVD, so as to stress the engines at maximum, quality wise. - As you can see we are consequently the winners in terms of high PSNR. And bear in mind that we have used the renowned high quality adaptive error compensation on our direct competitor in quality, DVDShrink. So our Physxtech engine does not need to hide from real comparisons with real numbers. We have also included the score of other competing applications for your reference.
2) Also take a look at the minimum (per frame) PSNR. This reflects the worst quality you will encounter in the recompressed movies and its the worst quality which strikes your eye and remains in mind as "the" quality of the copy. Here is where our engine really shines, we are about 7-8dB better than DVDShrink - that is 2.2 to 2.5 times better quality in the worst compressed frames. This result proves that our engine is way better able to capture the highly varying bit-budget needs for each scene and therefore does not deprieve bit-hungry, action-filled scenes from their quality by introducing blocking artefacts nor does it overcompress static scenes.
3) The Mean Absolute Deviation (MAD) is just another kind of measure for quality which is superseded by the PSNR. The maximum (per frame) MAD is therefore another measure for the worst quality encountered in the recompressed copy and verifies the minimum (per frame) PSNR result in that DVDShrink has about 2 times higher deviation from the uncompressed reference than our DVD Ranger.
4) The maximum (per frame) PSNR is always the same with any software as every movie usually has some black frames which cannot be further compressed and so have to be taken over 1:1 thus resulting in an infinite PSNR (the PSNR shown of above 100dB is due to limitations in the comparing program).
5) The Mean Deviation is a kind of measure for the bias to higher or lower luminosity values (lighter or darker frames). A completely unbiased compressor would not have any drastical impact on the final quality. Still we have included this benchmark in our quality test just to make sure we do not introduce any color tint or contrast/luminance shifts in the resulting movie.
Download the #1 DVD software utility from DVD-Ranger. Copy any DVD to your iPod, PSP, Zune, etc. DVD-Ranger is an all in one DVD and Blu-Ray burning software.
Copyright 2008-2010 Pixbyte Development SL. All Rights Reserved.
