Camera Image Quality: why DPReview may be sometimes wrong - The Visual Experience
Choose your Dream Workshop Check out our upcoming workshops calendar

Just between us: all photographers are crazy for cameras and lenses performances. Every time a new camera hit the market flocks of people flow on the Internet looking for files, pixel-peeping at 200% etc, producing tons of bytes on specialized forums. Even those who normally say they don’t care about gear at the end use some piece of equipment and had, maybe even only once in their lives, an interest for image producing tools and for the maximum quality they can reach using them.

DPReview paved the way for “scientific” comparison of camera performances, guiding passionate and professionals in understanding “what is worth to buy” and what is not. Similarly there are tons photographers on free or pay websites testing cameras and giving indications about performances, with a plethora of suggestions, hints, etc.

Here at “The Visual Experience” we did the same for some cameras, so we’re no exception.

Back in the argentique world, the image quality (note: pls stick to image quality only, not to other considerations) of a camera was mostly linked to the lens used. Bigger formats, better lenses, larger film surface, bigger prints. Sure: different films with different characteristics and “ASA” were also responsible for image quality. Films were interchangeble, so we had the opportunity to decide on the final image quality (on purpose here I will not talk about printing processes). In the digital world the lens is only a part of the equation since the Sensor is, as we all know, very important as well. Full frame, APSC, Micro4/3, “Medium Format” the sensor is the new recording media and, being not anymore a film, something we buy and have to live with for the camera lifetime. Sure, a new firmware release may better some feature, but the sensor intrisic characteristics remain the same. Bayer mosaic for most (CCD or CMOS), Xtrans or Foveon, and new comers directly from the research world.

The product of these sensors is, today on most of the serious cameras, a RAW file. This is “digital negative” for all serious photographers, giving all the advantages we know in terms of dynamic range, post-shot white balance etc.

But there is another part of the equation, which is mostly mathematic: the algorithm for reading (de-mosaicing) a RAW file.

The observation is pretty simple: DPReview uses the industry standard ACR (Adobe Camera RAW) for interpreting the results of a camera RAW ouput with, generally, this workflow:

  1. Load RAW file into Adobe Camera RAW (Auto mode disabled)
  2. Set Sharpness to zero (all other settings default)
  3. Open file to Photoshop
  4. Apply a Unsharp mask: 100%, Radius 0.6, Threshold 0
  5. Save as a TIFF (for cropping) and as a JPEG quality 11 for download

So ACR is the “meter” used to compare cameras for what concerns RAW. You can jump to the “Image Quality” of each review and use the tools to compare RAW performances. Gladly you can also download the original RAW file and play with them yourself, reproducing the same workflow… So you may discover that maybe the camera you’re looking at works better then they say. What if the “meter” is not anymore the right “meter”? What if we measure things with a wrong meter, not using the one that sits at the Bureau International des Poids et Mesures?

These are our findings with the output of the FUJIFILM X100s (shot by DPReview) processed with IRIDIENT DEVELOPER 2.1.1 and ACR 8.1 following DPReview workflow, so all set to default, no sharpening in the RAW processors, USM as in point 4. We left to you the conclusions. All we can say is: before choosing a camera do your own tests. And to DPreview: please start taking into account different RAW processors or, at least, make some claim ¬†that the RAW results people are inspecting may be not representative of the REAL POTENTIAL a camera may have (since the meter you’re using is, somehow, not the best).

PS=Please not that no further post processing was done in the file, just the basic DPreview workflow is used. Some Chromatic Aberration in the IRIDIENT output could be mitigated working a bit on the file but the idea here was to use default setting for both processor with no sharpening and further elaborations.

PPS=We are NOT LINKED to Brian @ Iridient Digital!

cameras-image-quality-why-dpreview-may-be-sometimes-wrong

 

 

 

 

The Visual Experience



{ 15 comments… read them below or add one }

CameraDude July 3, 2013 at 17:32

That's a big difference. You should email these results to Adobe's raw processing team if possible. They should see that Iridient is the bar by which they are measured for Fuji Xtrans raw processing now. They could learn something from Brian and probably should hire him as a consultant.

Reply

Joohan July 3, 2013 at 19:47

As far as I now all the RAW processing software apply some sort of adjustments. These photos seems to have more artefacts and look jaggier so I would presume the same results could be achieved with a bit of settings in ACR. It is always hard to compare these things. Use the software you are most confident in and happy with. Is the result good for you then all is well.

Reply

tve July 3, 2013 at 20:47

Hi Joohan, thank you for the comment. Sure Acr can do better then those crops but here the point was to follow dpreview worflow on both software and compare the results. If this is the workflow used to compare Raw output looks like they're using something that penalizes xtrans output too much.

Reply

Someone July 3, 2013 at 23:40

You should notice that every RAW processor applies sharpening and denoising. The amount is at the processor's own discretion. In the Iridient rendering, the perceived higher details is just the higher default sharpening. For example, the text of the Bailey bottle label shows halos, which are results of sharpening, rather than real details.

Reply

tve July 4, 2013 at 02:08

Hi,
have a look here http://www.thevisualexperience.org/web/processing… for additional inputs about details and sharpening.

Reply

Someone July 5, 2013 at 01:03

Granted, Iridient is better at sharpening the image than Adobe Camera Raw. So, Iridient is better at lying?

Your post implies that Adobe Camera Raw does not show the full potential of the Xtrans-Sensor, like Adobe Camera Raw is not able to show some details which the chip has physically recorded. I merely stated that your comparison method is possible flawed, as you in fact compare two different sharpening techniques and settings. Unlike you allure Iridient's default does include sharpening. If I misunderstood that part. I am sorry.

If you turn off sharpening in Iridient, you will get a soft rendering. Its is probably the exact result produced by Apple raw converter. In fact, if you open the image in Preview.app, you get the same soft image. So Apple does not provide much more details than Adobe. Just as an aside, the sharpening in Preview.app produces better results than in Iridient. It has less chromatic noise. Probably Apple raw converter has a greater impact in the "better image quality" than the sharpening technique in Iridient anyway.

I am surely not thinking Adobe Camera Raw is great. In fact I use Aperture. I want only point out that you made a valid point but with some unsound data.

Reply

tve July 5, 2013 at 06:15

Thank you for your comment. Two important points: sharpening is turned off on both Raw processors. We followed Dpreview workflow only so unsharp mask is only applied in Photoshop with the settings highlighted in the post.
So, for acr and Iridient all default, sharpening Off, creating tiff, apply USM in Photoshop.
Regarding the preview: what you see in preview is a low res jpg version embedded in the raw file. So no surprises the previews are generally soft. Have a look here http://www.fujirumors.com/raw-for-jpeg-shooters/.

Reply

jaduffy007 July 30, 2013 at 21:44

I tried that and you are incorrect. Iridient is doing a LOT of things far better than ACR…and noticeably, but less dramatically better than Aperture.

Reply

JJJ July 4, 2013 at 14:35

Excellent comparison but also valid points made concerning the different sharpening and denoising by different RAW processors.

Perhaps to make the point stronger Visual Experience can include the same comparison for a standard Bayer filter camera.

Reply

Dr. Nicolas Rao July 30, 2013 at 17:49

Is this forum concerning only the Fuji RAW files or RAW files from all camera makers.

Reply

tve July 31, 2013 at 07:07

Specifically the XTrans RAWs. To some extent it could be extended also to other camera makers.

Reply

matus July 30, 2013 at 19:20

Interesting comparison and valid comments. However looking the at the provided examples there is, IMO, very little the Irident has to offer over Acr. The image has strong contrast (very different sharpening approach) what makes it appear sharper – in particular this brings up low-contrast detail which may seem to be missing in Acr. However there are many false (I suppose) chromatic halos around fine detail edges which would suggest that Irident can not handle properly the X-trans raw data. Yes – the Acr output looks 'too smooth' but it is not soft – the fine detail is all there.

So – if the point was to show that there are possibly better options than Acr for X-trans RAW files – why not to try to get most out of both converters and see what comes out? I would not be surprised if the differences would be rather minimal.

Reply

tve July 31, 2013 at 07:13

Hi, thank you for commenting.
Please note that the procedure adopted in this comparison is the same that DPReview adopts with ACR. In this case, true, there are some chromatic haloes. But, if you simply "enable transverse chromatic aberration" the haloes disappear.
A better processing of a real life file can be found here:
http://www.thevisualexperience.org/web/processing

Reply

Freeware Supporter January 22, 2014 at 13:31

Yes, most reviewers aren't clear in their writing. Last week I went through a review on TopTenReviews where they have declared iPage hosting as the 2nd top ranked hosting for the year 2013. That's fair at all. I'm using iPage's services and they are the worst. Their customers aren't happy with them.

Similarly, DPReview might haven't put truth about the actual performance of FUJIFILM X100s.

Reply

Doorbot April 15, 2014 at 00:08

Excellent comparison! I like your review more than DPReview! Keep up the good work!! ;D

Reply

Leave a Comment