It is a sad fact that many lenses arrive for review in something of a sorry state, with items missing from the box or generally looking the worse for wear. Im now starting to wonder whether this might affect the results of their tests...
The centre of this flower is nothing short of bitingly sharp in the original image (the web version doesn’t do it justice). Unsurprisingly, the lens was not only carefully used but had also been carefully looked after when not in use! Photograph (c) Jon Tarrant.
The current issue of What Digital Camera (February 2010) contains my review of six fast 50mm prime lenses, not one of which has scored under 90%. Next month’s lenses do not fall into a neat group and have therefore been tested separately but again none scores less than 90%. This sudden bout of high scores took me by surprise until I thought about it more carefully.
The primes arrived in lovely condition and I’m fairly certain I must have been the first person to test some of them. The lenses wereneatly bagged, with lens caps in place and the documentation so crisp that it had clearly never been unfolded. Examination of the front and rear elements revealed no marks of any sort and the focussing mechanism was smooth in every case.
I feel it was no surprise that the lenses performed well, not only because of the highly-evolved nature of 50mm primes but also because of their condition. Next month’s short-zoom collection arrived in a similarly pristine condition and performed similarly well. Coincidence? I don’t think so.
There is also the other side to this coin; some lenses arrive without caps, sometimes even without boxes and often with no documentation. Handing a lens to a courier, clothed in bubble-wrap with its exterior elements exposed to the plastic is probably stacking the odds against a good performance in critical MTF tests at the other end of its journey.
You might be forgiven for thinking that such careless could only come from another publication that had been asked to ship a lens to one of its competitors at short notice but in fact I’ve had lenses come direct from UK distributors packed as described. I’ve also had what look suspiciously like demonstration lenses or salesmen’s samples, either of which could well get knocked about in use.
Perhaps it is unreasonable to expect to get a pristine lens for testing every time, especially in the case of lenses that have been around for a while, but evidence suggests it is equally unreasonable to expect well-used lenses to perform as well as brand new ones. And this, of course, complicates matters because the scores given to different lenses in a group test may, in part at least, reflect the condition of the lenses as much as their innate optical characteristics. It may also explain why duplicate tests of the same lens on different occasions may give different results.
With all of this in mind I strongly urge manufacturers to bear in mind the likely affect that condition may have on the reported scores. For my part, I hope to remember to comment on lens condition in future so that you have a better idea about whether the scores are representative of a brand new lens or one that is slightly battle-scarred.